2024-09-18T00:04:45.861 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-18T00:04:45.909 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T00:04:45.986 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909638 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} email: ceph-qa@ceph.com first_in_suite: false job_id: '7909638' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: client: rbd cache: true rbd cache policy: writearound global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zlib bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b0d356b98e08f6bc59ef15652f0d0e4298fc83c7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 1983 sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f sleep_before_teardown: 0 subset: 79/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b0d356b98e08f6bc59ef15652f0d0e4298fc83c7 targets: smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBKiv2JxQqEnyhl6E8Nw6NRxPA7jGKrfkLs4AJiPKwJHrMpkaiuj88zbbM5o9TDDyuBM3ab542ZlCxN9s5TubZE= smithi114.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHuHns1Gtq6K5UGcKyGbCRcRtEpwx3KdQU/ipZzHlvcWBSInPDAL69nQiroDDcRWAUT8g+U1CkHb6AhwRih+A7M= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-18T00:04:45.986 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa; will attempt to use it 2024-09-18T00:04:45.987 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa/tasks 2024-09-18T00:04:45.987 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-18T00:04:45.989 INFO:teuthology.task.internal:Checking packages... 2024-09-18T00:04:46.008 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'c2ddf9721ef254645201cdbabd9c8f49c30a586f' 2024-09-18T00:04:46.008 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-18T00:04:46.008 INFO:teuthology.packaging:ref: None 2024-09-18T00:04:46.008 INFO:teuthology.packaging:tag: None 2024-09-18T00:04:46.008 INFO:teuthology.packaging:branch: squid 2024-09-18T00:04:46.008 INFO:teuthology.packaging:sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T00:04:46.008 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-09-18T00:04:46.215 INFO:teuthology.task.internal:Found packages for ceph version 19.2.0-1jammy 2024-09-18T00:04:46.217 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-18T00:04:46.228 INFO:teuthology.task.internal:no buildpackages task found 2024-09-18T00:04:46.228 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-18T00:04:46.262 INFO:teuthology.task.internal:Saving configuration 2024-09-18T00:04:46.274 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-18T00:04:46.311 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-18T00:04:46.332 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909638', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 23:59:09.970027', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBh2xx/JExieQ+YVEL1krl+gQWDjdqwe0+MoWtgXWUUA'} 2024-09-18T00:04:46.350 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909638', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 23:59:09.970864', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIA/PX8NLmnsHvdZX3+LIDjNV1aZew/1+gNuwWl8Y66g4'} 2024-09-18T00:04:46.350 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-18T00:04:46.380 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-18T00:04:46.380 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-18T00:04:46.380 INFO:teuthology.run_tasks:Running task console_log... 2024-09-18T00:04:46.461 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f55a7353520>, signals=[15]) 2024-09-18T00:04:46.462 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-18T00:04:46.465 INFO:teuthology.task.internal:Opening connections... 2024-09-18T00:04:46.465 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-09-18T00:04:46.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:04:46.557 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi114.front.sepia.ceph.com 2024-09-18T00:04:46.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:04:46.643 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-18T00:04:46.669 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-09-18T00:04:46.674 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-09-18T00:04:46.674 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-09-18T00:04:46.720 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-18T00:04:46.720 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Ubuntu" 2024-09-18T00:04:46.720 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="22.04" 2024-09-18T00:04:46.720 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-18T00:04:46.720 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_CODENAME=jammy 2024-09-18T00:04:46.721 INFO:teuthology.orchestra.run.smithi097.stdout:ID=ubuntu 2024-09-18T00:04:46.721 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE=debian 2024-09-18T00:04:46.721 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-18T00:04:46.721 INFO:teuthology.orchestra.run.smithi097.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-18T00:04:46.721 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-18T00:04:46.721 INFO:teuthology.orchestra.run.smithi097.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-18T00:04:46.721 INFO:teuthology.orchestra.run.smithi097.stdout:UBUNTU_CODENAME=jammy 2024-09-18T00:04:46.722 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-09-18T00:04:46.739 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2024-09-18T00:04:46.745 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2024-09-18T00:04:46.745 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2024-09-18T00:04:46.793 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-18T00:04:46.793 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="Ubuntu" 2024-09-18T00:04:46.793 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="22.04" 2024-09-18T00:04:46.793 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-18T00:04:46.793 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_CODENAME=jammy 2024-09-18T00:04:46.793 INFO:teuthology.orchestra.run.smithi114.stdout:ID=ubuntu 2024-09-18T00:04:46.793 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE=debian 2024-09-18T00:04:46.793 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-18T00:04:46.793 INFO:teuthology.orchestra.run.smithi114.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-18T00:04:46.793 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-18T00:04:46.794 INFO:teuthology.orchestra.run.smithi114.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-18T00:04:46.794 INFO:teuthology.orchestra.run.smithi114.stdout:UBUNTU_CODENAME=jammy 2024-09-18T00:04:46.794 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2024-09-18T00:04:46.814 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-18T00:04:46.822 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-18T00:04:46.840 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-18T00:04:46.840 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-09-18T00:04:46.843 DEBUG:teuthology.orchestra.run.smithi114:> test '!' -e /home/ubuntu/cephtest 2024-09-18T00:04:46.847 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-18T00:04:46.869 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-18T00:04:46.869 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-09-18T00:04:46.890 DEBUG:teuthology.orchestra.run.smithi114:> test -z $(ls -A /var/lib/ceph) 2024-09-18T00:04:46.905 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-18T00:04:46.955 INFO:teuthology.run_tasks:Running task kernel... 2024-09-18T00:04:46.973 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-18T00:04:46.973 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-18T00:04:46.974 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T00:04:46.974 DEBUG:teuthology.orchestra.run.smithi114:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T00:04:46.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:04:46.979 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-09-18T00:04:46.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:04:46.980 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-09-18T00:04:47.024 INFO:teuthology.orchestra.run.smithi097.stdout:5.15.0-56-generic 2024-09-18T00:04:47.025 INFO:teuthology.task.kernel:Running kernel on smithi097: 5.15.0-56-generic 2024-09-18T00:04:47.025 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get clean 2024-09-18T00:04:47.026 INFO:teuthology.orchestra.run.smithi114.stdout:5.15.0-56-generic 2024-09-18T00:04:47.026 INFO:teuthology.task.kernel:Running kernel on smithi114: 5.15.0-56-generic 2024-09-18T00:04:47.026 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-get clean 2024-09-18T00:04:47.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get update 2024-09-18T00:04:47.162 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-get update 2024-09-18T00:04:47.303 INFO:teuthology.orchestra.run.smithi114.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-18T00:04:47.314 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-18T00:04:47.314 INFO:teuthology.orchestra.run.smithi097.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-18T00:04:47.314 INFO:teuthology.orchestra.run.smithi097.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-18T00:04:47.383 INFO:teuthology.orchestra.run.smithi097.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-18T00:04:47.421 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-18T00:04:47.502 INFO:teuthology.orchestra.run.smithi114.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-18T00:04:47.513 INFO:teuthology.orchestra.run.smithi114.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-18T00:04:47.557 INFO:teuthology.orchestra.run.smithi097.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-18T00:04:47.615 INFO:teuthology.orchestra.run.smithi114.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-18T00:04:47.620 INFO:teuthology.orchestra.run.smithi114.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-18T00:04:47.622 INFO:teuthology.orchestra.run.smithi114.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-18T00:04:47.622 INFO:teuthology.orchestra.run.smithi114.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-18T00:04:47.625 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-18T00:04:47.634 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,058 kB] 2024-09-18T00:04:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-18T00:04:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-18T00:04:47.648 INFO:teuthology.orchestra.run.smithi114.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-18T00:04:47.648 INFO:teuthology.orchestra.run.smithi114.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [903 kB] 2024-09-18T00:04:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [628 kB] 2024-09-18T00:04:47.660 INFO:teuthology.orchestra.run.smithi114.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-18T00:04:47.662 INFO:teuthology.orchestra.run.smithi114.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-18T00:04:47.662 INFO:teuthology.orchestra.run.smithi114.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-18T00:04:47.662 INFO:teuthology.orchestra.run.smithi114.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-18T00:04:47.663 INFO:teuthology.orchestra.run.smithi097.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-18T00:04:47.665 INFO:teuthology.orchestra.run.smithi097.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-18T00:04:47.665 INFO:teuthology.orchestra.run.smithi097.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-18T00:04:47.666 INFO:teuthology.orchestra.run.smithi097.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-18T00:04:47.699 INFO:teuthology.orchestra.run.smithi114.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-18T00:04:47.699 INFO:teuthology.orchestra.run.smithi114.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-18T00:04:47.731 INFO:teuthology.orchestra.run.smithi097.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-18T00:04:47.735 INFO:teuthology.orchestra.run.smithi097.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-18T00:04:47.735 INFO:teuthology.orchestra.run.smithi097.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [903 kB] 2024-09-18T00:04:47.746 INFO:teuthology.orchestra.run.smithi097.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [628 kB] 2024-09-18T00:04:47.747 INFO:teuthology.orchestra.run.smithi097.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-18T00:04:47.749 INFO:teuthology.orchestra.run.smithi097.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-18T00:04:47.749 INFO:teuthology.orchestra.run.smithi097.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-18T00:04:47.750 INFO:teuthology.orchestra.run.smithi097.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-18T00:04:47.750 INFO:teuthology.orchestra.run.smithi097.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-18T00:04:47.750 INFO:teuthology.orchestra.run.smithi097.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-18T00:04:47.751 INFO:teuthology.orchestra.run.smithi097.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [701 kB] 2024-09-18T00:04:47.758 INFO:teuthology.orchestra.run.smithi097.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [355 kB] 2024-09-18T00:04:47.761 INFO:teuthology.orchestra.run.smithi097.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-18T00:04:47.761 INFO:teuthology.orchestra.run.smithi097.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,495 kB] 2024-09-18T00:04:47.783 INFO:teuthology.orchestra.run.smithi097.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-18T00:04:47.783 INFO:teuthology.orchestra.run.smithi097.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [429 kB] 2024-09-18T00:04:47.787 INFO:teuthology.orchestra.run.smithi097.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-18T00:04:47.787 INFO:teuthology.orchestra.run.smithi097.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-18T00:04:47.797 INFO:teuthology.orchestra.run.smithi097.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-18T00:04:47.802 INFO:teuthology.orchestra.run.smithi097.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-18T00:04:47.805 INFO:teuthology.orchestra.run.smithi097.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-18T00:04:47.805 INFO:teuthology.orchestra.run.smithi097.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-18T00:04:47.805 INFO:teuthology.orchestra.run.smithi097.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-18T00:04:47.823 INFO:teuthology.orchestra.run.smithi097.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-18T00:04:47.824 INFO:teuthology.orchestra.run.smithi097.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-18T00:04:47.824 INFO:teuthology.orchestra.run.smithi097.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-18T00:04:47.824 INFO:teuthology.orchestra.run.smithi097.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-18T00:04:47.828 INFO:teuthology.orchestra.run.smithi097.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-18T00:04:47.829 INFO:teuthology.orchestra.run.smithi097.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-18T00:04:47.829 INFO:teuthology.orchestra.run.smithi097.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-18T00:04:47.829 INFO:teuthology.orchestra.run.smithi097.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-18T00:04:47.845 INFO:teuthology.orchestra.run.smithi097.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-18T00:04:47.845 INFO:teuthology.orchestra.run.smithi097.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-18T00:04:47.858 INFO:teuthology.orchestra.run.smithi114.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-18T00:04:47.997 INFO:teuthology.orchestra.run.smithi114.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,058 kB] 2024-09-18T00:04:48.317 INFO:teuthology.orchestra.run.smithi114.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [701 kB] 2024-09-18T00:04:48.351 INFO:teuthology.orchestra.run.smithi114.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [355 kB] 2024-09-18T00:04:48.366 INFO:teuthology.orchestra.run.smithi114.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-18T00:04:48.366 INFO:teuthology.orchestra.run.smithi114.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,495 kB] 2024-09-18T00:04:48.438 INFO:teuthology.orchestra.run.smithi114.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-18T00:04:48.439 INFO:teuthology.orchestra.run.smithi114.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [429 kB] 2024-09-18T00:04:48.454 INFO:teuthology.orchestra.run.smithi114.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-18T00:04:48.454 INFO:teuthology.orchestra.run.smithi114.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-18T00:04:48.488 INFO:teuthology.orchestra.run.smithi114.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-18T00:04:48.506 INFO:teuthology.orchestra.run.smithi114.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-18T00:04:48.511 INFO:teuthology.orchestra.run.smithi114.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-18T00:04:48.513 INFO:teuthology.orchestra.run.smithi114.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-18T00:04:48.513 INFO:teuthology.orchestra.run.smithi114.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-18T00:04:48.562 INFO:teuthology.orchestra.run.smithi114.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-18T00:04:48.563 INFO:teuthology.orchestra.run.smithi114.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-18T00:04:48.563 INFO:teuthology.orchestra.run.smithi114.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-18T00:04:48.564 INFO:teuthology.orchestra.run.smithi114.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-18T00:04:48.595 INFO:teuthology.orchestra.run.smithi114.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-18T00:04:48.596 INFO:teuthology.orchestra.run.smithi114.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-18T00:04:48.596 INFO:teuthology.orchestra.run.smithi114.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-18T00:04:48.597 INFO:teuthology.orchestra.run.smithi114.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-18T00:04:48.597 INFO:teuthology.orchestra.run.smithi114.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-18T00:04:48.643 INFO:teuthology.orchestra.run.smithi114.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-18T00:04:53.957 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 16.2 MB in 2s (7,460 kB/s) 2024-09-18T00:04:54.003 INFO:teuthology.orchestra.run.smithi114.stdout:Fetched 16.2 MB in 2s (7,506 kB/s) 2024-09-18T00:04:55.365 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2024-09-18T00:04:55.383 DEBUG:teuthology.orchestra.run.smithi114:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T00:04:55.437 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-09-18T00:04:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2024-09-18T00:04:55.456 DEBUG:teuthology.orchestra.run.smithi097:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T00:04:55.522 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-09-18T00:04:55.653 INFO:teuthology.orchestra.run.smithi114.stdout:Building dependency tree... 2024-09-18T00:04:55.654 INFO:teuthology.orchestra.run.smithi114.stdout:Reading state information... 2024-09-18T00:04:55.735 INFO:teuthology.orchestra.run.smithi097.stdout:Building dependency tree... 2024-09-18T00:04:55.735 INFO:teuthology.orchestra.run.smithi097.stdout:Reading state information... 2024-09-18T00:04:55.835 INFO:teuthology.orchestra.run.smithi114.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T00:04:55.835 INFO:teuthology.orchestra.run.smithi114.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-18T00:04:55.835 INFO:teuthology.orchestra.run.smithi114.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-18T00:04:55.836 INFO:teuthology.orchestra.run.smithi114.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-18T00:04:55.836 INFO:teuthology.orchestra.run.smithi114.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-18T00:04:55.836 INFO:teuthology.orchestra.run.smithi114.stdout: python2.7-minimal 2024-09-18T00:04:55.836 INFO:teuthology.orchestra.run.smithi114.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T00:04:55.837 INFO:teuthology.orchestra.run.smithi114.stdout:The following additional packages will be installed: 2024-09-18T00:04:55.837 INFO:teuthology.orchestra.run.smithi114.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-18T00:04:55.837 INFO:teuthology.orchestra.run.smithi114.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-18T00:04:55.838 INFO:teuthology.orchestra.run.smithi114.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-18T00:04:55.839 INFO:teuthology.orchestra.run.smithi114.stdout:Suggested packages: 2024-09-18T00:04:55.839 INFO:teuthology.orchestra.run.smithi114.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-18T00:04:55.839 INFO:teuthology.orchestra.run.smithi114.stdout:Recommended packages: 2024-09-18T00:04:55.839 INFO:teuthology.orchestra.run.smithi114.stdout: thermald 2024-09-18T00:04:55.887 INFO:teuthology.orchestra.run.smithi114.stdout:The following NEW packages will be installed: 2024-09-18T00:04:55.887 INFO:teuthology.orchestra.run.smithi114.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-18T00:04:55.887 INFO:teuthology.orchestra.run.smithi114.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-18T00:04:55.888 INFO:teuthology.orchestra.run.smithi114.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-18T00:04:55.889 INFO:teuthology.orchestra.run.smithi114.stdout:The following packages will be upgraded: 2024-09-18T00:04:55.889 INFO:teuthology.orchestra.run.smithi114.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-18T00:04:55.924 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T00:04:55.924 INFO:teuthology.orchestra.run.smithi097.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-18T00:04:55.924 INFO:teuthology.orchestra.run.smithi097.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-18T00:04:55.925 INFO:teuthology.orchestra.run.smithi097.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-18T00:04:55.925 INFO:teuthology.orchestra.run.smithi097.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-18T00:04:55.925 INFO:teuthology.orchestra.run.smithi097.stdout: python2.7-minimal 2024-09-18T00:04:55.925 INFO:teuthology.orchestra.run.smithi097.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T00:04:55.927 INFO:teuthology.orchestra.run.smithi097.stdout:The following additional packages will be installed: 2024-09-18T00:04:55.927 INFO:teuthology.orchestra.run.smithi097.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-18T00:04:55.927 INFO:teuthology.orchestra.run.smithi097.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-18T00:04:55.927 INFO:teuthology.orchestra.run.smithi097.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-18T00:04:55.929 INFO:teuthology.orchestra.run.smithi097.stdout:Suggested packages: 2024-09-18T00:04:55.929 INFO:teuthology.orchestra.run.smithi097.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-18T00:04:55.929 INFO:teuthology.orchestra.run.smithi097.stdout:Recommended packages: 2024-09-18T00:04:55.929 INFO:teuthology.orchestra.run.smithi097.stdout: thermald 2024-09-18T00:04:55.940 INFO:teuthology.orchestra.run.smithi114.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-18T00:04:55.940 INFO:teuthology.orchestra.run.smithi114.stdout:Need to get 113 MB of archives. 2024-09-18T00:04:55.940 INFO:teuthology.orchestra.run.smithi114.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-18T00:04:55.940 INFO:teuthology.orchestra.run.smithi114.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-18T00:04:55.979 INFO:teuthology.orchestra.run.smithi097.stdout:The following NEW packages will be installed: 2024-09-18T00:04:55.979 INFO:teuthology.orchestra.run.smithi097.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-18T00:04:55.980 INFO:teuthology.orchestra.run.smithi097.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-18T00:04:55.980 INFO:teuthology.orchestra.run.smithi097.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-18T00:04:55.980 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages will be upgraded: 2024-09-18T00:04:55.981 INFO:teuthology.orchestra.run.smithi097.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-18T00:04:56.033 INFO:teuthology.orchestra.run.smithi097.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-18T00:04:56.033 INFO:teuthology.orchestra.run.smithi097.stdout:Need to get 113 MB of archives. 2024-09-18T00:04:56.034 INFO:teuthology.orchestra.run.smithi097.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-18T00:04:56.034 INFO:teuthology.orchestra.run.smithi097.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-18T00:04:56.391 INFO:teuthology.orchestra.run.smithi114.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-18T00:04:56.474 INFO:teuthology.orchestra.run.smithi097.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-18T00:04:56.491 INFO:teuthology.orchestra.run.smithi114.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-18T00:04:56.565 INFO:teuthology.orchestra.run.smithi097.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-18T00:04:57.327 INFO:teuthology.orchestra.run.smithi114.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-18T00:04:57.327 INFO:teuthology.orchestra.run.smithi114.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-18T00:04:57.327 INFO:teuthology.orchestra.run.smithi114.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-18T00:04:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-18T00:04:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-18T00:04:57.378 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-18T00:04:57.453 INFO:teuthology.orchestra.run.smithi114.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-18T00:04:57.476 INFO:teuthology.orchestra.run.smithi114.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-18T00:04:57.496 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-18T00:04:57.516 INFO:teuthology.orchestra.run.smithi097.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-18T00:04:57.863 INFO:teuthology.orchestra.run.smithi114.stdout:Fetched 113 MB in 2s (71.7 MB/s) 2024-09-18T00:04:57.977 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 113 MB in 2s (71.2 MB/s) 2024-09-18T00:04:58.031 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-18T00:04:58.156 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-18T00:05:00.104 INFO:teuthology.orchestra.run.smithi114.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-18T00:05:00.108 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-18T00:05:00.142 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-18T00:05:00.146 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-18T00:05:00.186 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:00.262 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:02.280 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-18T00:05:02.301 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-18T00:05:02.401 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:02.824 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-18T00:05:02.845 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-18T00:05:02.874 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:02.964 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-18T00:05:02.985 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-18T00:05:03.093 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:03.526 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-18T00:05:03.546 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-18T00:05:03.584 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:09.598 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-18T00:05:09.717 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-18T00:05:10.059 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-18T00:05:10.187 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-18T00:05:10.485 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-18T00:05:10.490 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-18T00:05:10.501 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-18T00:05:10.540 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-18T00:05:10.613 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-18T00:05:10.954 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-18T00:05:11.058 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-18T00:05:11.386 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-18T00:05:11.397 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-18T00:05:11.436 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-18T00:05:16.131 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-18T00:05:16.168 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-18T00:05:16.206 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:16.969 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-18T00:05:17.006 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-18T00:05:17.043 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:18.785 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-18T00:05:18.909 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-18T00:05:19.241 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-18T00:05:19.363 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:19.489 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-18T00:05:19.618 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:20.447 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-18T00:05:20.572 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-18T00:05:20.944 INFO:teuthology.orchestra.run.smithi114.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-18T00:05:20.944 INFO:teuthology.orchestra.run.smithi114.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-18T00:05:20.944 INFO:teuthology.orchestra.run.smithi114.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-18T00:05:20.944 INFO:teuthology.orchestra.run.smithi114.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-18T00:05:20.955 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-18T00:05:21.077 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:21.136 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:21.203 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-18T00:05:21.332 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:22.546 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-18T00:05:22.691 INFO:teuthology.orchestra.run.smithi097.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-18T00:05:22.691 INFO:teuthology.orchestra.run.smithi097.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-18T00:05:22.691 INFO:teuthology.orchestra.run.smithi097.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-18T00:05:22.691 INFO:teuthology.orchestra.run.smithi097.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-18T00:05:22.994 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:23.106 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-18T00:05:23.724 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:25.111 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-18T00:05:25.117 INFO:teuthology.orchestra.run.smithi114.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:25.170 INFO:teuthology.orchestra.run.smithi114.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-18T00:05:25.171 INFO:teuthology.orchestra.run.smithi114.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-18T00:05:25.262 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-18T00:05:25.388 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:26.815 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-18T00:05:26.882 INFO:teuthology.orchestra.run.smithi097.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-18T00:05:26.882 INFO:teuthology.orchestra.run.smithi097.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-18T00:05:39.748 INFO:teuthology.orchestra.run.smithi114.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-18T00:05:39.748 INFO:teuthology.orchestra.run.smithi114.stdout:Sourcing file `/etc/default/grub' 2024-09-18T00:05:39.759 INFO:teuthology.orchestra.run.smithi114.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T00:05:39.786 INFO:teuthology.orchestra.run.smithi114.stdout:Generating grub configuration file ... 2024-09-18T00:05:40.053 INFO:teuthology.orchestra.run.smithi114.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-18T00:05:40.088 INFO:teuthology.orchestra.run.smithi114.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-18T00:05:40.375 INFO:teuthology.orchestra.run.smithi114.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-18T00:05:40.382 INFO:teuthology.orchestra.run.smithi114.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-18T00:05:40.451 INFO:teuthology.orchestra.run.smithi114.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-18T00:05:40.456 INFO:teuthology.orchestra.run.smithi114.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-18T00:05:40.625 INFO:teuthology.orchestra.run.smithi114.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-18T00:05:40.625 INFO:teuthology.orchestra.run.smithi114.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-18T00:05:40.625 INFO:teuthology.orchestra.run.smithi114.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-18T00:05:40.665 INFO:teuthology.orchestra.run.smithi114.stdout:done 2024-09-18T00:05:40.792 INFO:teuthology.orchestra.run.smithi097.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-18T00:05:40.793 INFO:teuthology.orchestra.run.smithi097.stdout:Sourcing file `/etc/default/grub' 2024-09-18T00:05:40.807 INFO:teuthology.orchestra.run.smithi097.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T00:05:40.830 INFO:teuthology.orchestra.run.smithi097.stdout:Generating grub configuration file ... 2024-09-18T00:05:41.104 INFO:teuthology.orchestra.run.smithi097.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-18T00:05:41.128 INFO:teuthology.orchestra.run.smithi097.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-18T00:05:41.178 DEBUG:teuthology.orchestra.run.smithi114:> dpkg -s linux-image-generic 2024-09-18T00:05:41.204 INFO:teuthology.orchestra.run.smithi114.stdout:Package: linux-image-generic 2024-09-18T00:05:41.204 INFO:teuthology.orchestra.run.smithi114.stdout:Status: install ok installed 2024-09-18T00:05:41.204 INFO:teuthology.orchestra.run.smithi114.stdout:Priority: optional 2024-09-18T00:05:41.204 INFO:teuthology.orchestra.run.smithi114.stdout:Section: kernel 2024-09-18T00:05:41.204 INFO:teuthology.orchestra.run.smithi114.stdout:Installed-Size: 21 2024-09-18T00:05:41.204 INFO:teuthology.orchestra.run.smithi114.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T00:05:41.204 INFO:teuthology.orchestra.run.smithi114.stdout:Architecture: amd64 2024-09-18T00:05:41.205 INFO:teuthology.orchestra.run.smithi114.stdout:Source: linux-meta 2024-09-18T00:05:41.205 INFO:teuthology.orchestra.run.smithi114.stdout:Version: 5.15.0.122.122 2024-09-18T00:05:41.205 INFO:teuthology.orchestra.run.smithi114.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-18T00:05:41.205 INFO:teuthology.orchestra.run.smithi114.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T00:05:41.205 INFO:teuthology.orchestra.run.smithi114.stdout:Recommends: thermald 2024-09-18T00:05:41.205 INFO:teuthology.orchestra.run.smithi114.stdout:Description: Generic Linux kernel image 2024-09-18T00:05:41.205 INFO:teuthology.orchestra.run.smithi114.stdout: This package will always depend on the latest generic kernel image 2024-09-18T00:05:41.205 INFO:teuthology.orchestra.run.smithi114.stdout: available. 2024-09-18T00:05:41.205 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-18T00:05:41.205 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T00:05:41.205 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T00:05:41.206 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-18T00:05:41.206 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi114.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T00:05:41.206 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-get clean 2024-09-18T00:05:41.407 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-get update 2024-09-18T00:05:41.445 INFO:teuthology.orchestra.run.smithi097.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-18T00:05:41.450 INFO:teuthology.orchestra.run.smithi097.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-18T00:05:41.519 INFO:teuthology.orchestra.run.smithi097.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-18T00:05:41.525 INFO:teuthology.orchestra.run.smithi097.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-18T00:05:41.527 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-18T00:05:41.647 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-18T00:05:41.710 INFO:teuthology.orchestra.run.smithi097.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-18T00:05:41.711 INFO:teuthology.orchestra.run.smithi097.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-18T00:05:41.711 INFO:teuthology.orchestra.run.smithi097.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-18T00:05:41.729 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-18T00:05:41.757 INFO:teuthology.orchestra.run.smithi097.stdout:done 2024-09-18T00:05:41.809 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-18T00:05:42.247 DEBUG:teuthology.orchestra.run.smithi097:> dpkg -s linux-image-generic 2024-09-18T00:05:42.275 INFO:teuthology.orchestra.run.smithi097.stdout:Package: linux-image-generic 2024-09-18T00:05:42.275 INFO:teuthology.orchestra.run.smithi097.stdout:Status: install ok installed 2024-09-18T00:05:42.275 INFO:teuthology.orchestra.run.smithi097.stdout:Priority: optional 2024-09-18T00:05:42.276 INFO:teuthology.orchestra.run.smithi097.stdout:Section: kernel 2024-09-18T00:05:42.276 INFO:teuthology.orchestra.run.smithi097.stdout:Installed-Size: 21 2024-09-18T00:05:42.276 INFO:teuthology.orchestra.run.smithi097.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T00:05:42.276 INFO:teuthology.orchestra.run.smithi097.stdout:Architecture: amd64 2024-09-18T00:05:42.276 INFO:teuthology.orchestra.run.smithi097.stdout:Source: linux-meta 2024-09-18T00:05:42.276 INFO:teuthology.orchestra.run.smithi097.stdout:Version: 5.15.0.122.122 2024-09-18T00:05:42.276 INFO:teuthology.orchestra.run.smithi097.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-18T00:05:42.276 INFO:teuthology.orchestra.run.smithi097.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T00:05:42.276 INFO:teuthology.orchestra.run.smithi097.stdout:Recommends: thermald 2024-09-18T00:05:42.276 INFO:teuthology.orchestra.run.smithi097.stdout:Description: Generic Linux kernel image 2024-09-18T00:05:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: This package will always depend on the latest generic kernel image 2024-09-18T00:05:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: available. 2024-09-18T00:05:42.278 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-18T00:05:42.278 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T00:05:42.278 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T00:05:42.278 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-18T00:05:42.278 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi097.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T00:05:42.278 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get clean 2024-09-18T00:05:42.445 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get update 2024-09-18T00:05:42.696 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-18T00:05:42.696 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-18T00:05:42.767 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-18T00:05:42.849 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-18T00:05:42.921 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2024-09-18T00:05:42.939 DEBUG:teuthology.orchestra.run.smithi114:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T00:05:43.005 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2024-09-18T00:05:43.212 INFO:teuthology.orchestra.run.smithi114.stdout:Building dependency tree... 2024-09-18T00:05:43.213 INFO:teuthology.orchestra.run.smithi114.stdout:Reading state information... 2024-09-18T00:05:43.403 INFO:teuthology.orchestra.run.smithi114.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-18T00:05:43.403 INFO:teuthology.orchestra.run.smithi114.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T00:05:43.403 INFO:teuthology.orchestra.run.smithi114.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-18T00:05:43.403 INFO:teuthology.orchestra.run.smithi114.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-18T00:05:43.404 INFO:teuthology.orchestra.run.smithi114.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-18T00:05:43.404 INFO:teuthology.orchestra.run.smithi114.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-18T00:05:43.404 INFO:teuthology.orchestra.run.smithi114.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-18T00:05:43.404 INFO:teuthology.orchestra.run.smithi114.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T00:05:43.447 INFO:teuthology.orchestra.run.smithi114.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-18T00:05:43.449 DEBUG:teuthology.orchestra.run.smithi114:> dpkg -s linux-image-generic 2024-09-18T00:05:43.465 INFO:teuthology.orchestra.run.smithi114.stdout:Package: linux-image-generic 2024-09-18T00:05:43.465 INFO:teuthology.orchestra.run.smithi114.stdout:Status: install ok installed 2024-09-18T00:05:43.465 INFO:teuthology.orchestra.run.smithi114.stdout:Priority: optional 2024-09-18T00:05:43.465 INFO:teuthology.orchestra.run.smithi114.stdout:Section: kernel 2024-09-18T00:05:43.465 INFO:teuthology.orchestra.run.smithi114.stdout:Installed-Size: 21 2024-09-18T00:05:43.465 INFO:teuthology.orchestra.run.smithi114.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T00:05:43.465 INFO:teuthology.orchestra.run.smithi114.stdout:Architecture: amd64 2024-09-18T00:05:43.465 INFO:teuthology.orchestra.run.smithi114.stdout:Source: linux-meta 2024-09-18T00:05:43.465 INFO:teuthology.orchestra.run.smithi114.stdout:Version: 5.15.0.122.122 2024-09-18T00:05:43.465 INFO:teuthology.orchestra.run.smithi114.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-18T00:05:43.465 INFO:teuthology.orchestra.run.smithi114.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T00:05:43.466 INFO:teuthology.orchestra.run.smithi114.stdout:Recommends: thermald 2024-09-18T00:05:43.466 INFO:teuthology.orchestra.run.smithi114.stdout:Description: Generic Linux kernel image 2024-09-18T00:05:43.466 INFO:teuthology.orchestra.run.smithi114.stdout: This package will always depend on the latest generic kernel image 2024-09-18T00:05:43.466 INFO:teuthology.orchestra.run.smithi114.stdout: available. 2024-09-18T00:05:43.466 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-09-18T00:05:43.512 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.RGNDXK8uDs 2024-09-18T00:05:43.513 DEBUG:teuthology.orchestra.run.smithi114:> sudo cp /boot/grub/grub.cfg /tmp/tmp.RGNDXK8uDs 2024-09-18T00:05:43.568 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.RGNDXK8uDs 2024-09-18T00:05:43.683 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.RGNDXK8uDs is 10KB 2024-09-18T00:05:43.696 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.RGNDXK8uDs 2024-09-18T00:05:43.701 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-18T00:05:43.756 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:05:43.756 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-18T00:05:43.756 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-18T00:05:43.822 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-18T00:05:43.822 DEBUG:teuthology.orchestra.run.smithi114:> sudo update-grub 2024-09-18T00:05:44.177 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-09-18T00:05:44.195 DEBUG:teuthology.orchestra.run.smithi097:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T00:05:44.247 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-09-18T00:05:44.434 INFO:teuthology.orchestra.run.smithi097.stdout:Building dependency tree... 2024-09-18T00:05:44.434 INFO:teuthology.orchestra.run.smithi097.stdout:Reading state information... 2024-09-18T00:05:44.604 INFO:teuthology.orchestra.run.smithi097.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-18T00:05:44.604 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T00:05:44.604 INFO:teuthology.orchestra.run.smithi097.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-18T00:05:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-18T00:05:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-18T00:05:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-18T00:05:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-18T00:05:44.605 INFO:teuthology.orchestra.run.smithi097.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T00:05:44.645 INFO:teuthology.orchestra.run.smithi097.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-18T00:05:44.647 DEBUG:teuthology.orchestra.run.smithi097:> dpkg -s linux-image-generic 2024-09-18T00:05:44.702 INFO:teuthology.orchestra.run.smithi097.stdout:Package: linux-image-generic 2024-09-18T00:05:44.703 INFO:teuthology.orchestra.run.smithi097.stdout:Status: install ok installed 2024-09-18T00:05:44.703 INFO:teuthology.orchestra.run.smithi097.stdout:Priority: optional 2024-09-18T00:05:44.703 INFO:teuthology.orchestra.run.smithi097.stdout:Section: kernel 2024-09-18T00:05:44.703 INFO:teuthology.orchestra.run.smithi097.stdout:Installed-Size: 21 2024-09-18T00:05:44.703 INFO:teuthology.orchestra.run.smithi097.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T00:05:44.703 INFO:teuthology.orchestra.run.smithi097.stdout:Architecture: amd64 2024-09-18T00:05:44.703 INFO:teuthology.orchestra.run.smithi097.stdout:Source: linux-meta 2024-09-18T00:05:44.703 INFO:teuthology.orchestra.run.smithi097.stdout:Version: 5.15.0.122.122 2024-09-18T00:05:44.703 INFO:teuthology.orchestra.run.smithi097.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-18T00:05:44.703 INFO:teuthology.orchestra.run.smithi097.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T00:05:44.703 INFO:teuthology.orchestra.run.smithi097.stdout:Recommends: thermald 2024-09-18T00:05:44.703 INFO:teuthology.orchestra.run.smithi097.stdout:Description: Generic Linux kernel image 2024-09-18T00:05:44.703 INFO:teuthology.orchestra.run.smithi097.stdout: This package will always depend on the latest generic kernel image 2024-09-18T00:05:44.703 INFO:teuthology.orchestra.run.smithi097.stdout: available. 2024-09-18T00:05:44.704 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-09-18T00:05:44.747 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.IzTqwtlaEw 2024-09-18T00:05:44.747 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /boot/grub/grub.cfg /tmp/tmp.IzTqwtlaEw 2024-09-18T00:05:44.797 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.IzTqwtlaEw 2024-09-18T00:05:44.911 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.IzTqwtlaEw is 10KB 2024-09-18T00:05:44.923 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.IzTqwtlaEw 2024-09-18T00:05:44.927 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-18T00:05:44.981 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:05:44.981 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-18T00:05:44.981 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-18T00:05:44.997 INFO:teuthology.orchestra.run.smithi114.stderr:Sourcing file `/etc/default/grub' 2024-09-18T00:05:44.998 INFO:teuthology.orchestra.run.smithi114.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T00:05:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:Generating grub configuration file ... 2024-09-18T00:05:45.057 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-18T00:05:45.057 DEBUG:teuthology.orchestra.run.smithi097:> sudo update-grub 2024-09-18T00:05:45.189 INFO:teuthology.orchestra.run.smithi114.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-18T00:05:45.195 INFO:teuthology.orchestra.run.smithi114.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-18T00:05:45.433 INFO:teuthology.orchestra.run.smithi114.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-18T00:05:45.440 INFO:teuthology.orchestra.run.smithi114.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-18T00:05:45.484 INFO:teuthology.orchestra.run.smithi114.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-18T00:05:45.489 INFO:teuthology.orchestra.run.smithi114.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-18T00:05:45.612 INFO:teuthology.orchestra.run.smithi114.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-18T00:05:45.612 INFO:teuthology.orchestra.run.smithi114.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-18T00:05:45.612 INFO:teuthology.orchestra.run.smithi114.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-18T00:05:45.630 INFO:teuthology.orchestra.run.smithi114.stderr:done 2024-09-18T00:05:45.639 DEBUG:teuthology.orchestra.run.smithi114:> sudo shutdown -r now 2024-09-18T00:05:46.416 INFO:teuthology.orchestra.run.smithi097.stderr:Sourcing file `/etc/default/grub' 2024-09-18T00:05:46.417 INFO:teuthology.orchestra.run.smithi097.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T00:05:46.420 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2024-09-18T00:05:46.666 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-18T00:05:46.673 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-18T00:05:46.910 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-18T00:05:46.915 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-18T00:05:46.958 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-18T00:05:46.964 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-18T00:05:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-18T00:05:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-18T00:05:47.122 INFO:teuthology.orchestra.run.smithi097.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-18T00:05:47.140 INFO:teuthology.orchestra.run.smithi097.stderr:done 2024-09-18T00:05:47.143 DEBUG:teuthology.orchestra.run.smithi097:> sudo shutdown -r now 2024-09-18T00:06:15.668 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-18T00:06:15.668 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-18T00:06:15.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:06:17.147 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-18T00:06:17.147 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-18T00:06:17.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:06:34.045 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-09-18T00:06:35.581 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-09-18T00:06:43.047 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-18T00:06:43.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:06:44.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-18T00:06:44.584 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:06:46.109 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-09-18T00:06:47.645 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-09-18T00:06:58.117 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-18T00:06:58.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:06:59.647 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-18T00:06:59.648 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:07:16.541 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-09-18T00:07:31.379 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-09-18T00:07:31.542 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-18T00:07:31.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:07:31.737 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-09-18T00:07:31.957 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-18T00:07:31.957 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-122-generic"... 2024-09-18T00:07:31.957 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-09-18T00:07:32.006 INFO:teuthology.orchestra.run.smithi097.stdout:5.15.0-122-generic 2024-09-18T00:07:32.007 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-18T00:07:32.007 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T00:07:32.007 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-18T00:07:32.312 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-18T00:07:32.312 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-122-generic"... 2024-09-18T00:07:32.312 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-09-18T00:07:32.358 INFO:teuthology.orchestra.run.smithi114.stdout:5.15.0-122-generic 2024-09-18T00:07:32.358 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-18T00:07:32.358 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T00:07:32.358 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-18T00:07:33.008 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-18T00:07:33.008 DEBUG:teuthology.orchestra.run.smithi097:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T00:07:33.074 INFO:teuthology.orchestra.run.smithi097.stdout:ttyS1 2024-09-18T00:07:33.110 DEBUG:teuthology.parallel:result is None 2024-09-18T00:07:33.359 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-18T00:07:33.360 DEBUG:teuthology.orchestra.run.smithi114:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T00:07:33.434 INFO:teuthology.orchestra.run.smithi114.stdout:ttyS1 2024-09-18T00:07:33.475 DEBUG:teuthology.parallel:result is None 2024-09-18T00:07:33.475 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-18T00:07:33.546 INFO:teuthology.task.internal:Creating test directory... 2024-09-18T00:07:33.546 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T00:07:33.548 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T00:07:33.554 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-18T00:07:33.591 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-18T00:07:33.665 INFO:teuthology.task.internal:Creating archive directory... 2024-09-18T00:07:33.666 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T00:07:33.668 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T00:07:33.745 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-18T00:07:33.762 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-18T00:07:33.762 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T00:07:33.765 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T00:07:33.785 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T00:07:33.788 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T00:07:33.793 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T00:07:33.796 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T00:07:33.798 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-18T00:07:33.830 INFO:teuthology.task.internal:Configuring sudo... 2024-09-18T00:07:33.830 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T00:07:33.840 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T00:07:33.852 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-18T00:07:33.866 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-18T00:07:33.867 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T00:07:33.896 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T00:07:33.902 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T00:07:33.947 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T00:07:33.995 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:07:33.995 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T00:07:34.056 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T00:07:34.061 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T00:07:34.110 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:07:34.110 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T00:07:34.172 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-09-18T00:07:34.175 DEBUG:teuthology.orchestra.run.smithi114:> sudo service rsyslog restart 2024-09-18T00:07:34.249 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-18T00:07:34.292 INFO:teuthology.task.internal:Starting timer... 2024-09-18T00:07:34.292 INFO:teuthology.run_tasks:Running task pcp... 2024-09-18T00:07:34.352 INFO:teuthology.run_tasks:Running task selinux... 2024-09-18T00:07:34.405 DEBUG:teuthology.task.selinux:Excluding smithi097: OS 'ubuntu' does not support SELinux 2024-09-18T00:07:34.405 DEBUG:teuthology.task.selinux:Excluding smithi114: OS 'ubuntu' does not support SELinux 2024-09-18T00:07:34.405 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-18T00:07:34.406 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-18T00:07:34.406 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-18T00:07:34.406 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-18T00:07:34.443 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-18T00:07:34.447 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-18T00:07:34.562 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-18T00:07:34.573 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-18T00:07:34.574 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi097.front.sepia.ceph.com,smithi114.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-18T00:15:02.556 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi114.front.sepia.ceph.com')] 2024-09-18T00:15:02.557 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-18T00:15:02.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:15:02.642 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-09-18T00:15:02.716 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-18T00:15:02.717 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-18T00:15:02.717 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:15:02.795 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-09-18T00:15:02.872 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-18T00:15:02.873 INFO:teuthology.run_tasks:Running task clock... 2024-09-18T00:15:02.884 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-18T00:15:02.884 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T00:15:02.884 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T00:15:02.887 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T00:15:02.887 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T00:15:02.915 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-18T00:15:02.916 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: Command line: ntpd -gq 2024-09-18T00:15:02.916 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: ---------------------------------------------------- 2024-09-18T00:15:02.916 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: ntp-4 is maintained by Network Time Foundation, 2024-09-18T00:15:02.916 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-18T00:15:02.916 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: corporation. Support and training for ntp-4 are 2024-09-18T00:15:02.916 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: available at https://www.nwtime.org/support 2024-09-18T00:15:02.916 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: ---------------------------------------------------- 2024-09-18T00:15:02.917 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: proto: precision = 0.044 usec (-24) 2024-09-18T00:15:02.917 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: basedate set to 2022-02-04 2024-09-18T00:15:02.917 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: gps base set to 2022-02-06 (week 2196) 2024-09-18T00:15:02.917 INFO:teuthology.orchestra.run.smithi097.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T00:15:02.917 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T00:15:02.917 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: restrict ::: KOD does nothing without LIMITED. 2024-09-18T00:15:02.918 INFO:teuthology.orchestra.run.smithi097.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-18T00:15:02.919 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: Listen and drop on 0 v6wildcard [::]:123 2024-09-18T00:15:02.919 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-18T00:15:02.919 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: Listen normally on 2 lo 127.0.0.1:123 2024-09-18T00:15:02.919 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: Listen normally on 3 enp3s0f1 172.21.15.97:123 2024-09-18T00:15:02.919 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: Listen normally on 4 lo [::1]:123 2024-09-18T00:15:02.919 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d805%5]:123 2024-09-18T00:15:02.919 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:02 ntpd[18453]: Listening on routing socket on fd #22 for interface updates 2024-09-18T00:15:02.939 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-18T00:15:02.939 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: Command line: ntpd -gq 2024-09-18T00:15:02.939 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: ---------------------------------------------------- 2024-09-18T00:15:02.939 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: ntp-4 is maintained by Network Time Foundation, 2024-09-18T00:15:02.939 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-18T00:15:02.939 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: corporation. Support and training for ntp-4 are 2024-09-18T00:15:02.939 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: available at https://www.nwtime.org/support 2024-09-18T00:15:02.939 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: ---------------------------------------------------- 2024-09-18T00:15:02.940 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: proto: precision = 0.061 usec (-24) 2024-09-18T00:15:02.940 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: basedate set to 2022-02-04 2024-09-18T00:15:02.940 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: gps base set to 2022-02-06 (week 2196) 2024-09-18T00:15:02.940 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T00:15:02.941 INFO:teuthology.orchestra.run.smithi114.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T00:15:02.941 INFO:teuthology.orchestra.run.smithi114.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-18T00:15:02.942 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: restrict ::: KOD does nothing without LIMITED. 2024-09-18T00:15:02.942 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: Listen and drop on 0 v6wildcard [::]:123 2024-09-18T00:15:02.942 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-18T00:15:02.942 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: Listen normally on 2 lo 127.0.0.1:123 2024-09-18T00:15:02.942 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: Listen normally on 3 enp3s0f1 172.21.15.114:123 2024-09-18T00:15:02.942 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: Listen normally on 4 lo [::1]:123 2024-09-18T00:15:02.942 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cb63%5]:123 2024-09-18T00:15:02.942 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:02 ntpd[18470]: Listening on routing socket on fd #22 for interface updates 2024-09-18T00:15:03.916 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:03 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:03 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:03 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:03 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:03 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:03 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:03 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:03 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:03 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:03 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:03 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:03 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:03 ntpd[18470]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:03 ntpd[18470]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:03.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:03 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:03.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:03 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:03.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:03 ntpd[18470]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:03.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:03 ntpd[18470]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:04.916 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:04 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:04.916 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:04 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:05.916 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:05 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:05.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:05 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:05.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:05 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:05.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:05 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:05.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:05 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:05.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:05 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:05.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:05 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:05.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:05 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:05.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:05 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:05.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:05 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:05.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:05 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:05.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:05 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:05.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:05 ntpd[18470]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:05.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:05 ntpd[18470]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:05.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:05 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:05.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:05 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:05.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:05 ntpd[18470]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:05.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:05 ntpd[18470]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:06.916 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:06 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:06.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:06 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:06.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:06 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:06.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:06 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:06.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:06 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:06.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:06 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:06.939 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:06 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:06.939 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:06 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:06.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:06 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:06.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:06 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:06.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:06 ntpd[18470]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:06.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:06 ntpd[18470]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:07.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:07 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:07.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:07 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:07.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:07 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:07.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:07 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:07.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:07 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:07.918 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:07 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:07.918 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:07 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:07.918 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:07 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:07.918 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:07 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:07.918 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:07 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:07.918 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:07 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:07.918 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:07 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:07.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:07 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:07.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:07 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:07.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:07 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:07.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:07 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:07.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:07 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:07 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:07 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:07 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:07 ntpd[18470]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:07 ntpd[18470]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:07 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:07 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:07 ntpd[18470]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:07.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:07 ntpd[18470]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:08.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:08 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:08.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:08 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:08.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:08 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:08.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:08 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:08.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:08 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:08.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:08 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:08.939 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:08 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:08.939 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:08 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:08.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:08 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:08.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:08 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:08.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:08 ntpd[18470]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:08.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:08 ntpd[18470]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:09 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:09 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:09 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:09 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:09 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:09.917 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:09 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:09.918 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:09 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:09.918 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:09 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:09.918 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:09 ntpd[18453]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:09.918 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:09 ntpd[18453]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:09.918 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:09 ntpd[18453]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:09.918 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:09 ntpd[18453]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:09.918 INFO:teuthology.orchestra.run.smithi097.stdout:18 Sep 00:15:09 ntpd[18453]: ntpd: time slew +0.000629 s 2024-09-18T00:15:09.918 INFO:teuthology.orchestra.run.smithi097.stdout:ntpd: time slew +0.000629s 2024-09-18T00:15:09.919 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:09 ntpd[18453]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-18T00:15:09.919 INFO:teuthology.orchestra.run.smithi097.stderr:18 Sep 00:15:09 ntpd[18453]: can't open /var/log/ntpstats/loopstats.20240918: Permission denied 2024-09-18T00:15:09.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:09 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:09.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:09 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:09.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:09 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:09.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:09 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:09.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:09 ntpd[18470]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T00:15:09.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:09 ntpd[18470]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T00:15:09.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:09 ntpd[18470]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T00:15:09.940 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:09 ntpd[18470]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T00:15:09.941 INFO:teuthology.orchestra.run.smithi114.stdout:18 Sep 00:15:09 ntpd[18470]: ntpd: time slew -0.002643 s 2024-09-18T00:15:09.941 INFO:teuthology.orchestra.run.smithi114.stdout:ntpd: time slew -0.002643s 2024-09-18T00:15:09.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:09 ntpd[18470]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-18T00:15:09.941 INFO:teuthology.orchestra.run.smithi114.stderr:18 Sep 00:15:09 ntpd[18470]: can't open /var/log/ntpstats/loopstats.20240918: Permission denied 2024-09-18T00:15:09.971 INFO:teuthology.orchestra.run.smithi097.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T00:15:09.971 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================== 2024-09-18T00:15:09.971 INFO:teuthology.orchestra.run.smithi097.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:15:09.971 INFO:teuthology.orchestra.run.smithi097.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:15:09.971 INFO:teuthology.orchestra.run.smithi097.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:15:09.971 INFO:teuthology.orchestra.run.smithi097.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:15:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T00:15:09.996 INFO:teuthology.orchestra.run.smithi114.stdout:============================================================================== 2024-09-18T00:15:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:15:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:15:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:15:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:15:09.997 INFO:teuthology.run_tasks:Running task install... 2024-09-18T00:15:10.013 DEBUG:teuthology.task.install:project ceph 2024-09-18T00:15:10.014 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}} 2024-09-18T00:15:10.014 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'} 2024-09-18T00:15:10.014 INFO:teuthology.task.install:Using flavor: default 2024-09-18T00:15:10.019 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-18T00:15:10.019 INFO:teuthology.task.install:extra packages: [] 2024-09-18T00:15:10.020 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-key list | grep Ceph 2024-09-18T00:15:10.020 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-key list | grep Ceph 2024-09-18T00:15:10.073 INFO:teuthology.orchestra.run.smithi097.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-18T00:15:10.095 INFO:teuthology.orchestra.run.smithi114.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-18T00:15:10.099 INFO:teuthology.orchestra.run.smithi097.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-18T00:15:10.099 INFO:teuthology.orchestra.run.smithi097.stdout:uid [ unknown] Ceph.com (release key) 2024-09-18T00:15:10.100 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-18T00:15:10.101 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T00:15:10.120 INFO:teuthology.orchestra.run.smithi114.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-18T00:15:10.120 INFO:teuthology.orchestra.run.smithi114.stdout:uid [ unknown] Ceph.com (release key) 2024-09-18T00:15:10.121 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-18T00:15:10.121 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T00:15:10.248 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default/ 2024-09-18T00:15:10.248 INFO:teuthology.task.install.deb:Package version is 19.1.1-330-gc2ddf972-1jammy 2024-09-18T00:15:10.265 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default/ 2024-09-18T00:15:10.265 INFO:teuthology.task.install.deb:Package version is 19.1.1-330-gc2ddf972-1jammy 2024-09-18T00:15:10.361 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:15:10.361 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-18T00:15:10.374 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:15:10.374 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-18T00:15:10.375 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get update 2024-09-18T00:15:10.389 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-get update 2024-09-18T00:15:10.598 INFO:teuthology.orchestra.run.smithi097.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy InRelease 2024-09-18T00:15:10.611 INFO:teuthology.orchestra.run.smithi114.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy InRelease 2024-09-18T00:15:10.620 INFO:teuthology.orchestra.run.smithi097.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-18T00:15:10.633 INFO:teuthology.orchestra.run.smithi114.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-18T00:15:10.643 INFO:teuthology.orchestra.run.smithi097.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-18T00:15:10.656 INFO:teuthology.orchestra.run.smithi114.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-18T00:15:10.673 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-18T00:15:10.673 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-18T00:15:10.673 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,409 B] 2024-09-18T00:15:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-18T00:15:10.686 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-18T00:15:10.687 INFO:teuthology.orchestra.run.smithi114.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-18T00:15:10.702 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-18T00:15:10.717 INFO:teuthology.orchestra.run.smithi114.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,409 B] 2024-09-18T00:15:10.745 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-18T00:15:10.759 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-18T00:15:10.827 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-18T00:15:10.841 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-18T00:15:11.016 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 28.9 kB in 0s (62.6 kB/s) 2024-09-18T00:15:11.053 INFO:teuthology.orchestra.run.smithi114.stdout:Fetched 28.9 kB in 0s (59.6 kB/s) 2024-09-18T00:15:11.990 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-09-18T00:15:12.007 DEBUG:teuthology.orchestra.run.smithi097:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-330-gc2ddf972-1jammy cephadm=19.1.1-330-gc2ddf972-1jammy ceph-mds=19.1.1-330-gc2ddf972-1jammy ceph-mgr=19.1.1-330-gc2ddf972-1jammy ceph-common=19.1.1-330-gc2ddf972-1jammy ceph-fuse=19.1.1-330-gc2ddf972-1jammy ceph-test=19.1.1-330-gc2ddf972-1jammy ceph-volume=19.1.1-330-gc2ddf972-1jammy radosgw=19.1.1-330-gc2ddf972-1jammy python3-rados=19.1.1-330-gc2ddf972-1jammy python3-rgw=19.1.1-330-gc2ddf972-1jammy python3-cephfs=19.1.1-330-gc2ddf972-1jammy python3-rbd=19.1.1-330-gc2ddf972-1jammy libcephfs2=19.1.1-330-gc2ddf972-1jammy libcephfs-dev=19.1.1-330-gc2ddf972-1jammy librados2=19.1.1-330-gc2ddf972-1jammy librbd1=19.1.1-330-gc2ddf972-1jammy rbd-fuse=19.1.1-330-gc2ddf972-1jammy 2024-09-18T00:15:12.052 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2024-09-18T00:15:12.069 DEBUG:teuthology.orchestra.run.smithi114:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-330-gc2ddf972-1jammy cephadm=19.1.1-330-gc2ddf972-1jammy ceph-mds=19.1.1-330-gc2ddf972-1jammy ceph-mgr=19.1.1-330-gc2ddf972-1jammy ceph-common=19.1.1-330-gc2ddf972-1jammy ceph-fuse=19.1.1-330-gc2ddf972-1jammy ceph-test=19.1.1-330-gc2ddf972-1jammy ceph-volume=19.1.1-330-gc2ddf972-1jammy radosgw=19.1.1-330-gc2ddf972-1jammy python3-rados=19.1.1-330-gc2ddf972-1jammy python3-rgw=19.1.1-330-gc2ddf972-1jammy python3-cephfs=19.1.1-330-gc2ddf972-1jammy python3-rbd=19.1.1-330-gc2ddf972-1jammy libcephfs2=19.1.1-330-gc2ddf972-1jammy libcephfs-dev=19.1.1-330-gc2ddf972-1jammy librados2=19.1.1-330-gc2ddf972-1jammy librbd1=19.1.1-330-gc2ddf972-1jammy rbd-fuse=19.1.1-330-gc2ddf972-1jammy 2024-09-18T00:15:12.070 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2024-09-18T00:15:12.123 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2024-09-18T00:15:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:Building dependency tree... 2024-09-18T00:15:12.275 INFO:teuthology.orchestra.run.smithi097.stdout:Reading state information... 2024-09-18T00:15:12.310 INFO:teuthology.orchestra.run.smithi114.stdout:Building dependency tree... 2024-09-18T00:15:12.311 INFO:teuthology.orchestra.run.smithi114.stdout:Reading state information... 2024-09-18T00:15:12.470 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T00:15:12.470 INFO:teuthology.orchestra.run.smithi097.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-18T00:15:12.471 INFO:teuthology.orchestra.run.smithi097.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-18T00:15:12.471 INFO:teuthology.orchestra.run.smithi097.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-18T00:15:12.471 INFO:teuthology.orchestra.run.smithi097.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-18T00:15:12.471 INFO:teuthology.orchestra.run.smithi097.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-18T00:15:12.471 INFO:teuthology.orchestra.run.smithi097.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T00:15:12.473 INFO:teuthology.orchestra.run.smithi097.stdout:The following additional packages will be installed: 2024-09-18T00:15:12.473 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-18T00:15:12.473 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-18T00:15:12.473 INFO:teuthology.orchestra.run.smithi097.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-18T00:15:12.473 INFO:teuthology.orchestra.run.smithi097.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-18T00:15:12.473 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-18T00:15:12.473 INFO:teuthology.orchestra.run.smithi097.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-18T00:15:12.473 INFO:teuthology.orchestra.run.smithi097.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-18T00:15:12.473 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-18T00:15:12.473 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-18T00:15:12.473 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-18T00:15:12.474 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-18T00:15:12.474 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-18T00:15:12.474 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-18T00:15:12.474 INFO:teuthology.orchestra.run.smithi097.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-18T00:15:12.474 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-18T00:15:12.474 INFO:teuthology.orchestra.run.smithi097.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-18T00:15:12.474 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-18T00:15:12.474 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-18T00:15:12.474 INFO:teuthology.orchestra.run.smithi097.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-18T00:15:12.474 INFO:teuthology.orchestra.run.smithi097.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-18T00:15:12.474 INFO:teuthology.orchestra.run.smithi097.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-18T00:15:12.474 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-18T00:15:12.474 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-18T00:15:12.475 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet zip 2024-09-18T00:15:12.475 INFO:teuthology.orchestra.run.smithi097.stdout:Suggested packages: 2024-09-18T00:15:12.475 INFO:teuthology.orchestra.run.smithi097.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-18T00:15:12.475 INFO:teuthology.orchestra.run.smithi097.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-18T00:15:12.475 INFO:teuthology.orchestra.run.smithi097.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-18T00:15:12.475 INFO:teuthology.orchestra.run.smithi097.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-18T00:15:12.475 INFO:teuthology.orchestra.run.smithi097.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-18T00:15:12.475 INFO:teuthology.orchestra.run.smithi097.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-18T00:15:12.475 INFO:teuthology.orchestra.run.smithi097.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-18T00:15:12.476 INFO:teuthology.orchestra.run.smithi097.stdout:Recommended packages: 2024-09-18T00:15:12.476 INFO:teuthology.orchestra.run.smithi097.stdout: btrfs-tools 2024-09-18T00:15:12.497 INFO:teuthology.orchestra.run.smithi114.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T00:15:12.497 INFO:teuthology.orchestra.run.smithi114.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-18T00:15:12.497 INFO:teuthology.orchestra.run.smithi114.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-18T00:15:12.497 INFO:teuthology.orchestra.run.smithi114.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-18T00:15:12.498 INFO:teuthology.orchestra.run.smithi114.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-18T00:15:12.498 INFO:teuthology.orchestra.run.smithi114.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-18T00:15:12.498 INFO:teuthology.orchestra.run.smithi114.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T00:15:12.499 INFO:teuthology.orchestra.run.smithi114.stdout:The following additional packages will be installed: 2024-09-18T00:15:12.499 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-18T00:15:12.499 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-18T00:15:12.499 INFO:teuthology.orchestra.run.smithi114.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-18T00:15:12.499 INFO:teuthology.orchestra.run.smithi114.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-18T00:15:12.500 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-18T00:15:12.500 INFO:teuthology.orchestra.run.smithi114.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-18T00:15:12.500 INFO:teuthology.orchestra.run.smithi114.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-18T00:15:12.501 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-18T00:15:12.501 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-18T00:15:12.501 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-18T00:15:12.501 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-18T00:15:12.501 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-18T00:15:12.501 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-18T00:15:12.501 INFO:teuthology.orchestra.run.smithi114.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-18T00:15:12.501 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-18T00:15:12.501 INFO:teuthology.orchestra.run.smithi114.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-18T00:15:12.501 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-18T00:15:12.501 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-18T00:15:12.501 INFO:teuthology.orchestra.run.smithi114.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-18T00:15:12.501 INFO:teuthology.orchestra.run.smithi114.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-18T00:15:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-18T00:15:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-18T00:15:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-18T00:15:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet zip 2024-09-18T00:15:12.502 INFO:teuthology.orchestra.run.smithi114.stdout:Suggested packages: 2024-09-18T00:15:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-18T00:15:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-18T00:15:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-18T00:15:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-18T00:15:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-18T00:15:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-18T00:15:12.503 INFO:teuthology.orchestra.run.smithi114.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-18T00:15:12.503 INFO:teuthology.orchestra.run.smithi114.stdout:Recommended packages: 2024-09-18T00:15:12.503 INFO:teuthology.orchestra.run.smithi114.stdout: btrfs-tools 2024-09-18T00:15:12.576 INFO:teuthology.orchestra.run.smithi097.stdout:The following NEW packages will be installed: 2024-09-18T00:15:12.576 INFO:teuthology.orchestra.run.smithi097.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-18T00:15:12.577 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-18T00:15:12.577 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-18T00:15:12.577 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-18T00:15:12.577 INFO:teuthology.orchestra.run.smithi097.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-18T00:15:12.577 INFO:teuthology.orchestra.run.smithi097.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-18T00:15:12.578 INFO:teuthology.orchestra.run.smithi097.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-18T00:15:12.578 INFO:teuthology.orchestra.run.smithi097.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-18T00:15:12.578 INFO:teuthology.orchestra.run.smithi097.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-18T00:15:12.578 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-18T00:15:12.578 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-18T00:15:12.578 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-18T00:15:12.578 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-18T00:15:12.578 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-18T00:15:12.579 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-18T00:15:12.579 INFO:teuthology.orchestra.run.smithi097.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-18T00:15:12.579 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-18T00:15:12.579 INFO:teuthology.orchestra.run.smithi097.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-18T00:15:12.579 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-18T00:15:12.579 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-18T00:15:12.579 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-18T00:15:12.579 INFO:teuthology.orchestra.run.smithi097.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-18T00:15:12.579 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-18T00:15:12.579 INFO:teuthology.orchestra.run.smithi097.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-18T00:15:12.579 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-18T00:15:12.579 INFO:teuthology.orchestra.run.smithi097.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-18T00:15:12.605 INFO:teuthology.orchestra.run.smithi114.stdout:The following NEW packages will be installed: 2024-09-18T00:15:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-18T00:15:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-18T00:15:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-18T00:15:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-18T00:15:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-18T00:15:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-18T00:15:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-18T00:15:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-18T00:15:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-18T00:15:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-18T00:15:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-18T00:15:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-18T00:15:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-18T00:15:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-18T00:15:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-18T00:15:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-18T00:15:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-18T00:15:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-18T00:15:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-18T00:15:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-18T00:15:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-18T00:15:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-18T00:15:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-18T00:15:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-18T00:15:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-18T00:15:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-18T00:15:12.668 INFO:teuthology.orchestra.run.smithi114.stdout:0 upgraded, 118 newly installed, 0 to remove and 331 not upgraded. 2024-09-18T00:15:12.668 INFO:teuthology.orchestra.run.smithi114.stdout:Need to get 181 MB of archives. 2024-09-18T00:15:12.668 INFO:teuthology.orchestra.run.smithi114.stdout:After this operation, 829 MB of additional disk space will be used. 2024-09-18T00:15:12.668 INFO:teuthology.orchestra.run.smithi114.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-18T00:15:12.720 INFO:teuthology.orchestra.run.smithi097.stdout:0 upgraded, 118 newly installed, 0 to remove and 331 not upgraded. 2024-09-18T00:15:12.720 INFO:teuthology.orchestra.run.smithi097.stdout:Need to get 181 MB of archives. 2024-09-18T00:15:12.720 INFO:teuthology.orchestra.run.smithi097.stdout:After this operation, 829 MB of additional disk space will be used. 2024-09-18T00:15:12.720 INFO:teuthology.orchestra.run.smithi097.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-330-gc2ddf972-1jammy [3,591 kB] 2024-09-18T00:15:12.751 INFO:teuthology.orchestra.run.smithi114.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-330-gc2ddf972-1jammy [3,591 kB] 2024-09-18T00:15:12.756 INFO:teuthology.orchestra.run.smithi114.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-18T00:15:12.759 INFO:teuthology.orchestra.run.smithi114.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-18T00:15:12.765 INFO:teuthology.orchestra.run.smithi097.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-18T00:15:12.776 INFO:teuthology.orchestra.run.smithi114.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-18T00:15:12.836 INFO:teuthology.orchestra.run.smithi114.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-18T00:15:12.841 INFO:teuthology.orchestra.run.smithi114.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-18T00:15:12.854 INFO:teuthology.orchestra.run.smithi114.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-18T00:15:12.859 INFO:teuthology.orchestra.run.smithi114.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-18T00:15:12.859 INFO:teuthology.orchestra.run.smithi114.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-18T00:15:12.860 INFO:teuthology.orchestra.run.smithi114.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-18T00:15:12.869 INFO:teuthology.orchestra.run.smithi114.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-18T00:15:12.871 INFO:teuthology.orchestra.run.smithi114.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-18T00:15:12.873 INFO:teuthology.orchestra.run.smithi114.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-18T00:15:12.875 INFO:teuthology.orchestra.run.smithi114.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-18T00:15:12.875 INFO:teuthology.orchestra.run.smithi114.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-18T00:15:12.878 INFO:teuthology.orchestra.run.smithi114.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-18T00:15:12.892 INFO:teuthology.orchestra.run.smithi114.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-18T00:15:12.894 INFO:teuthology.orchestra.run.smithi114.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-18T00:15:12.894 INFO:teuthology.orchestra.run.smithi114.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-18T00:15:12.895 INFO:teuthology.orchestra.run.smithi114.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-18T00:15:12.896 INFO:teuthology.orchestra.run.smithi114.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-18T00:15:12.909 INFO:teuthology.orchestra.run.smithi114.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-18T00:15:12.909 INFO:teuthology.orchestra.run.smithi114.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-18T00:15:12.909 INFO:teuthology.orchestra.run.smithi114.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-18T00:15:12.910 INFO:teuthology.orchestra.run.smithi114.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-18T00:15:12.910 INFO:teuthology.orchestra.run.smithi114.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-18T00:15:12.926 INFO:teuthology.orchestra.run.smithi114.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-18T00:15:12.929 INFO:teuthology.orchestra.run.smithi114.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-18T00:15:12.929 INFO:teuthology.orchestra.run.smithi114.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-18T00:15:12.929 INFO:teuthology.orchestra.run.smithi114.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-18T00:15:12.930 INFO:teuthology.orchestra.run.smithi114.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-18T00:15:12.934 INFO:teuthology.orchestra.run.smithi097.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-330-gc2ddf972-1jammy [3,242 kB] 2024-09-18T00:15:12.943 INFO:teuthology.orchestra.run.smithi114.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-18T00:15:12.944 INFO:teuthology.orchestra.run.smithi114.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-18T00:15:12.944 INFO:teuthology.orchestra.run.smithi114.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-18T00:15:12.944 INFO:teuthology.orchestra.run.smithi114.stdout:Get:36 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-330-gc2ddf972-1jammy [3,242 kB] 2024-09-18T00:15:12.945 INFO:teuthology.orchestra.run.smithi114.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-18T00:15:12.946 INFO:teuthology.orchestra.run.smithi114.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-18T00:15:12.960 INFO:teuthology.orchestra.run.smithi114.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-18T00:15:12.965 INFO:teuthology.orchestra.run.smithi114.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-18T00:15:12.965 INFO:teuthology.orchestra.run.smithi114.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-18T00:15:12.966 INFO:teuthology.orchestra.run.smithi114.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-18T00:15:12.967 INFO:teuthology.orchestra.run.smithi114.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-18T00:15:12.977 INFO:teuthology.orchestra.run.smithi114.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-18T00:15:12.979 INFO:teuthology.orchestra.run.smithi114.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-18T00:15:12.980 INFO:teuthology.orchestra.run.smithi114.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-18T00:15:12.981 INFO:teuthology.orchestra.run.smithi114.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-18T00:15:12.982 INFO:teuthology.orchestra.run.smithi114.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-18T00:15:12.994 INFO:teuthology.orchestra.run.smithi114.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-18T00:15:12.995 INFO:teuthology.orchestra.run.smithi114.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-18T00:15:12.995 INFO:teuthology.orchestra.run.smithi114.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-18T00:15:12.998 INFO:teuthology.orchestra.run.smithi114.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-18T00:15:13.021 INFO:teuthology.orchestra.run.smithi097.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-330-gc2ddf972-1jammy [719 kB] 2024-09-18T00:15:13.042 INFO:teuthology.orchestra.run.smithi114.stdout:Get:53 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-330-gc2ddf972-1jammy [719 kB] 2024-09-18T00:15:13.051 INFO:teuthology.orchestra.run.smithi114.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-18T00:15:13.052 INFO:teuthology.orchestra.run.smithi114.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-18T00:15:13.052 INFO:teuthology.orchestra.run.smithi097.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-330-gc2ddf972-1jammy [356 kB] 2024-09-18T00:15:13.053 INFO:teuthology.orchestra.run.smithi114.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-18T00:15:13.053 INFO:teuthology.orchestra.run.smithi114.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-18T00:15:13.054 INFO:teuthology.orchestra.run.smithi114.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-18T00:15:13.060 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-330-gc2ddf972-1jammy [32.8 kB] 2024-09-18T00:15:13.061 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-330-gc2ddf972-1jammy [180 kB] 2024-09-18T00:15:13.065 INFO:teuthology.orchestra.run.smithi097.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-330-gc2ddf972-1jammy [67.8 kB] 2024-09-18T00:15:13.067 INFO:teuthology.orchestra.run.smithi097.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-330-gc2ddf972-1jammy [333 kB] 2024-09-18T00:15:13.071 INFO:teuthology.orchestra.run.smithi114.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-18T00:15:13.073 INFO:teuthology.orchestra.run.smithi114.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-18T00:15:13.073 INFO:teuthology.orchestra.run.smithi114.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-18T00:15:13.073 INFO:teuthology.orchestra.run.smithi097.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-330-gc2ddf972-1jammy [6,875 kB] 2024-09-18T00:15:13.074 INFO:teuthology.orchestra.run.smithi114.stdout:Get:62 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-330-gc2ddf972-1jammy [356 kB] 2024-09-18T00:15:13.082 INFO:teuthology.orchestra.run.smithi114.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-330-gc2ddf972-1jammy [32.8 kB] 2024-09-18T00:15:13.083 INFO:teuthology.orchestra.run.smithi114.stdout:Get:64 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-330-gc2ddf972-1jammy [180 kB] 2024-09-18T00:15:13.087 INFO:teuthology.orchestra.run.smithi114.stdout:Get:65 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-330-gc2ddf972-1jammy [67.8 kB] 2024-09-18T00:15:13.088 INFO:teuthology.orchestra.run.smithi114.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-18T00:15:13.088 INFO:teuthology.orchestra.run.smithi114.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-18T00:15:13.089 INFO:teuthology.orchestra.run.smithi114.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-18T00:15:13.089 INFO:teuthology.orchestra.run.smithi114.stdout:Get:69 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-330-gc2ddf972-1jammy [333 kB] 2024-09-18T00:15:13.089 INFO:teuthology.orchestra.run.smithi114.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-18T00:15:13.089 INFO:teuthology.orchestra.run.smithi114.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-18T00:15:13.090 INFO:teuthology.orchestra.run.smithi114.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-18T00:15:13.095 INFO:teuthology.orchestra.run.smithi114.stdout:Get:73 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-330-gc2ddf972-1jammy [6,875 kB] 2024-09-18T00:15:13.105 INFO:teuthology.orchestra.run.smithi114.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-18T00:15:13.106 INFO:teuthology.orchestra.run.smithi114.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-18T00:15:13.109 INFO:teuthology.orchestra.run.smithi114.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-18T00:15:13.122 INFO:teuthology.orchestra.run.smithi114.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-18T00:15:13.123 INFO:teuthology.orchestra.run.smithi114.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-18T00:15:13.124 INFO:teuthology.orchestra.run.smithi114.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-18T00:15:13.127 INFO:teuthology.orchestra.run.smithi114.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-18T00:15:13.130 INFO:teuthology.orchestra.run.smithi114.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-18T00:15:13.130 INFO:teuthology.orchestra.run.smithi114.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-18T00:15:13.131 INFO:teuthology.orchestra.run.smithi114.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-18T00:15:13.139 INFO:teuthology.orchestra.run.smithi114.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-18T00:15:13.140 INFO:teuthology.orchestra.run.smithi114.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-18T00:15:13.142 INFO:teuthology.orchestra.run.smithi114.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-18T00:15:13.156 INFO:teuthology.orchestra.run.smithi114.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-18T00:15:13.157 INFO:teuthology.orchestra.run.smithi114.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-18T00:15:13.157 INFO:teuthology.orchestra.run.smithi114.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-18T00:15:13.159 INFO:teuthology.orchestra.run.smithi114.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-18T00:15:13.159 INFO:teuthology.orchestra.run.smithi114.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-18T00:15:13.165 INFO:teuthology.orchestra.run.smithi114.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-18T00:15:13.166 INFO:teuthology.orchestra.run.smithi114.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-18T00:15:13.173 INFO:teuthology.orchestra.run.smithi114.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-18T00:15:13.175 INFO:teuthology.orchestra.run.smithi114.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-18T00:15:13.176 INFO:teuthology.orchestra.run.smithi114.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-18T00:15:13.188 INFO:teuthology.orchestra.run.smithi097.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-18T00:15:13.200 INFO:teuthology.orchestra.run.smithi097.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-18T00:15:13.282 INFO:teuthology.orchestra.run.smithi097.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-18T00:15:13.307 INFO:teuthology.orchestra.run.smithi097.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-330-gc2ddf972-1jammy [112 kB] 2024-09-18T00:15:13.310 INFO:teuthology.orchestra.run.smithi097.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-330-gc2ddf972-1jammy [472 kB] 2024-09-18T00:15:13.319 INFO:teuthology.orchestra.run.smithi097.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-330-gc2ddf972-1jammy [25.9 MB] 2024-09-18T00:15:13.336 INFO:teuthology.orchestra.run.smithi114.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-330-gc2ddf972-1jammy [112 kB] 2024-09-18T00:15:13.338 INFO:teuthology.orchestra.run.smithi114.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-330-gc2ddf972-1jammy [472 kB] 2024-09-18T00:15:13.348 INFO:teuthology.orchestra.run.smithi114.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-330-gc2ddf972-1jammy [25.9 MB] 2024-09-18T00:15:13.511 INFO:teuthology.orchestra.run.smithi097.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-18T00:15:13.522 INFO:teuthology.orchestra.run.smithi097.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-18T00:15:13.551 INFO:teuthology.orchestra.run.smithi097.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-18T00:15:13.560 INFO:teuthology.orchestra.run.smithi097.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-18T00:15:13.561 INFO:teuthology.orchestra.run.smithi097.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-18T00:15:13.562 INFO:teuthology.orchestra.run.smithi097.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-18T00:15:13.580 INFO:teuthology.orchestra.run.smithi097.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-18T00:15:13.584 INFO:teuthology.orchestra.run.smithi097.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-18T00:15:13.589 INFO:teuthology.orchestra.run.smithi097.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-18T00:15:13.593 INFO:teuthology.orchestra.run.smithi097.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-18T00:15:13.673 INFO:teuthology.orchestra.run.smithi097.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-18T00:15:13.677 INFO:teuthology.orchestra.run.smithi097.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-18T00:15:13.681 INFO:teuthology.orchestra.run.smithi097.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-18T00:15:13.684 INFO:teuthology.orchestra.run.smithi097.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-18T00:15:13.685 INFO:teuthology.orchestra.run.smithi097.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-18T00:15:13.686 INFO:teuthology.orchestra.run.smithi097.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-18T00:15:13.688 INFO:teuthology.orchestra.run.smithi097.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-18T00:15:13.689 INFO:teuthology.orchestra.run.smithi097.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-18T00:15:13.689 INFO:teuthology.orchestra.run.smithi097.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-18T00:15:13.756 INFO:teuthology.orchestra.run.smithi097.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-18T00:15:13.772 INFO:teuthology.orchestra.run.smithi097.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-18T00:15:13.773 INFO:teuthology.orchestra.run.smithi097.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-18T00:15:13.773 INFO:teuthology.orchestra.run.smithi097.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-18T00:15:13.779 INFO:teuthology.orchestra.run.smithi097.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-18T00:15:13.780 INFO:teuthology.orchestra.run.smithi097.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-18T00:15:13.838 INFO:teuthology.orchestra.run.smithi097.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-18T00:15:13.839 INFO:teuthology.orchestra.run.smithi097.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-18T00:15:13.840 INFO:teuthology.orchestra.run.smithi097.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-18T00:15:13.841 INFO:teuthology.orchestra.run.smithi097.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-18T00:15:13.919 INFO:teuthology.orchestra.run.smithi097.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-18T00:15:13.921 INFO:teuthology.orchestra.run.smithi097.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-18T00:15:13.923 INFO:teuthology.orchestra.run.smithi097.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-18T00:15:13.924 INFO:teuthology.orchestra.run.smithi097.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-18T00:15:13.935 INFO:teuthology.orchestra.run.smithi097.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-18T00:15:14.001 INFO:teuthology.orchestra.run.smithi097.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-18T00:15:14.001 INFO:teuthology.orchestra.run.smithi097.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-18T00:15:14.002 INFO:teuthology.orchestra.run.smithi097.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-18T00:15:14.005 INFO:teuthology.orchestra.run.smithi097.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-18T00:15:14.009 INFO:teuthology.orchestra.run.smithi097.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-18T00:15:14.082 INFO:teuthology.orchestra.run.smithi097.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-18T00:15:14.084 INFO:teuthology.orchestra.run.smithi097.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-18T00:15:14.086 INFO:teuthology.orchestra.run.smithi097.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-18T00:15:14.090 INFO:teuthology.orchestra.run.smithi097.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-18T00:15:14.091 INFO:teuthology.orchestra.run.smithi097.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-18T00:15:14.163 INFO:teuthology.orchestra.run.smithi097.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-18T00:15:14.222 INFO:teuthology.orchestra.run.smithi097.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-18T00:15:14.264 INFO:teuthology.orchestra.run.smithi097.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-330-gc2ddf972-1jammy [5,145 kB] 2024-09-18T00:15:14.312 INFO:teuthology.orchestra.run.smithi114.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-330-gc2ddf972-1jammy [5,145 kB] 2024-09-18T00:15:14.343 INFO:teuthology.orchestra.run.smithi097.stdout:Get:64 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-330-gc2ddf972-1jammy [245 kB] 2024-09-18T00:15:14.346 INFO:teuthology.orchestra.run.smithi097.stdout:Get:65 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-330-gc2ddf972-1jammy [124 kB] 2024-09-18T00:15:14.347 INFO:teuthology.orchestra.run.smithi097.stdout:Get:66 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-330-gc2ddf972-1jammy [1,555 kB] 2024-09-18T00:15:14.378 INFO:teuthology.orchestra.run.smithi097.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-330-gc2ddf972-1jammy [6,143 kB] 2024-09-18T00:15:14.380 INFO:teuthology.orchestra.run.smithi097.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-18T00:15:14.381 INFO:teuthology.orchestra.run.smithi097.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-18T00:15:14.384 INFO:teuthology.orchestra.run.smithi097.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-18T00:15:14.384 INFO:teuthology.orchestra.run.smithi097.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-18T00:15:14.387 INFO:teuthology.orchestra.run.smithi097.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-18T00:15:14.396 INFO:teuthology.orchestra.run.smithi114.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-330-gc2ddf972-1jammy [245 kB] 2024-09-18T00:15:14.420 INFO:teuthology.orchestra.run.smithi114.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-330-gc2ddf972-1jammy [124 kB] 2024-09-18T00:15:14.421 INFO:teuthology.orchestra.run.smithi097.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-18T00:15:14.422 INFO:teuthology.orchestra.run.smithi114.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-330-gc2ddf972-1jammy [1,555 kB] 2024-09-18T00:15:14.455 INFO:teuthology.orchestra.run.smithi097.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-18T00:15:14.455 INFO:teuthology.orchestra.run.smithi097.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-18T00:15:14.472 INFO:teuthology.orchestra.run.smithi114.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-330-gc2ddf972-1jammy [6,143 kB] 2024-09-18T00:15:14.496 INFO:teuthology.orchestra.run.smithi097.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-18T00:15:14.496 INFO:teuthology.orchestra.run.smithi097.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-18T00:15:14.546 INFO:teuthology.orchestra.run.smithi097.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-18T00:15:14.547 INFO:teuthology.orchestra.run.smithi097.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-18T00:15:14.547 INFO:teuthology.orchestra.run.smithi097.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-18T00:15:14.548 INFO:teuthology.orchestra.run.smithi097.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-18T00:15:14.549 INFO:teuthology.orchestra.run.smithi097.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-18T00:15:14.550 INFO:teuthology.orchestra.run.smithi097.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-18T00:15:14.557 INFO:teuthology.orchestra.run.smithi097.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-18T00:15:14.579 INFO:teuthology.orchestra.run.smithi097.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-330-gc2ddf972-1jammy [22.8 MB] 2024-09-18T00:15:14.627 INFO:teuthology.orchestra.run.smithi097.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-18T00:15:14.630 INFO:teuthology.orchestra.run.smithi097.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-18T00:15:14.632 INFO:teuthology.orchestra.run.smithi097.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-18T00:15:14.689 INFO:teuthology.orchestra.run.smithi114.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-330-gc2ddf972-1jammy [22.8 MB] 2024-09-18T00:15:14.709 INFO:teuthology.orchestra.run.smithi097.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-18T00:15:14.715 INFO:teuthology.orchestra.run.smithi097.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-18T00:15:14.717 INFO:teuthology.orchestra.run.smithi097.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-18T00:15:14.718 INFO:teuthology.orchestra.run.smithi097.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-18T00:15:14.729 INFO:teuthology.orchestra.run.smithi097.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-18T00:15:14.731 INFO:teuthology.orchestra.run.smithi097.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-18T00:15:14.738 INFO:teuthology.orchestra.run.smithi097.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-18T00:15:14.790 INFO:teuthology.orchestra.run.smithi097.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-18T00:15:14.791 INFO:teuthology.orchestra.run.smithi097.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-18T00:15:14.792 INFO:teuthology.orchestra.run.smithi097.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-18T00:15:14.871 INFO:teuthology.orchestra.run.smithi097.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-18T00:15:14.873 INFO:teuthology.orchestra.run.smithi097.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-18T00:15:14.891 INFO:teuthology.orchestra.run.smithi097.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-18T00:15:14.891 INFO:teuthology.orchestra.run.smithi097.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-18T00:15:14.892 INFO:teuthology.orchestra.run.smithi097.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-18T00:15:14.897 INFO:teuthology.orchestra.run.smithi097.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-18T00:15:14.898 INFO:teuthology.orchestra.run.smithi097.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-18T00:15:15.350 INFO:teuthology.orchestra.run.smithi097.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-330-gc2ddf972-1jammy [14.1 kB] 2024-09-18T00:15:15.351 INFO:teuthology.orchestra.run.smithi097.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-330-gc2ddf972-1jammy [877 kB] 2024-09-18T00:15:15.366 INFO:teuthology.orchestra.run.smithi097.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-330-gc2ddf972-1jammy [2,462 kB] 2024-09-18T00:15:15.407 INFO:teuthology.orchestra.run.smithi097.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-330-gc2ddf972-1jammy [794 kB] 2024-09-18T00:15:15.428 INFO:teuthology.orchestra.run.smithi097.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-330-gc2ddf972-1jammy [152 kB] 2024-09-18T00:15:15.431 INFO:teuthology.orchestra.run.smithi097.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-330-gc2ddf972-1jammy [2,275 kB] 2024-09-18T00:15:15.494 INFO:teuthology.orchestra.run.smithi097.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-330-gc2ddf972-1jammy [8,625 kB] 2024-09-18T00:15:15.569 INFO:teuthology.orchestra.run.smithi114.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-330-gc2ddf972-1jammy [14.1 kB] 2024-09-18T00:15:15.569 INFO:teuthology.orchestra.run.smithi114.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-330-gc2ddf972-1jammy [877 kB] 2024-09-18T00:15:15.586 INFO:teuthology.orchestra.run.smithi114.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-330-gc2ddf972-1jammy [2,462 kB] 2024-09-18T00:15:15.628 INFO:teuthology.orchestra.run.smithi114.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-330-gc2ddf972-1jammy [794 kB] 2024-09-18T00:15:15.638 INFO:teuthology.orchestra.run.smithi114.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-330-gc2ddf972-1jammy [152 kB] 2024-09-18T00:15:15.641 INFO:teuthology.orchestra.run.smithi114.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-330-gc2ddf972-1jammy [2,275 kB] 2024-09-18T00:15:15.727 INFO:teuthology.orchestra.run.smithi114.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-330-gc2ddf972-1jammy [8,625 kB] 2024-09-18T00:15:15.734 INFO:teuthology.orchestra.run.smithi097.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-330-gc2ddf972-1jammy [14.2 kB] 2024-09-18T00:15:15.734 INFO:teuthology.orchestra.run.smithi097.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-330-gc2ddf972-1jammy [51.4 MB] 2024-09-18T00:15:16.021 INFO:teuthology.orchestra.run.smithi114.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-330-gc2ddf972-1jammy [14.2 kB] 2024-09-18T00:15:16.021 INFO:teuthology.orchestra.run.smithi114.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-330-gc2ddf972-1jammy [51.4 MB] 2024-09-18T00:15:17.602 INFO:teuthology.orchestra.run.smithi097.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-330-gc2ddf972-1jammy [130 kB] 2024-09-18T00:15:17.605 INFO:teuthology.orchestra.run.smithi097.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-330-gc2ddf972-1jammy [39.0 kB] 2024-09-18T00:15:17.606 INFO:teuthology.orchestra.run.smithi097.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-330-gc2ddf972-1jammy [13.6 MB] 2024-09-18T00:15:17.851 INFO:teuthology.orchestra.run.smithi114.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-330-gc2ddf972-1jammy [130 kB] 2024-09-18T00:15:17.853 INFO:teuthology.orchestra.run.smithi097.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-330-gc2ddf972-1jammy [91.4 kB] 2024-09-18T00:15:17.854 INFO:teuthology.orchestra.run.smithi114.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-330-gc2ddf972-1jammy [39.0 kB] 2024-09-18T00:15:17.855 INFO:teuthology.orchestra.run.smithi114.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-330-gc2ddf972-1jammy [13.6 MB] 2024-09-18T00:15:18.044 INFO:teuthology.orchestra.run.smithi114.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-330-gc2ddf972-1jammy [91.4 kB] 2024-09-18T00:15:18.424 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 181 MB in 5s (34.4 MB/s) 2024-09-18T00:15:18.586 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-18T00:15:18.615 INFO:teuthology.orchestra.run.smithi114.stdout:Fetched 181 MB in 5s (33.3 MB/s) 2024-09-18T00:15:18.685 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-18T00:15:18.691 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-18T00:15:18.745 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-18T00:15:18.784 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-18T00:15:18.843 INFO:teuthology.orchestra.run.smithi114.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-18T00:15:18.848 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-18T00:15:18.919 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-18T00:15:19.121 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-18T00:15:19.148 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-18T00:15:19.187 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-18T00:15:19.256 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-18T00:15:19.282 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-18T00:15:19.322 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-18T00:15:19.465 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-18T00:15:19.492 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-18T00:15:19.523 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-18T00:15:19.658 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-18T00:15:19.685 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-18T00:15:19.725 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-18T00:15:19.960 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-18T00:15:19.987 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-18T00:15:20.036 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:15:20.145 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-18T00:15:20.171 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-18T00:15:20.211 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:15:20.405 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-18T00:15:20.432 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-18T00:15:20.471 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:15:20.573 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-18T00:15:20.599 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-18T00:15:20.631 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:15:20.774 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-18T00:15:20.801 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-18T00:15:20.907 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:15:20.960 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-18T00:15:20.985 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-18T00:15:21.026 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:15:21.286 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-18T00:15:21.313 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-18T00:15:21.352 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-18T00:15:21.404 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-18T00:15:21.430 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-18T00:15:21.470 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-18T00:15:21.672 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librados2. 2024-09-18T00:15:21.698 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../007-librados2_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:21.738 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librados2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:21.815 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package librados2. 2024-09-18T00:15:21.841 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../007-librados2_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:21.890 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking librados2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:22.149 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librbd1. 2024-09-18T00:15:22.176 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../008-librbd1_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:22.216 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librbd1 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:22.376 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package librbd1. 2024-09-18T00:15:22.402 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../008-librbd1_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:22.443 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking librbd1 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:22.644 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libcephfs2. 2024-09-18T00:15:22.670 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:22.710 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libcephfs2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:22.895 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libcephfs2. 2024-09-18T00:15:22.921 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:22.962 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libcephfs2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:22.988 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rados. 2024-09-18T00:15:23.014 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../010-python3-rados_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:23.054 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rados (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:23.248 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-rados. 2024-09-18T00:15:23.274 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../010-python3-rados_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:23.314 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-rados (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:23.373 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-18T00:15:23.400 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:15:23.440 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-ceph-argparse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:23.692 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-18T00:15:23.701 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cephfs. 2024-09-18T00:15:23.718 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:15:23.728 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:23.759 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cephfs (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:23.767 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-ceph-argparse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:24.045 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-18T00:15:24.069 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-cephfs. 2024-09-18T00:15:24.072 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:15:24.085 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:24.112 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-ceph-common (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:24.128 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-cephfs (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:24.431 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-18T00:15:24.439 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-18T00:15:24.458 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-18T00:15:24.465 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:15:24.498 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-18T00:15:24.506 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-ceph-common (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:24.818 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-prettytable. 2024-09-18T00:15:24.834 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-18T00:15:24.844 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-18T00:15:24.860 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-18T00:15:24.884 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-18T00:15:24.900 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-18T00:15:25.145 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rbd. 2024-09-18T00:15:25.172 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:25.211 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rbd (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:25.253 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-prettytable. 2024-09-18T00:15:25.281 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-18T00:15:25.328 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-18T00:15:25.548 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-18T00:15:25.574 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-18T00:15:25.614 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-rbd. 2024-09-18T00:15:25.614 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-18T00:15:25.629 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:25.672 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-rbd (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:26.017 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-18T00:15:26.043 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-18T00:15:26.083 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-18T00:15:26.166 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-18T00:15:26.182 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-18T00:15:26.225 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-18T00:15:26.419 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-18T00:15:26.446 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-18T00:15:26.486 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-18T00:15:26.520 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-18T00:15:26.546 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-18T00:15:26.586 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-18T00:15:26.822 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-18T00:15:26.848 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-18T00:15:26.897 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-18T00:15:26.914 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package lua5.1. 2024-09-18T00:15:26.941 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-18T00:15:26.981 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-18T00:15:27.250 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package lua5.1. 2024-09-18T00:15:27.276 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-18T00:15:27.317 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-18T00:15:27.325 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package lua-any. 2024-09-18T00:15:27.341 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-18T00:15:27.375 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-18T00:15:27.586 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package zip. 2024-09-18T00:15:27.613 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-18T00:15:27.620 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package lua-any. 2024-09-18T00:15:27.646 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-18T00:15:27.653 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking zip (3.0-12build2) ... 2024-09-18T00:15:27.686 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-18T00:15:27.956 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package zip. 2024-09-18T00:15:27.964 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package unzip. 2024-09-18T00:15:27.982 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-18T00:15:27.991 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-18T00:15:28.031 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking zip (3.0-12build2) ... 2024-09-18T00:15:28.031 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-18T00:15:28.401 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package luarocks. 2024-09-18T00:15:28.428 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-18T00:15:28.434 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package unzip. 2024-09-18T00:15:28.460 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-18T00:15:28.468 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-18T00:15:28.509 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-18T00:15:28.834 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librgw2. 2024-09-18T00:15:28.854 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package luarocks. 2024-09-18T00:15:28.860 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../025-librgw2_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:28.880 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-18T00:15:28.900 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librgw2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:28.920 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-18T00:15:29.253 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package librgw2. 2024-09-18T00:15:29.279 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../025-librgw2_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:29.311 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking librgw2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:29.419 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rgw. 2024-09-18T00:15:29.435 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:29.477 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rgw (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:29.805 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-18T00:15:29.832 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-18T00:15:29.855 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-rgw. 2024-09-18T00:15:29.871 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:29.872 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-18T00:15:29.939 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-rgw (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:30.184 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libradosstriper1. 2024-09-18T00:15:30.210 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:30.249 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libradosstriper1 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:30.300 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-18T00:15:30.327 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-18T00:15:30.358 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-18T00:15:30.620 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-common. 2024-09-18T00:15:30.646 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../029-ceph-common_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:30.661 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libradosstriper1. 2024-09-18T00:15:30.686 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-common (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:30.687 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:30.728 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libradosstriper1 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:31.316 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-common. 2024-09-18T00:15:31.342 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../029-ceph-common_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:31.481 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-common (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:32.091 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-base. 2024-09-18T00:15:32.118 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../030-ceph-base_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:32.167 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-base (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:32.679 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-18T00:15:32.705 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-18T00:15:32.745 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-18T00:15:32.819 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-base. 2024-09-18T00:15:32.846 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../030-ceph-base_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:32.887 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-base (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:33.048 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cheroot. 2024-09-18T00:15:33.074 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-18T00:15:33.114 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-18T00:15:33.409 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-18T00:15:33.424 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-18T00:15:33.435 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-18T00:15:33.450 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-18T00:15:33.475 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-18T00:15:33.490 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-18T00:15:33.776 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-cheroot. 2024-09-18T00:15:33.778 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-18T00:15:33.802 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-18T00:15:33.804 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-18T00:15:33.843 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-18T00:15:33.845 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-18T00:15:34.122 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-18T00:15:34.149 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-18T00:15:34.171 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-18T00:15:34.180 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-18T00:15:34.197 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-18T00:15:34.237 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-18T00:15:34.549 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-18T00:15:34.566 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-tempora. 2024-09-18T00:15:34.575 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-18T00:15:34.593 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-18T00:15:34.615 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-18T00:15:34.633 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-18T00:15:34.868 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-18T00:15:34.883 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-18T00:15:34.918 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-18T00:15:34.936 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-portend. 2024-09-18T00:15:34.962 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-18T00:15:35.002 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-18T00:15:35.212 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-tempora. 2024-09-18T00:15:35.238 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-18T00:15:35.279 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-18T00:15:35.305 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-18T00:15:35.332 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-18T00:15:35.372 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-18T00:15:35.582 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-portend. 2024-09-18T00:15:35.609 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-18T00:15:35.633 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-18T00:15:35.648 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-18T00:15:35.648 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-18T00:15:35.691 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-18T00:15:35.943 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-18T00:15:35.969 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-18T00:15:36.009 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-18T00:15:36.045 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-natsort. 2024-09-18T00:15:36.071 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-18T00:15:36.111 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-18T00:15:36.254 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-18T00:15:36.269 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-18T00:15:36.320 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-18T00:15:36.573 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-logutils. 2024-09-18T00:15:36.599 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-18T00:15:36.631 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-18T00:15:36.676 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-natsort. 2024-09-18T00:15:36.702 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-18T00:15:36.742 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-18T00:15:36.925 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-18T00:15:36.951 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-18T00:15:36.992 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-18T00:15:37.054 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-logutils. 2024-09-18T00:15:37.080 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-18T00:15:37.120 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-18T00:15:37.303 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-mako. 2024-09-18T00:15:37.329 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-18T00:15:37.369 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-18T00:15:37.373 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-18T00:15:37.388 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-18T00:15:37.423 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-18T00:15:37.689 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-18T00:15:37.716 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-18T00:15:37.717 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-mako. 2024-09-18T00:15:37.744 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-18T00:15:37.756 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-18T00:15:37.784 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-18T00:15:38.075 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-18T00:15:38.101 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-18T00:15:38.133 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-18T00:15:38.145 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-18T00:15:38.172 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-18T00:15:38.212 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-18T00:15:38.420 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-webob. 2024-09-18T00:15:38.446 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-18T00:15:38.486 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-18T00:15:38.515 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-18T00:15:38.541 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-18T00:15:38.573 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-18T00:15:38.822 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-waitress. 2024-09-18T00:15:38.849 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-18T00:15:38.868 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-webob. 2024-09-18T00:15:38.891 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-18T00:15:38.894 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-18T00:15:38.934 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-18T00:15:39.200 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-tempita. 2024-09-18T00:15:39.226 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-18T00:15:39.266 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-18T00:15:39.279 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-waitress. 2024-09-18T00:15:39.305 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-18T00:15:39.349 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-18T00:15:39.569 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-paste. 2024-09-18T00:15:39.596 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-18T00:15:39.636 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-18T00:15:39.656 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-tempita. 2024-09-18T00:15:39.683 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-18T00:15:39.723 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-18T00:15:39.939 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-18T00:15:39.954 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-18T00:15:39.989 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-18T00:15:40.026 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-paste. 2024-09-18T00:15:40.053 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-18T00:15:40.092 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-18T00:15:40.283 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-18T00:15:40.310 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-18T00:15:40.350 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-18T00:15:40.437 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-18T00:15:40.464 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-18T00:15:40.504 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-18T00:15:40.628 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-webtest. 2024-09-18T00:15:40.656 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-18T00:15:40.695 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-18T00:15:40.798 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-18T00:15:40.825 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-18T00:15:40.865 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-18T00:15:40.981 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pecan. 2024-09-18T00:15:41.008 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-18T00:15:41.039 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-18T00:15:41.134 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-webtest. 2024-09-18T00:15:41.150 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-18T00:15:41.184 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-18T00:15:41.377 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-certifi. 2024-09-18T00:15:41.404 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-18T00:15:41.445 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-18T00:15:41.496 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pecan. 2024-09-18T00:15:41.522 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-18T00:15:41.562 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-18T00:15:41.897 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-idna. 2024-09-18T00:15:41.909 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-certifi. 2024-09-18T00:15:41.923 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-18T00:15:41.935 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-18T00:15:41.963 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-18T00:15:41.975 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-18T00:15:42.274 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-urllib3. 2024-09-18T00:15:42.295 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-idna. 2024-09-18T00:15:42.301 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-18T00:15:42.321 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-18T00:15:42.340 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-18T00:15:42.361 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-18T00:15:42.644 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-requests. 2024-09-18T00:15:42.671 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-18T00:15:42.672 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-urllib3. 2024-09-18T00:15:42.699 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-18T00:15:42.710 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-18T00:15:42.739 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-18T00:15:43.022 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-18T00:15:43.042 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-requests. 2024-09-18T00:15:43.048 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-18T00:15:43.068 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-18T00:15:43.088 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-18T00:15:43.117 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-18T00:15:43.357 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-18T00:15:43.373 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:15:43.378 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-18T00:15:43.394 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-18T00:15:43.415 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-modules-core (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:43.428 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-18T00:15:43.697 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-18T00:15:43.713 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:15:43.739 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-18T00:15:43.747 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr-modules-core (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:43.766 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:43.805 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:44.100 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-18T00:15:44.102 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-18T00:15:44.127 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-18T00:15:44.129 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:44.166 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-18T00:15:44.169 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:44.477 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-openssl. 2024-09-18T00:15:44.497 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-18T00:15:44.504 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-18T00:15:44.523 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-18T00:15:44.544 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-18T00:15:44.563 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-18T00:15:44.833 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-openssl. 2024-09-18T00:15:44.859 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-18T00:15:44.890 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-18T00:15:44.897 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr. 2024-09-18T00:15:44.924 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:44.963 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:45.227 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr. 2024-09-18T00:15:45.254 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:45.293 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:45.367 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mon. 2024-09-18T00:15:45.394 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:45.433 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mon (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:45.697 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mon. 2024-09-18T00:15:45.724 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:46.080 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mon (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:46.837 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-osd. 2024-09-18T00:15:46.865 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:46.903 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-osd (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:47.367 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-osd. 2024-09-18T00:15:47.394 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:47.425 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-osd (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:47.923 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph. 2024-09-18T00:15:47.950 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../066-ceph_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:47.990 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:48.259 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-fuse. 2024-09-18T00:15:48.286 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:48.326 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-fuse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:48.470 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph. 2024-09-18T00:15:48.486 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../066-ceph_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:48.528 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:48.695 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mds. 2024-09-18T00:15:48.722 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:48.787 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mds (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:48.873 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-fuse. 2024-09-18T00:15:48.899 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:48.939 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-fuse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:49.200 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package cephadm. 2024-09-18T00:15:49.225 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../069-cephadm_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:49.264 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking cephadm (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:49.309 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mds. 2024-09-18T00:15:49.336 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:49.375 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mds (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:49.551 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-18T00:15:49.577 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-18T00:15:49.617 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-18T00:15:49.754 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package cephadm. 2024-09-18T00:15:49.780 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../069-cephadm_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:15:49.820 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking cephadm (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:49.920 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-18T00:15:49.947 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-18T00:15:49.986 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-18T00:15:50.140 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-18T00:15:50.166 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-18T00:15:50.206 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-18T00:15:50.509 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-18T00:15:50.535 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-18T00:15:50.575 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-18T00:15:50.941 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-babel. 2024-09-18T00:15:50.968 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-18T00:15:51.007 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-18T00:15:51.327 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jinja2. 2024-09-18T00:15:51.354 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-18T00:15:51.393 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-18T00:15:51.539 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-babel. 2024-09-18T00:15:51.554 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-18T00:15:51.597 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-18T00:15:51.763 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-18T00:15:51.790 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:15:51.917 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-jinja2. 2024-09-18T00:15:51.943 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-18T00:15:51.983 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-18T00:15:52.013 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-cephadm (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:52.319 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-18T00:15:52.346 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:15:52.383 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-18T00:15:52.386 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr-cephadm (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:52.410 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-18T00:15:52.449 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-18T00:15:52.723 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-18T00:15:52.749 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-18T00:15:52.752 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-routes. 2024-09-18T00:15:52.779 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-18T00:15:52.789 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-18T00:15:52.819 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-18T00:15:53.092 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-routes. 2024-09-18T00:15:53.118 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-18T00:15:53.158 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-18T00:15:53.164 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-18T00:15:53.191 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:15:53.230 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-dashboard (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:53.503 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-18T00:15:53.530 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:15:53.570 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr-dashboard (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:54.451 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-18T00:15:54.467 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-18T00:15:54.510 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-18T00:15:54.841 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-18T00:15:54.857 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-18T00:15:54.899 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-18T00:15:54.972 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-joblib. 2024-09-18T00:15:54.998 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-18T00:15:55.029 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-18T00:15:55.392 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-18T00:15:55.418 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-18T00:15:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-joblib. 2024-09-18T00:15:55.458 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-18T00:15:55.463 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-18T00:15:55.503 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-18T00:15:55.761 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-sklearn. 2024-09-18T00:15:55.788 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-18T00:15:55.823 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-18T00:15:55.827 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-18T00:15:55.850 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-18T00:15:55.881 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-18T00:15:56.209 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-sklearn. 2024-09-18T00:15:56.236 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-18T00:15:56.275 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-18T00:15:56.502 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-18T00:15:56.527 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:15:56.654 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:56.992 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-18T00:15:57.020 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:15:57.152 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:15:57.608 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cachetools. 2024-09-18T00:15:57.634 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-18T00:15:57.674 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-18T00:15:57.977 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-18T00:15:58.003 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-18T00:15:58.043 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-18T00:15:58.098 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-cachetools. 2024-09-18T00:15:58.125 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-18T00:15:58.164 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-18T00:15:58.388 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-18T00:15:58.415 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-18T00:15:58.417 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-18T00:15:58.444 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-18T00:15:58.454 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-18T00:15:58.475 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-18T00:15:58.749 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rsa. 2024-09-18T00:15:58.770 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-18T00:15:58.776 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-18T00:15:58.797 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-18T00:15:58.815 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-18T00:15:58.836 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-18T00:15:59.078 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-google-auth. 2024-09-18T00:15:59.105 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-18T00:15:59.114 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-rsa. 2024-09-18T00:15:59.141 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-18T00:15:59.144 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-18T00:15:59.181 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-18T00:15:59.456 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-18T00:15:59.483 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-18T00:15:59.501 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-google-auth. 2024-09-18T00:15:59.528 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-18T00:15:59.539 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-18T00:15:59.568 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-18T00:15:59.850 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-websocket. 2024-09-18T00:15:59.877 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-18T00:15:59.888 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-18T00:15:59.915 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-18T00:15:59.916 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-18T00:15:59.954 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-18T00:16:00.203 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-18T00:16:00.230 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-18T00:16:00.249 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-websocket. 2024-09-18T00:16:00.261 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-18T00:16:00.276 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-18T00:16:00.315 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-18T00:16:00.610 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-18T00:16:00.637 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-18T00:16:00.677 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-18T00:16:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-18T00:16:00.907 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:16:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:01.227 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-18T00:16:01.255 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-18T00:16:01.285 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-18T00:16:01.316 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-18T00:16:01.343 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:16:01.382 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:01.605 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-18T00:16:01.633 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-18T00:16:01.672 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-18T00:16:01.701 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-18T00:16:01.729 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-18T00:16:01.776 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-18T00:16:01.933 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package jq. 2024-09-18T00:16:01.948 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-18T00:16:01.991 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-18T00:16:02.079 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-18T00:16:02.107 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-18T00:16:02.137 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-18T00:16:02.235 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package socat. 2024-09-18T00:16:02.251 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-18T00:16:02.335 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-18T00:16:02.390 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package jq. 2024-09-18T00:16:02.417 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-18T00:16:02.457 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-18T00:16:02.701 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package socat. 2024-09-18T00:16:02.728 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-18T00:16:02.767 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-18T00:16:02.780 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package xmlstarlet. 2024-09-18T00:16:02.807 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-18T00:16:02.846 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-18T00:16:03.046 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package xmlstarlet. 2024-09-18T00:16:03.061 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-18T00:16:03.096 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-18T00:16:03.201 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-test. 2024-09-18T00:16:03.228 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../097-ceph-test_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:16:03.267 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-test (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:03.466 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-test. 2024-09-18T00:16:03.482 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../097-ceph-test_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:16:03.516 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-test (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:05.546 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-volume. 2024-09-18T00:16:05.573 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:16:05.613 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-volume (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:05.771 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-volume. 2024-09-18T00:16:05.798 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-330-gc2ddf972-1jammy_all.deb ... 2024-09-18T00:16:05.845 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-volume (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:05.916 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-18T00:16:05.933 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:16:05.975 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libcephfs-dev (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:06.166 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-18T00:16:06.193 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:16:06.241 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libcephfs-dev (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:06.294 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-18T00:16:06.323 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-18T00:16:06.360 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-18T00:16:06.636 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-18T00:16:06.663 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-18T00:16:06.707 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-18T00:16:06.710 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-18T00:16:06.735 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-18T00:16:06.773 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-18T00:16:07.057 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-18T00:16:07.084 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-18T00:16:07.094 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package nvme-cli. 2024-09-18T00:16:07.122 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-18T00:16:07.123 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-18T00:16:07.160 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-18T00:16:07.530 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package nvme-cli. 2024-09-18T00:16:07.558 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-18T00:16:07.660 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-18T00:16:07.665 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package pkg-config. 2024-09-18T00:16:07.693 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-18T00:16:07.731 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-18T00:16:07.993 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-18T00:16:08.009 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-18T00:16:08.051 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-18T00:16:08.099 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package pkg-config. 2024-09-18T00:16:08.126 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-18T00:16:08.165 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-18T00:16:08.418 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-18T00:16:08.433 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-18T00:16:08.463 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-18T00:16:08.467 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-18T00:16:08.490 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-18T00:16:08.529 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-18T00:16:08.840 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pastescript. 2024-09-18T00:16:08.868 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-18T00:16:08.906 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-18T00:16:08.913 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-18T00:16:08.940 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-18T00:16:08.979 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-18T00:16:09.226 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pluggy. 2024-09-18T00:16:09.254 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-18T00:16:09.265 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pastescript. 2024-09-18T00:16:09.281 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-18T00:16:09.292 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-18T00:16:09.314 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-18T00:16:09.568 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pluggy. 2024-09-18T00:16:09.587 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-psutil. 2024-09-18T00:16:09.595 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-18T00:16:09.615 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-18T00:16:09.626 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-18T00:16:09.645 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-18T00:16:09.938 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-psutil. 2024-09-18T00:16:09.965 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-18T00:16:09.973 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-py. 2024-09-18T00:16:10.001 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-18T00:16:10.004 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-18T00:16:10.040 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-18T00:16:10.332 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-py. 2024-09-18T00:16:10.359 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-18T00:16:10.360 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pygments. 2024-09-18T00:16:10.375 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-18T00:16:10.398 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-18T00:16:10.409 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-18T00:16:10.727 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pygments. 2024-09-18T00:16:10.754 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-18T00:16:10.793 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-18T00:16:10.847 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-18T00:16:10.875 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-18T00:16:10.913 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-18T00:16:11.208 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-toml. 2024-09-18T00:16:11.235 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-18T00:16:11.274 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-18T00:16:11.289 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-18T00:16:11.317 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-18T00:16:11.355 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-18T00:16:11.585 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pytest. 2024-09-18T00:16:11.613 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-18T00:16:11.624 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-toml. 2024-09-18T00:16:11.640 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-18T00:16:11.651 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-18T00:16:11.674 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-18T00:16:11.978 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pytest. 2024-09-18T00:16:12.005 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-simplejson. 2024-09-18T00:16:12.005 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-18T00:16:12.033 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-18T00:16:12.044 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-18T00:16:12.071 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-18T00:16:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-18T00:16:12.347 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-simplejson. 2024-09-18T00:16:12.357 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-18T00:16:12.363 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-18T00:16:12.397 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-18T00:16:12.399 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-18T00:16:12.625 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-18T00:16:12.641 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-18T00:16:12.675 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-18T00:16:13.078 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package radosgw. 2024-09-18T00:16:13.095 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../116-radosgw_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:16:13.135 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking radosgw (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:13.412 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package radosgw. 2024-09-18T00:16:13.440 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../116-radosgw_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:16:13.478 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking radosgw (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:13.947 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package rbd-fuse. 2024-09-18T00:16:13.963 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:16:14.014 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking rbd-fuse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:14.264 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package rbd-fuse. 2024-09-18T00:16:14.280 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-330-gc2ddf972-1jammy_amd64.deb ... 2024-09-18T00:16:14.297 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-18T00:16:14.323 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking rbd-fuse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:14.568 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-18T00:16:14.627 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-18T00:16:14.678 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-18T00:16:14.894 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-18T00:16:14.935 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-18T00:16:15.021 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-18T00:16:15.265 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-18T00:16:15.285 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-18T00:16:15.612 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-18T00:16:15.699 INFO:teuthology.orchestra.run.smithi097.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-18T00:16:15.762 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up cephadm (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:16.026 INFO:teuthology.orchestra.run.smithi114.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-18T00:16:16.089 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up cephadm (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:16.279 INFO:teuthology.orchestra.run.smithi097.stdout:Adding system user cephadm....done 2024-09-18T00:16:16.435 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-18T00:16:16.683 INFO:teuthology.orchestra.run.smithi114.stdout:Adding system user cephadm....done 2024-09-18T00:16:16.703 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-18T00:16:16.839 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-18T00:16:16.963 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-18T00:16:17.097 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-18T00:16:17.107 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-18T00:16:17.356 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-18T00:16:17.358 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-18T00:16:17.484 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-18T00:16:17.652 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-18T00:16:17.772 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-18T00:16:18.034 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-18T00:16:18.328 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-18T00:16:18.360 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-18T00:16:18.438 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-18T00:16:18.704 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-18T00:16:18.764 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-18T00:16:18.963 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-18T00:16:19.133 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-18T00:16:19.239 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-18T00:16:19.384 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-18T00:16:19.508 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-18T00:16:19.652 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-18T00:16:19.767 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-18T00:16:19.920 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-18T00:16:20.093 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-18T00:16:20.188 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-18T00:16:20.336 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-ceph-argparse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:20.447 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-18T00:16:20.570 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-18T00:16:20.690 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-ceph-argparse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:20.696 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-18T00:16:20.830 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-18T00:16:20.949 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-18T00:16:20.956 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-18T00:16:21.092 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-18T00:16:21.218 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-18T00:16:21.257 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-18T00:16:21.352 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-18T00:16:21.595 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-18T00:16:21.639 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-18T00:16:21.671 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-18T00:16:21.736 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-18T00:16:21.836 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-18T00:16:22.009 INFO:teuthology.orchestra.run.smithi114.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-18T00:16:22.042 INFO:teuthology.orchestra.run.smithi114.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-18T00:16:22.115 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-18T00:16:22.187 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-18T00:16:22.241 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-18T00:16:22.313 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-18T00:16:22.581 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-18T00:16:22.600 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-18T00:16:22.727 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-18T00:16:22.849 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-18T00:16:22.986 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-18T00:16:23.237 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-18T00:16:23.275 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up zip (3.0-12build2) ... 2024-09-18T00:16:23.401 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-18T00:16:23.513 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up zip (3.0-12build2) ... 2024-09-18T00:16:23.639 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-18T00:16:24.069 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-18T00:16:24.321 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-18T00:16:24.333 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-18T00:16:24.446 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-18T00:16:24.592 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-18T00:16:24.614 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-18T00:16:24.718 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-18T00:16:24.836 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-18T00:16:24.915 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-18T00:16:25.137 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-18T00:16:25.300 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-18T00:16:25.530 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-18T00:16:25.551 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-18T00:16:25.781 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-18T00:16:25.910 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-18T00:16:26.157 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-18T00:16:26.178 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-18T00:16:26.442 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-18T00:16:26.521 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-18T00:16:26.781 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-18T00:16:26.793 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-18T00:16:26.907 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-ceph-common (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:27.036 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-18T00:16:27.154 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-ceph-common (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:27.208 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-18T00:16:27.455 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-18T00:16:27.525 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-18T00:16:27.814 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-18T00:16:28.691 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-18T00:16:28.937 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-18T00:16:28.943 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:16:29.061 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-18T00:16:29.163 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:16:29.281 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-18T00:16:29.303 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-18T00:16:29.429 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-18T00:16:29.555 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-18T00:16:29.565 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-18T00:16:29.691 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-18T00:16:29.818 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-18T00:16:29.848 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-18T00:16:30.116 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-18T00:16:30.127 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-18T00:16:30.367 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:16:30.369 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-18T00:16:30.493 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-18T00:16:30.613 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:16:30.739 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-18T00:16:30.803 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-18T00:16:31.040 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-18T00:16:31.062 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-18T00:16:31.291 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-18T00:16:31.321 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-18T00:16:31.553 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-18T00:16:31.573 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-18T00:16:31.810 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-18T00:16:31.841 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-18T00:16:32.052 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-18T00:16:32.117 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-18T00:16:32.243 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-18T00:16:32.295 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-18T00:16:32.396 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-18T00:16:32.519 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-18T00:16:32.636 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-18T00:16:32.656 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-18T00:16:32.785 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-18T00:16:32.929 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-18T00:16:33.075 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-18T00:16:33.247 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-18T00:16:33.409 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-18T00:16:33.506 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-18T00:16:33.633 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-18T00:16:33.669 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-18T00:16:33.758 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-18T00:16:33.769 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-18T00:16:33.847 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-18T00:16:33.879 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-18T00:16:33.968 INFO:teuthology.orchestra.run.smithi114.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-18T00:16:34.085 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-18T00:16:34.194 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-18T00:16:34.206 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-18T00:16:34.332 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-18T00:16:34.562 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-18T00:16:34.716 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-18T00:16:34.822 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-18T00:16:34.948 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-18T00:16:34.976 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-18T00:16:35.102 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-18T00:16:35.216 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:16:35.345 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-18T00:16:35.375 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-18T00:16:35.479 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-18T00:16:35.684 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-18T00:16:35.797 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-18T00:16:35.977 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-18T00:16:36.081 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-18T00:16:36.237 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-18T00:16:36.357 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-18T00:16:36.362 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-18T00:16:36.492 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-18T00:16:36.639 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-18T00:16:36.759 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-18T00:16:36.906 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-18T00:16:37.035 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-18T00:16:37.243 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-18T00:16:37.389 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-18T00:16:37.595 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-18T00:16:37.799 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-18T00:16:37.879 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-18T00:16:38.083 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-18T00:16:38.223 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-18T00:16:38.350 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librados2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:38.436 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-18T00:16:38.563 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up librados2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:38.659 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libsqlite3-mod-ceph (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:38.681 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libsqlite3-mod-ceph (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:38.785 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-18T00:16:38.806 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-18T00:16:39.130 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libcephfs2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:39.142 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libcephfs2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:39.256 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libradosstriper1 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:39.268 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libradosstriper1 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:39.382 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-18T00:16:39.386 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-18T00:16:39.646 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up librbd1 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:39.692 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librbd1 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:39.772 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr-modules-core (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:39.809 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-modules-core (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:39.898 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-fuse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:39.919 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-fuse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:40.123 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T00:16:40.124 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T00:16:40.126 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T00:16:40.126 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T00:16:40.608 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libcephfs-dev (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:40.628 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libcephfs-dev (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:40.750 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-rados (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:40.755 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rados (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:40.877 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up librgw2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:40.880 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librgw2 (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:41.003 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-18T00:16:41.006 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-18T00:16:42.131 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-rbd (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:42.160 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rbd (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:42.261 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up rbd-fuse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:42.282 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up rbd-fuse (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:42.379 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rgw (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:42.433 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-rgw (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:42.530 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cephfs (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:42.584 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-cephfs (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:42.656 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-common (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:42.710 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-common (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:43.199 INFO:teuthology.orchestra.run.smithi097.stdout:Adding group ceph....done 2024-09-18T00:16:43.228 INFO:teuthology.orchestra.run.smithi114.stdout:Adding group ceph....done 2024-09-18T00:16:43.575 INFO:teuthology.orchestra.run.smithi097.stdout:Adding system user ceph....done 2024-09-18T00:16:43.705 INFO:teuthology.orchestra.run.smithi114.stdout:Adding system user ceph....done 2024-09-18T00:16:43.743 INFO:teuthology.orchestra.run.smithi097.stdout:Setting system user ceph properties....done 2024-09-18T00:16:43.763 INFO:teuthology.orchestra.run.smithi097.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-18T00:16:43.899 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-18T00:16:43.907 INFO:teuthology.orchestra.run.smithi114.stdout:Setting system user ceph properties....done 2024-09-18T00:16:43.927 INFO:teuthology.orchestra.run.smithi114.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-18T00:16:44.064 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-18T00:16:44.245 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-18T00:16:44.381 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-18T00:16:44.750 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-test (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:44.879 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-test (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:44.909 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up radosgw (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:45.013 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up radosgw (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:45.375 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T00:16:45.375 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T00:16:45.486 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T00:16:45.487 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T00:16:45.889 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-base (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:45.990 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-base (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:46.369 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-18T00:16:46.395 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-18T00:16:46.850 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mds (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:46.864 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mds (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:47.056 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T00:16:47.056 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T00:16:47.091 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T00:16:47.091 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T00:16:47.519 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:47.550 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:47.729 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T00:16:47.729 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T00:16:47.768 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T00:16:47.768 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T00:16:48.226 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-osd (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:48.245 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-osd (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:48.500 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T00:16:48.500 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T00:16:48.526 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T00:16:48.526 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T00:16:49.080 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr-k8sevents (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:49.181 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:49.245 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-k8sevents (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:49.324 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mon (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:49.457 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:49.552 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T00:16:49.552 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T00:16:49.634 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mon (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:49.869 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T00:16:49.869 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T00:16:50.059 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr-cephadm (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:50.226 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:50.352 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr-dashboard (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:50.360 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-cephadm (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:50.503 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-volume (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:50.511 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:50.637 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-dashboard (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:50.788 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-volume (19.1.1-330-gc2ddf972-1jammy) ... 2024-09-18T00:16:50.865 INFO:teuthology.orchestra.run.smithi114.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-18T00:16:50.974 INFO:teuthology.orchestra.run.smithi114.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-18T00:16:51.126 INFO:teuthology.orchestra.run.smithi114.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-18T00:16:51.141 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-18T00:16:51.241 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-18T00:16:51.394 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-18T00:16:52.160 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-18T00:16:52.260 INFO:teuthology.orchestra.run.smithi114.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-18T00:16:52.829 INFO:teuthology.orchestra.run.smithi097.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-18T00:16:52.834 DEBUG:teuthology.parallel:result is None 2024-09-18T00:16:52.929 INFO:teuthology.orchestra.run.smithi114.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-18T00:16:52.933 DEBUG:teuthology.parallel:result is None 2024-09-18T00:16:52.933 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T00:16:53.101 DEBUG:teuthology.orchestra.run.smithi097:> dpkg-query -W -f '${Version}' ceph 2024-09-18T00:16:53.120 INFO:teuthology.orchestra.run.smithi097.stdout:19.1.1-330-gc2ddf972-1jammy 2024-09-18T00:16:53.120 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330-gc2ddf972-1jammy 2024-09-18T00:16:53.120 INFO:teuthology.task.install:The correct ceph version 19.1.1-330-gc2ddf972-1jammy is installed. 2024-09-18T00:16:53.122 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T00:16:53.271 DEBUG:teuthology.orchestra.run.smithi114:> dpkg-query -W -f '${Version}' ceph 2024-09-18T00:16:53.290 INFO:teuthology.orchestra.run.smithi114.stdout:19.1.1-330-gc2ddf972-1jammy 2024-09-18T00:16:53.290 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330-gc2ddf972-1jammy 2024-09-18T00:16:53.291 INFO:teuthology.task.install:The correct ceph version 19.1.1-330-gc2ddf972-1jammy is installed. 2024-09-18T00:16:53.293 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-18T00:16:53.293 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:16:53.293 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T00:16:53.305 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:16:53.305 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T00:16:53.343 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-18T00:16:53.343 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:16:53.343 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T00:16:53.361 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T00:16:53.417 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:16:53.417 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T00:16:53.431 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T00:16:53.486 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-18T00:16:53.487 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:16:53.487 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T00:16:53.500 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T00:16:53.553 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:16:53.553 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T00:16:53.566 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T00:16:53.619 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-18T00:16:53.619 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:16:53.619 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T00:16:53.633 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T00:16:53.689 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:16:53.689 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T00:16:53.703 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T00:16:53.759 INFO:teuthology.run_tasks:Running task ceph... 2024-09-18T00:16:53.837 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-18T00:16:53.838 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /var/log/ceph 2024-09-18T00:16:53.840 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 777 /var/log/ceph 2024-09-18T00:16:53.853 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-18T00:16:53.853 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T00:16:53.894 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T00:16:53.907 INFO:tasks.ceph:Creating extra log directories... 2024-09-18T00:16:53.907 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T00:16:53.947 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T00:16:53.973 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-18T00:16:53.973 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-18T00:16:53.973 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909638', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7909638', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 1983, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'sleep_before_teardown': 0, 'subset': '79/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7', 'targets': {'smithi097.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBKiv2JxQqEnyhl6E8Nw6NRxPA7jGKrfkLs4AJiPKwJHrMpkaiuj88zbbM5o9TDDyuBM3ab542ZlCxN9s5TubZE=', 'smithi114.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHuHns1Gtq6K5UGcKyGbCRcRtEpwx3KdQU/ipZzHlvcWBSInPDAL69nQiroDDcRWAUT8g+U1CkHb6AhwRih+A7M='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-18T00:16:53.974 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T00:16:54.002 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T00:16:54.011 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T00:16:54.050 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T00:16:54.068 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:16:54.068 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T00:16:54.109 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T00:16:54.110 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-09-18T00:16:54.158 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T00:16:54.158 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T00:16:54.158 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-09-18T00:16:54.158 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T00:16:54.158 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-18 00:11:34.174072329 +0000 2024-09-18T00:16:54.158 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-18 00:11:33.906074723 +0000 2024-09-18T00:16:54.158 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-18 00:11:33.906074723 +0000 2024-09-18T00:16:54.158 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2024-09-18T00:16:54.158 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T00:16:54.212 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-18T00:16:54.212 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-18T00:16:54.212 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000449613 s, 1.1 MB/s 2024-09-18T00:16:54.213 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T00:16:54.263 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-09-18T00:16:54.310 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T00:16:54.310 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T00:16:54.310 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-09-18T00:16:54.310 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T00:16:54.310 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-18 00:11:34.646068112 +0000 2024-09-18T00:16:54.310 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-18 00:11:34.402070293 +0000 2024-09-18T00:16:54.310 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-18 00:11:34.402070293 +0000 2024-09-18T00:16:54.310 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2024-09-18T00:16:54.310 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T00:16:54.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-18T00:16:54.364 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-18T00:16:54.364 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000435037 s, 1.2 MB/s 2024-09-18T00:16:54.365 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T00:16:54.414 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-09-18T00:16:54.466 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T00:16:54.466 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T00:16:54.466 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 921 Links: 1 2024-09-18T00:16:54.466 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T00:16:54.466 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-18 00:11:35.158063539 +0000 2024-09-18T00:16:54.466 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-18 00:11:34.894065897 +0000 2024-09-18T00:16:54.466 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-18 00:11:34.894065897 +0000 2024-09-18T00:16:54.466 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2024-09-18T00:16:54.467 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T00:16:54.520 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-18T00:16:54.520 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-18T00:16:54.520 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000327124 s, 1.6 MB/s 2024-09-18T00:16:54.522 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T00:16:54.571 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-09-18T00:16:54.617 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T00:16:54.617 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T00:16:54.617 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 937 Links: 1 2024-09-18T00:16:54.617 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T00:16:54.618 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-18 00:11:35.622059395 +0000 2024-09-18T00:16:54.618 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-18 00:11:35.410061289 +0000 2024-09-18T00:16:54.618 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-18 00:11:35.410061289 +0000 2024-09-18T00:16:54.618 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2024-09-18T00:16:54.618 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T00:16:54.674 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-18T00:16:54.674 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-18T00:16:54.674 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000292286 s, 1.8 MB/s 2024-09-18T00:16:54.675 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T00:16:54.722 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T00:16:54.723 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:16:54.723 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T00:16:54.728 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T00:16:54.729 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_1 2024-09-18T00:16:54.776 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T00:16:54.776 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T00:16:54.776 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 888 Links: 1 2024-09-18T00:16:54.776 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T00:16:54.776 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-18 00:11:35.628177775 +0000 2024-09-18T00:16:54.776 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-18 00:11:35.376187837 +0000 2024-09-18T00:16:54.776 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-18 00:11:35.376187837 +0000 2024-09-18T00:16:54.776 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2024-09-18T00:16:54.777 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T00:16:54.830 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-18T00:16:54.830 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-18T00:16:54.830 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000447144 s, 1.1 MB/s 2024-09-18T00:16:54.832 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T00:16:54.881 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_2 2024-09-18T00:16:54.932 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T00:16:54.932 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T00:16:54.932 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-09-18T00:16:54.932 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T00:16:54.933 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-18 00:11:36.120158246 +0000 2024-09-18T00:16:54.933 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-18 00:11:35.884167552 +0000 2024-09-18T00:16:54.933 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-18 00:11:35.884167552 +0000 2024-09-18T00:16:54.933 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2024-09-18T00:16:54.933 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T00:16:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-18T00:16:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-18T00:16:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000419527 s, 1.2 MB/s 2024-09-18T00:16:54.987 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T00:16:55.037 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_3 2024-09-18T00:16:55.088 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T00:16:55.088 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T00:16:55.088 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 922 Links: 1 2024-09-18T00:16:55.088 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T00:16:55.088 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-18 00:11:36.616138936 +0000 2024-09-18T00:16:55.088 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-18 00:11:36.360148903 +0000 2024-09-18T00:16:55.088 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-18 00:11:36.360148903 +0000 2024-09-18T00:16:55.088 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2024-09-18T00:16:55.089 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T00:16:55.145 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-18T00:16:55.146 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-18T00:16:55.146 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000439523 s, 1.2 MB/s 2024-09-18T00:16:55.147 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T00:16:55.197 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_4 2024-09-18T00:16:55.243 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T00:16:55.244 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T00:16:55.244 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 932 Links: 1 2024-09-18T00:16:55.244 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T00:16:55.244 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-18 00:11:37.140118674 +0000 2024-09-18T00:16:55.244 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-18 00:11:36.864129280 +0000 2024-09-18T00:16:55.244 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-18 00:11:36.864129280 +0000 2024-09-18T00:16:55.244 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2024-09-18T00:16:55.244 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T00:16:55.297 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-18T00:16:55.297 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-18T00:16:55.297 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000277553 s, 1.8 MB/s 2024-09-18T00:16:55.299 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T00:16:55.345 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T00:16:55.345 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-18T00:16:55.345 INFO:tasks.ceph:Generating config... 2024-09-18T00:16:55.346 INFO:tasks.ceph:[client] rbd cache = True 2024-09-18T00:16:55.347 INFO:tasks.ceph:[client] rbd cache policy = writearound 2024-09-18T00:16:55.347 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-18T00:16:55.347 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-18T00:16:55.347 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-18T00:16:55.347 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-18T00:16:55.347 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-18T00:16:55.347 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-18T00:16:55.347 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-18T00:16:55.347 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-18T00:16:55.347 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-18T00:16:55.347 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2024-09-18T00:16:55.347 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-18T00:16:55.347 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-18T00:16:55.348 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-18T00:16:55.348 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-18T00:16:55.348 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-18T00:16:55.348 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-18T00:16:55.348 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-18T00:16:55.348 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-18T00:16:55.348 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-18T00:16:55.348 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-18T00:16:55.348 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-18T00:16:55.348 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-18T00:16:55.348 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-18T00:16:55.348 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-18T00:16:55.348 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-18T00:16:55.349 INFO:tasks.ceph:Setting up mon.a... 2024-09-18T00:16:55.349 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-18T00:16:55.383 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.keyring 2024-09-18T00:16:55.393 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-18T00:16:55.474 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T00:16:55.528 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.97'), ('mon.c', '[v2:172.21.15.97:3301,v1:172.21.15.97:6790]'), ('mon.b', '172.21.15.114')] 2024-09-18T00:16:55.528 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.97,[v2:172.21.15.97:3301,v1:172.21.15.97:6790],172.21.15.114', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache policy': 'writearound'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-18T00:16:55.529 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:16:55.529 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T00:16:55.573 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.97 --addv c '[v2:172.21.15.97:3301,v1:172.21.15.97:6790]' --add b 172.21.15.114 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:16:55.658 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:16:55.659 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: generated fsid b993f56a-c4ef-4602-bb41-f32ae406b45a 2024-09-18T00:16:55.659 INFO:teuthology.orchestra.run.smithi097.stdout:setting min_mon_release = quincy 2024-09-18T00:16:55.659 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 0 2024-09-18T00:16:55.659 INFO:teuthology.orchestra.run.smithi097.stdout:fsid b993f56a-c4ef-4602-bb41-f32ae406b45a 2024-09-18T00:16:55.659 INFO:teuthology.orchestra.run.smithi097.stdout:last_changed 2024-09-18T00:16:55.659168+0000 2024-09-18T00:16:55.659 INFO:teuthology.orchestra.run.smithi097.stdout:created 2024-09-18T00:16:55.659168+0000 2024-09-18T00:16:55.659 INFO:teuthology.orchestra.run.smithi097.stdout:min_mon_release 17 (quincy) 2024-09-18T00:16:55.659 INFO:teuthology.orchestra.run.smithi097.stdout:election_strategy: 1 2024-09-18T00:16:55.659 INFO:teuthology.orchestra.run.smithi097.stdout:0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-09-18T00:16:55.659 INFO:teuthology.orchestra.run.smithi097.stdout:1: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.b 2024-09-18T00:16:55.659 INFO:teuthology.orchestra.run.smithi097.stdout:2: [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] mon.c 2024-09-18T00:16:55.659 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-18T00:16:55.661 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T00:16:55.709 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b993f56a-c4ef-4602-bb41-f32ae406b45a... 2024-09-18T00:16:55.710 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T00:16:55.754 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T00:16:55.783 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2024-09-18T00:16:55.783 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2024-09-18T00:16:55.783 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T00:16:55.783 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2024-09-18T00:16:55.783 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.784 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2024-09-18T00:16:55.784 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.784 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2024-09-18T00:16:55.784 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.784 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2024-09-18T00:16:55.784 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2024-09-18T00:16:55.784 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.784 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2024-09-18T00:16:55.784 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2024-09-18T00:16:55.784 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.784 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T00:16:55.784 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2024-09-18T00:16:55.784 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.784 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T00:16:55.785 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T00:16:55.785 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.785 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2024-09-18T00:16:55.785 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.785 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2024-09-18T00:16:55.785 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2024-09-18T00:16:55.785 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2024-09-18T00:16:55.785 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2024-09-18T00:16:55.785 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2024-09-18T00:16:55.785 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2024-09-18T00:16:55.785 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2024-09-18T00:16:55.785 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T00:16:55.785 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T00:16:55.786 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2024-09-18T00:16:55.786 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.786 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T00:16:55.786 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.786 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2024-09-18T00:16:55.786 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.786 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2024-09-18T00:16:55.786 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.786 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2024-09-18T00:16:55.786 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2024-09-18T00:16:55.786 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2024-09-18T00:16:55.786 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.97,[v2:172.21.15.97:3301,v1:172.21.15.97:6790],172.21.15.114" 2024-09-18T00:16:55.786 INFO:teuthology.orchestra.run.smithi097.stdout: mon client directed command retry = 5 2024-09-18T00:16:55.786 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on pool no app = False 2024-09-18T00:16:55.787 INFO:teuthology.orchestra.run.smithi097.stdout: ms inject socket failures = 5000 2024-09-18T00:16:55.787 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = b993f56a-c4ef-4602-bb41-f32ae406b45a 2024-09-18T00:16:55.787 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.787 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2024-09-18T00:16:55.787 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2024-09-18T00:16:55.787 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.787 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2024-09-18T00:16:55.787 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2024-09-18T00:16:55.787 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2024-09-18T00:16:55.787 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.787 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2024-09-18T00:16:55.787 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2024-09-18T00:16:55.787 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.787 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2024-09-18T00:16:55.787 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2024-09-18T00:16:55.788 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2024-09-18T00:16:55.788 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug trim objects = true 2024-09-18T00:16:55.788 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.788 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2024-09-18T00:16:55.788 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2024-09-18T00:16:55.788 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.788 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2024-09-18T00:16:55.788 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.788 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2024-09-18T00:16:55.788 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.788 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2024-09-18T00:16:55.788 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.788 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2024-09-18T00:16:55.789 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2024-09-18T00:16:55.789 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.789 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2024-09-18T00:16:55.789 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2024-09-18T00:16:55.789 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2024-09-18T00:16:55.789 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore compression algorithm = zlib 2024-09-18T00:16:55.789 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore compression mode = aggressive 2024-09-18T00:16:55.789 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2024-09-18T00:16:55.789 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 1/20 2024-09-18T00:16:55.789 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 1/20 2024-09-18T00:16:55.789 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-09-18T00:16:55.789 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2024-09-18T00:16:55.789 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 4/10 2024-09-18T00:16:55.789 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T00:16:55.790 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2024-09-18T00:16:55.790 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2024-09-18T00:16:55.790 INFO:teuthology.orchestra.run.smithi097.stdout: osd blocked scrub grace period = 3600 2024-09-18T00:16:55.790 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2024-09-18T00:16:55.790 INFO:teuthology.orchestra.run.smithi097.stdout: osd max markdown count = 1000 2024-09-18T00:16:55.790 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2024-09-18T00:16:55.790 INFO:teuthology.orchestra.run.smithi097.stdout: osd shutdown pgref assert = True 2024-09-18T00:16:55.790 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.790 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2024-09-18T00:16:55.790 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-09-18T00:16:55.790 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2024-09-18T00:16:55.790 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-09-18T00:16:55.790 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-09-18T00:16:55.790 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-09-18T00:16:55.790 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-09-18T00:16:55.791 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2024-09-18T00:16:55.791 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.791 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2024-09-18T00:16:55.791 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-09-18T00:16:55.791 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-09-18T00:16:55.791 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2024-09-18T00:16:55.791 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-09-18T00:16:55.791 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2024-09-18T00:16:55.791 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2024-09-18T00:16:55.791 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-09-18T00:16:55.791 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2024-09-18T00:16:55.791 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2024-09-18T00:16:55.791 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-09-18T00:16:55.791 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.792 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T00:16:55.792 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T00:16:55.792 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T00:16:55.792 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.792 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T00:16:55.792 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T00:16:55.792 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T00:16:55.792 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.792 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2024-09-18T00:16:55.792 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2024-09-18T00:16:55.792 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.792 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2024-09-18T00:16:55.792 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:16:55.792 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2024-09-18T00:16:55.793 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2024-09-18T00:16:55.793 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2024-09-18T00:16:55.793 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2024-09-18T00:16:55.793 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T00:16:55.793 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T00:16:55.793 INFO:teuthology.orchestra.run.smithi097.stdout: rbd cache = True 2024-09-18T00:16:55.793 INFO:teuthology.orchestra.run.smithi097.stdout: rbd cache policy = writearound 2024-09-18T00:16:55.793 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2024-09-18T00:16:55.793 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2024-09-18T00:16:55.793 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2024-09-18T00:16:55.794 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2024-09-18T00:16:55.794 INFO:teuthology.orchestra.run.smithi114.stdout: chdir = "" 2024-09-18T00:16:55.794 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T00:16:55.794 INFO:teuthology.orchestra.run.smithi114.stdout: auth supported = cephx 2024-09-18T00:16:55.794 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.794 INFO:teuthology.orchestra.run.smithi114.stdout: filestore xattr use omap = true 2024-09-18T00:16:55.794 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.794 INFO:teuthology.orchestra.run.smithi114.stdout: mon clock drift allowed = 1.000 2024-09-18T00:16:55.794 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.794 INFO:teuthology.orchestra.run.smithi114.stdout: osd crush chooseleaf type = 0 2024-09-18T00:16:55.795 INFO:teuthology.orchestra.run.smithi114.stdout: auth debug = true 2024-09-18T00:16:55.795 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.795 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on old message = true 2024-09-18T00:16:55.795 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on bug = true 2024-09-18T00:16:55.795 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.795 INFO:teuthology.orchestra.run.smithi114.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T00:16:55.795 INFO:teuthology.orchestra.run.smithi114.stdout: mon pg warn max object skew = 0 2024-09-18T00:16:55.795 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.795 INFO:teuthology.orchestra.run.smithi114.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T00:16:55.795 INFO:teuthology.orchestra.run.smithi114.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T00:16:55.795 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.795 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default size = 2 2024-09-18T00:16:55.795 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.795 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow primary affinity = true 2024-09-18T00:16:55.796 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow pg remap = true 2024-09-18T00:16:55.796 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on legacy crush tunables = false 2024-09-18T00:16:55.796 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on crush straw calc version zero = false 2024-09-18T00:16:55.796 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on no sortbitwise = false 2024-09-18T00:16:55.796 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on osd down out interval zero = false 2024-09-18T00:16:55.796 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on too few osds = false 2024-09-18T00:16:55.796 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T00:16:55.796 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T00:16:55.796 INFO:teuthology.orchestra.run.smithi114.stdout: mon_allow_pool_size_one = true 2024-09-18T00:16:55.796 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.796 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T00:16:55.796 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.796 INFO:teuthology.orchestra.run.smithi114.stdout: osd default data pool replay window = 5 2024-09-18T00:16:55.796 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.797 INFO:teuthology.orchestra.run.smithi114.stdout: mon allow pool delete = true 2024-09-18T00:16:55.797 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.797 INFO:teuthology.orchestra.run.smithi114.stdout: mon cluster log file level = debug 2024-09-18T00:16:55.797 INFO:teuthology.orchestra.run.smithi114.stdout: debug asserts on shutdown = true 2024-09-18T00:16:55.797 INFO:teuthology.orchestra.run.smithi114.stdout: mon health detail to clog = false 2024-09-18T00:16:55.797 INFO:teuthology.orchestra.run.smithi114.stdout: mon host = "172.21.15.97,[v2:172.21.15.97:3301,v1:172.21.15.97:6790],172.21.15.114" 2024-09-18T00:16:55.797 INFO:teuthology.orchestra.run.smithi114.stdout: mon client directed command retry = 5 2024-09-18T00:16:55.797 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on pool no app = False 2024-09-18T00:16:55.797 INFO:teuthology.orchestra.run.smithi114.stdout: ms inject socket failures = 5000 2024-09-18T00:16:55.797 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = b993f56a-c4ef-4602-bb41-f32ae406b45a 2024-09-18T00:16:55.797 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.797 INFO:teuthology.orchestra.run.smithi114.stdout:[osd] 2024-09-18T00:16:55.797 INFO:teuthology.orchestra.run.smithi114.stdout: osd journal size = 100 2024-09-18T00:16:55.797 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.797 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub load threshold = 5.0 2024-09-18T00:16:55.798 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub max interval = 600 2024-09-18T00:16:55.798 INFO:teuthology.orchestra.run.smithi114.stdout: osd mclock profile = high_recovery_ops 2024-09-18T00:16:55.798 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.798 INFO:teuthology.orchestra.run.smithi114.stdout: osd recover clone overlap = true 2024-09-18T00:16:55.798 INFO:teuthology.orchestra.run.smithi114.stdout: osd recovery max chunk = 1048576 2024-09-18T00:16:55.798 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.798 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug shutdown = true 2024-09-18T00:16:55.798 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug op order = true 2024-09-18T00:16:55.798 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify stray on activate = true 2024-09-18T00:16:55.798 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug trim objects = true 2024-09-18T00:16:55.798 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.798 INFO:teuthology.orchestra.run.smithi114.stdout: osd open classes on start = true 2024-09-18T00:16:55.798 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug pg log writeout = true 2024-09-18T00:16:55.798 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.799 INFO:teuthology.orchestra.run.smithi114.stdout: osd deep scrub update digest min age = 30 2024-09-18T00:16:55.799 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.799 INFO:teuthology.orchestra.run.smithi114.stdout: osd map max advance = 10 2024-09-18T00:16:55.799 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.799 INFO:teuthology.orchestra.run.smithi114.stdout: journal zero on create = true 2024-09-18T00:16:55.799 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.799 INFO:teuthology.orchestra.run.smithi114.stdout: filestore ondisk finisher threads = 3 2024-09-18T00:16:55.799 INFO:teuthology.orchestra.run.smithi114.stdout: filestore apply finisher threads = 3 2024-09-18T00:16:55.799 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.799 INFO:teuthology.orchestra.run.smithi114.stdout: bdev debug aio = true 2024-09-18T00:16:55.799 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug misdirected ops = true 2024-09-18T00:16:55.799 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore block size = 96636764160 2024-09-18T00:16:55.799 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore compression algorithm = zlib 2024-09-18T00:16:55.799 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore compression mode = aggressive 2024-09-18T00:16:55.799 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore fsck on mount = True 2024-09-18T00:16:55.800 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluefs = 1/20 2024-09-18T00:16:55.800 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluestore = 1/20 2024-09-18T00:16:55.800 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-09-18T00:16:55.800 INFO:teuthology.orchestra.run.smithi114.stdout: debug osd = 20 2024-09-18T00:16:55.800 INFO:teuthology.orchestra.run.smithi114.stdout: debug rocksdb = 4/10 2024-09-18T00:16:55.800 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T00:16:55.800 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd full ratio = 0.9 2024-09-18T00:16:55.800 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd nearfull ratio = 0.8 2024-09-18T00:16:55.800 INFO:teuthology.orchestra.run.smithi114.stdout: osd blocked scrub grace period = 3600 2024-09-18T00:16:55.800 INFO:teuthology.orchestra.run.smithi114.stdout: osd failsafe full ratio = 0.95 2024-09-18T00:16:55.800 INFO:teuthology.orchestra.run.smithi114.stdout: osd max markdown count = 1000 2024-09-18T00:16:55.800 INFO:teuthology.orchestra.run.smithi114.stdout: osd objectstore = bluestore 2024-09-18T00:16:55.800 INFO:teuthology.orchestra.run.smithi114.stdout: osd shutdown pgref assert = True 2024-09-18T00:16:55.800 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.800 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr] 2024-09-18T00:16:55.800 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-09-18T00:16:55.801 INFO:teuthology.orchestra.run.smithi114.stdout: debug mgr = 20 2024-09-18T00:16:55.801 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-09-18T00:16:55.801 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-09-18T00:16:55.801 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-09-18T00:16:55.801 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-09-18T00:16:55.801 INFO:teuthology.orchestra.run.smithi114.stdout: mgr/telemetry/nag = false 2024-09-18T00:16:55.801 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.801 INFO:teuthology.orchestra.run.smithi114.stdout:[mon] 2024-09-18T00:16:55.801 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-09-18T00:16:55.801 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-09-18T00:16:55.801 INFO:teuthology.orchestra.run.smithi114.stdout: debug paxos = 20 2024-09-18T00:16:55.801 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-09-18T00:16:55.801 INFO:teuthology.orchestra.run.smithi114.stdout: mon data avail warn = 5 2024-09-18T00:16:55.801 INFO:teuthology.orchestra.run.smithi114.stdout: mon mgr mkfs grace = 240 2024-09-18T00:16:55.802 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-09-18T00:16:55.802 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd reporter subtree level = osd 2024-09-18T00:16:55.802 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd prime pg temp = true 2024-09-18T00:16:55.802 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-09-18T00:16:55.802 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.802 INFO:teuthology.orchestra.run.smithi114.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T00:16:55.802 INFO:teuthology.orchestra.run.smithi114.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T00:16:55.802 INFO:teuthology.orchestra.run.smithi114.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T00:16:55.802 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.802 INFO:teuthology.orchestra.run.smithi114.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T00:16:55.802 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T00:16:55.802 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T00:16:55.802 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.802 INFO:teuthology.orchestra.run.smithi114.stdout: # 1m isn't quite enough 2024-09-18T00:16:55.802 INFO:teuthology.orchestra.run.smithi114.stdout: mon_down_mkfs_grace = 2m 2024-09-18T00:16:55.803 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.803 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_filestore_osds = false 2024-09-18T00:16:55.803 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-18T00:16:55.803 INFO:teuthology.orchestra.run.smithi114.stdout:[client] 2024-09-18T00:16:55.803 INFO:teuthology.orchestra.run.smithi114.stdout: rgw cache enabled = true 2024-09-18T00:16:55.803 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable ops log = true 2024-09-18T00:16:55.803 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable usage log = true 2024-09-18T00:16:55.803 INFO:teuthology.orchestra.run.smithi114.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T00:16:55.803 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T00:16:55.803 INFO:teuthology.orchestra.run.smithi114.stdout: rbd cache = True 2024-09-18T00:16:55.803 INFO:teuthology.orchestra.run.smithi114.stdout: rbd cache policy = writearound 2024-09-18T00:16:55.803 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.a] 2024-09-18T00:16:55.803 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.c] 2024-09-18T00:16:55.803 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.b] 2024-09-18T00:16:55.805 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-18T00:16:55.805 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-18T00:16:55.893 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-18T00:16:55.893 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:16:55.894 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-18T00:16:55.898 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:16:55.899 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-18T00:16:55.945 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi097.front.sepia.ceph.com 2024-09-18T00:16:55.945 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:16:55.946 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T00:16:55.946 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T00:16:56.009 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:16:56.010 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:16:56.061 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi114.front.sepia.ceph.com 2024-09-18T00:16:56.062 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:16:56.062 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T00:16:56.062 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T00:16:56.085 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:16:56.085 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:16:56.131 INFO:tasks.ceph:Setting up mon nodes... 2024-09-18T00:16:56.131 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-18T00:16:56.131 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-18T00:16:56.175 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-18T00:16:56.178 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T00:16:56.220 INFO:teuthology.orchestra.run.smithi114.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T00:16:56.235 INFO:tasks.ceph:Setting up mds nodes... 2024-09-18T00:16:56.235 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-18T00:16:56.236 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-18T00:16:56.269 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-18T00:16:56.279 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-18T00:16:56.316 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-18T00:16:56.330 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-18T00:16:56.330 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-18T00:16:56.330 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-18T00:16:56.343 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T00:16:56.343 INFO:tasks.ceph:role: osd.0 2024-09-18T00:16:56.343 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-18T00:16:56.343 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T00:16:56.404 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T00:16:56.405 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T00:16:56.405 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T00:16:56.405 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-18T00:16:56.405 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T00:16:56.405 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-09-18T00:16:56.405 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T00:16:56.405 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T00:16:56.405 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T00:16:56.405 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T00:16:56.853 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-09-18T00:16:56.855 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-09-18T00:16:56.856 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-18T00:16:56.942 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-18T00:16:56.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:16:56.952 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T00:16:56.952 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-18T00:16:57.003 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T00:16:57.003 INFO:tasks.ceph:role: osd.1 2024-09-18T00:16:57.003 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-18T00:16:57.003 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T00:16:57.065 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T00:16:57.065 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T00:16:57.065 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T00:16:57.065 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-18T00:16:57.065 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T00:16:57.066 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-09-18T00:16:57.066 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T00:16:57.066 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T00:16:57.066 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T00:16:57.066 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T00:16:57.538 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-09-18T00:16:57.540 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-09-18T00:16:57.540 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-18T00:16:57.652 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-18T00:16:57.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:16:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T00:16:57.664 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-18T00:16:57.717 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T00:16:57.717 INFO:tasks.ceph:role: osd.2 2024-09-18T00:16:57.717 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-18T00:16:57.717 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T00:16:57.781 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T00:16:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T00:16:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T00:16:57.781 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-18T00:16:57.781 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T00:16:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-09-18T00:16:57.782 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T00:16:57.782 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T00:16:57.782 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T00:16:57.782 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T00:16:58.263 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-09-18T00:16:58.265 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-09-18T00:16:58.265 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-18T00:16:58.397 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-18T00:16:58.407 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T00:16:58.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:16:58.408 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-18T00:16:58.460 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T00:16:58.460 INFO:tasks.ceph:role: osd.3 2024-09-18T00:16:58.460 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-18T00:16:58.460 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T00:16:58.526 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T00:16:58.526 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T00:16:58.526 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T00:16:58.526 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-18T00:16:58.526 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T00:16:58.526 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-09-18T00:16:58.526 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T00:16:58.526 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T00:16:58.526 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T00:16:58.526 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T00:16:58.973 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-09-18T00:16:58.976 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-09-18T00:16:58.976 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-18T00:16:59.050 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-18T00:16:59.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:16:59.102 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T00:16:59.102 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:16:59.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:16:59.215+0000 7f147145c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-18T00:16:59.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:16:59.219+0000 7f147145c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-18T00:16:59.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:16:59.219+0000 7f147145c7c0 -1 bdev(0x558f8b261800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-18T00:16:59.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:16:59.219+0000 7f147145c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-18T00:17:02.382 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-18T00:17:02.395 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:17:02.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:17:02.471+0000 7f9b877687c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-18T00:17:02.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:17:02.471+0000 7f9b877687c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-18T00:17:02.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:17:02.471+0000 7f9b877687c0 -1 bdev(0x55ce1c2c7800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-18T00:17:02.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:17:02.471+0000 7f9b877687c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-18T00:17:05.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-18T00:17:05.685 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:17:05.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:17:05.759+0000 7f09764347c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-18T00:17:05.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:17:05.763+0000 7f09764347c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-18T00:17:05.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:17:05.763+0000 7f09764347c0 -1 bdev(0x55b047955800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-18T00:17:05.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:17:05.763+0000 7f09764347c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-18T00:17:08.916 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-18T00:17:08.974 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:17:09.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:17:09.055+0000 7f2f6b3a97c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-18T00:17:09.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:17:09.055+0000 7f2f6b3a97c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-18T00:17:09.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:17:09.055+0000 7f2f6b3a97c0 -1 bdev(0x55927f0bd800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-18T00:17:09.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:17:09.055+0000 7f2f6b3a97c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-18T00:17:12.196 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-18T00:17:12.249 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-18T00:17:12.264 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T00:17:12.264 INFO:tasks.ceph:role: osd.4 2024-09-18T00:17:12.264 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-18T00:17:12.264 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T00:17:12.331 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T00:17:12.331 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T00:17:12.331 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T00:17:12.331 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-18T00:17:12.332 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T00:17:12.332 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-09-18T00:17:12.332 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T00:17:12.332 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T00:17:12.332 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T00:17:12.332 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T00:17:12.806 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-09-18T00:17:12.808 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-09-18T00:17:12.809 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-18T00:17:12.950 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-18T00:17:12.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:17:12.961 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T00:17:12.961 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-18T00:17:13.014 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T00:17:13.015 INFO:tasks.ceph:role: osd.5 2024-09-18T00:17:13.015 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-18T00:17:13.015 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T00:17:13.079 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T00:17:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T00:17:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T00:17:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-18T00:17:13.079 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T00:17:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-09-18T00:17:13.079 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T00:17:13.080 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T00:17:13.080 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T00:17:13.080 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T00:17:13.504 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-09-18T00:17:13.506 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-09-18T00:17:13.506 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-18T00:17:13.569 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-18T00:17:13.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:17:13.624 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T00:17:13.624 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-18T00:17:13.677 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T00:17:13.677 INFO:tasks.ceph:role: osd.6 2024-09-18T00:17:13.677 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-18T00:17:13.677 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T00:17:13.736 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T00:17:13.736 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T00:17:13.736 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T00:17:13.736 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-18T00:17:13.736 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T00:17:13.737 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-09-18T00:17:13.737 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T00:17:13.737 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T00:17:13.737 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T00:17:13.737 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T00:17:14.213 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-09-18T00:17:14.215 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-09-18T00:17:14.215 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-18T00:17:14.306 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-18T00:17:14.362 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T00:17:14.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:17:14.363 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-18T00:17:14.419 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T00:17:14.419 INFO:tasks.ceph:role: osd.7 2024-09-18T00:17:14.419 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-18T00:17:14.419 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T00:17:14.484 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T00:17:14.484 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T00:17:14.484 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T00:17:14.484 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-18T00:17:14.484 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T00:17:14.484 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-09-18T00:17:14.484 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T00:17:14.484 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T00:17:14.484 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T00:17:14.485 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T00:17:14.968 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-09-18T00:17:14.970 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-09-18T00:17:14.970 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-18T00:17:15.110 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-18T00:17:15.121 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T00:17:15.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:17:15.122 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:17:15.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-18T00:17:15.227+0000 7f539abbc7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-18T00:17:15.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-18T00:17:15.231+0000 7f539abbc7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-18T00:17:15.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-18T00:17:15.231+0000 7f539abbc7c0 -1 bdev(0x55cfc229b800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-18T00:17:15.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-18T00:17:15.231+0000 7f539abbc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-18T00:17:18.388 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-18T00:17:18.402 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:17:18.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-18T00:17:18.475+0000 7fd27311e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-18T00:17:18.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-18T00:17:18.475+0000 7fd27311e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-18T00:17:18.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-18T00:17:18.475+0000 7fd27311e7c0 -1 bdev(0x55f9f9a8d800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-18T00:17:18.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-18T00:17:18.475+0000 7fd27311e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-18T00:17:21.622 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-18T00:17:21.677 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:17:21.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-18T00:17:21.751+0000 7fa537ddb7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-18T00:17:21.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-18T00:17:21.751+0000 7fa537ddb7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-18T00:17:21.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-18T00:17:21.751+0000 7fa537ddb7c0 -1 bdev(0x55fe417e1800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-18T00:17:21.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-18T00:17:21.751+0000 7fa537ddb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-18T00:17:24.906 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-18T00:17:24.964 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:17:25.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-18T00:17:25.039+0000 7f53c210f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-18T00:17:25.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-18T00:17:25.039+0000 7f53c210f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-18T00:17:25.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-18T00:17:25.039+0000 7f53c210f7c0 -1 bdev(0x55792ca7d800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-18T00:17:25.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-18T00:17:25.039+0000 7f53c210f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-18T00:17:28.189 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-18T00:17:28.243 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-18T00:17:28.243 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:17:28.243 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-18T00:17:28.258 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:17:28.258 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-18T00:17:28.312 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:17:28.312 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-18T00:17:28.369 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:17:28.369 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-18T00:17:28.424 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:17:28.425 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-18T00:17:28.480 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:17:28.480 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-18T00:17:28.493 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:17:28.493 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-18T00:17:28.546 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:17:28.546 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-18T00:17:28.598 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:17:28.599 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-18T00:17:28.655 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:17:28.655 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-18T00:17:28.711 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:17:28.712 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-18T00:17:28.717 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:17:28.717 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-18T00:17:28.759 INFO:tasks.ceph:Adding keys to all mons... 2024-09-18T00:17:28.759 DEBUG:teuthology.orchestra.run.smithi097:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T00:17:28.762 DEBUG:teuthology.orchestra.run.smithi114:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T00:17:28.805 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.y] 2024-09-18T00:17:28.805 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQD4G+pmhVl7ChAAjocw0jtRidoag8Y/ZruUqg== 2024-09-18T00:17:28.805 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.0] 2024-09-18T00:17:28.806 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQD7G+pmV/8iDRAAHxT7O2EBSgvApoTRNWFo8w== 2024-09-18T00:17:28.806 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.1] 2024-09-18T00:17:28.806 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQD+G+pmdHpEHBAAtZGzA4jQkqUQQtYpP5jHZw== 2024-09-18T00:17:28.806 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.2] 2024-09-18T00:17:28.806 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQABHOpmdw2fLRAABRlPpY80DsVGrXWIXdP37w== 2024-09-18T00:17:28.806 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.3] 2024-09-18T00:17:28.806 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAFHOpmge+dAxAAtAheuf7LWb6CrC8UrYfFHA== 2024-09-18T00:17:28.806 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.x] 2024-09-18T00:17:28.806 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQD4G+pmGTYLDRAABtx9CYGhMxmVvIBpRi3TPw== 2024-09-18T00:17:28.806 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.4] 2024-09-18T00:17:28.806 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQALHOpmAgb0DRAAatdKM1Le2605RNkQSxI7gQ== 2024-09-18T00:17:28.806 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.5] 2024-09-18T00:17:28.806 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAOHOpm5ZuqHBAAD+f0daDgw4ejHxbzY1VFuA== 2024-09-18T00:17:28.806 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.6] 2024-09-18T00:17:28.807 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQARHOpmDD0mLRAAeKqRjYrVL/ErPUfOV4O/tw== 2024-09-18T00:17:28.807 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.7] 2024-09-18T00:17:28.807 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAVHOpmheOwAhAAh+umt4tX74uruXZvh+PGFw== 2024-09-18T00:17:28.807 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-09-18T00:17:28.807 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQD4G+pmwhMREBAAU8MzB9flR9S66pzQn+LlHA== 2024-09-18T00:17:28.807 INFO:teuthology.orchestra.run.smithi097.stdout:[client.1] 2024-09-18T00:17:28.807 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQD4G+pmDUTSEhAAZp500j3zCZ6SMi1tTNkDoQ== 2024-09-18T00:17:28.813 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr.y] 2024-09-18T00:17:28.813 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQD4G+pmhVl7ChAAjocw0jtRidoag8Y/ZruUqg== 2024-09-18T00:17:28.813 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.0] 2024-09-18T00:17:28.813 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQD7G+pmV/8iDRAAHxT7O2EBSgvApoTRNWFo8w== 2024-09-18T00:17:28.813 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.1] 2024-09-18T00:17:28.813 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQD+G+pmdHpEHBAAtZGzA4jQkqUQQtYpP5jHZw== 2024-09-18T00:17:28.813 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.2] 2024-09-18T00:17:28.813 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQABHOpmdw2fLRAABRlPpY80DsVGrXWIXdP37w== 2024-09-18T00:17:28.813 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.3] 2024-09-18T00:17:28.814 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAFHOpmge+dAxAAtAheuf7LWb6CrC8UrYfFHA== 2024-09-18T00:17:28.814 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr.x] 2024-09-18T00:17:28.814 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQD4G+pmGTYLDRAABtx9CYGhMxmVvIBpRi3TPw== 2024-09-18T00:17:28.814 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.4] 2024-09-18T00:17:28.814 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQALHOpmAgb0DRAAatdKM1Le2605RNkQSxI7gQ== 2024-09-18T00:17:28.814 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.5] 2024-09-18T00:17:28.814 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAOHOpm5ZuqHBAAD+f0daDgw4ejHxbzY1VFuA== 2024-09-18T00:17:28.814 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.6] 2024-09-18T00:17:28.814 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQARHOpmDD0mLRAAeKqRjYrVL/ErPUfOV4O/tw== 2024-09-18T00:17:28.814 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.7] 2024-09-18T00:17:28.814 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAVHOpmheOwAhAAh+umt4tX74uruXZvh+PGFw== 2024-09-18T00:17:28.814 INFO:teuthology.orchestra.run.smithi114.stdout:[client.0] 2024-09-18T00:17:28.814 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQD4G+pmwhMREBAAU8MzB9flR9S66pzQn+LlHA== 2024-09-18T00:17:28.814 INFO:teuthology.orchestra.run.smithi114.stdout:[client.1] 2024-09-18T00:17:28.815 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQD4G+pmDUTSEhAAZp500j3zCZ6SMi1tTNkDoQ== 2024-09-18T00:17:28.815 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T00:17:28.850 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T00:17:28.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:17:28.938 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:17:28.989 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:17:29.022 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:17:29.071 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:17:29.106 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:17:29.153 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:17:29.156 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:17:29.233 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T00:17:29.238 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T00:17:29.314 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:17:29.322 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:17:29.394 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:17:29.402 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:17:29.478 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:17:29.482 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:17:29.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:17:29.564 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T00:17:29.641 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T00:17:29.646 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T00:17:29.725 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T00:17:29.727 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T00:17:29.808 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-18T00:17:29.809 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-18T00:17:29.819 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T00:17:29.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-18T00:17:29.954 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-18T00:17:30.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T00:17:30.102 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-18T00:17:30.112 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-18T00:17:30.125 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T00:17:30.218 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-18T00:17:30.230 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:17:30.232 DEBUG:teuthology.orchestra.run.smithi114:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T00:17:30.274 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-18T00:17:30.275 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-18T00:17:30.275 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-18T00:17:30.278 INFO:tasks.ceph.mon.a:Started 2024-09-18T00:17:30.278 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-18T00:17:30.278 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-18T00:17:30.281 INFO:tasks.ceph.mon.c:Started 2024-09-18T00:17:30.281 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-18T00:17:30.282 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-18T00:17:30.320 INFO:tasks.ceph.mon.b:Started 2024-09-18T00:17:30.320 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-18T00:17:30.321 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-18T00:17:30.321 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-18T00:17:30.323 INFO:tasks.ceph.mgr.y:Started 2024-09-18T00:17:30.323 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-18T00:17:30.323 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-18T00:17:30.325 INFO:tasks.ceph.mgr.x:Started 2024-09-18T00:17:30.325 INFO:tasks.ceph:Setting crush tunables to default 2024-09-18T00:17:30.326 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd crush tunables default 2024-09-18T00:17:35.786 INFO:teuthology.orchestra.run.smithi097.stderr:adjusted tunables profile to default 2024-09-18T00:17:35.804 INFO:tasks.ceph:check_enable_crimson: False 2024-09-18T00:17:35.804 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-18T00:17:35.804 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:17:35.804 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-18T00:17:35.816 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:17:35.817 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-18T00:17:35.870 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:17:35.871 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-18T00:17:35.894 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:35.891+0000 7f73db1b8180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-18T00:17:35.905 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-18T00:17:35.905 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-18T00:17:35.963 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:17:35.963 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-18T00:17:35.964 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:35.959+0000 7f2ee31c2180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-18T00:17:35.976 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:17:35.976 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-18T00:17:36.031 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:17:36.031 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-18T00:17:36.086 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-18T00:17:36.086 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-18T00:17:36.142 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new bfda7950-3d83-435d-8313-2f14345096ec 0 2024-09-18T00:17:36.393 INFO:teuthology.orchestra.run.smithi114.stdout:0 2024-09-18T00:17:36.408 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:36.403+0000 7f73db1b8180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-18T00:17:36.411 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 35949061-aa30-4aff-b39d-90aebeffb115 1 2024-09-18T00:17:36.465 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:36.459+0000 7f2ee31c2180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-18T00:17:36.588 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:36.583+0000 7f73db1b8180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-18T00:17:36.614 INFO:teuthology.orchestra.run.smithi114.stdout:1 2024-09-18T00:17:36.630 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 31b3e1c1-7648-44fe-9931-758ffe9a53ca 2 2024-09-18T00:17:36.678 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:36.671+0000 7f2ee31c2180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-18T00:17:36.831 INFO:teuthology.orchestra.run.smithi114.stdout:2 2024-09-18T00:17:36.848 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new d7daebbc-0e33-42ab-97ee-a6c1b449ae8d 3 2024-09-18T00:17:37.050 INFO:teuthology.orchestra.run.smithi114.stdout:3 2024-09-18T00:17:37.066 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new d55ca69b-5000-45ff-acc4-00f7c4763cad 4 2024-09-18T00:17:37.144 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:37.139+0000 7f73db1b8180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-18T00:17:37.199 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:37.195+0000 7f73db1b8180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-18T00:17:37.251 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:37.247+0000 7f73db1b8180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-18T00:17:37.262 INFO:teuthology.orchestra.run.smithi114.stdout:4 2024-09-18T00:17:37.273 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:37.267+0000 7f2ee31c2180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-18T00:17:37.277 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 196a329b-bd93-4cc2-85bb-64ed81d3918b 5 2024-09-18T00:17:37.329 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:37.323+0000 7f2ee31c2180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-18T00:17:37.329 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:37.327+0000 7f73db1b8180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-18T00:17:37.381 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:37.379+0000 7f73db1b8180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-18T00:17:37.382 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:37.375+0000 7f2ee31c2180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-18T00:17:37.458 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:37.455+0000 7f73db1b8180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-18T00:17:37.475 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:37.471+0000 7f2ee31c2180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-18T00:17:37.484 INFO:teuthology.orchestra.run.smithi114.stdout:5 2024-09-18T00:17:37.500 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new dc9712f0-8858-467a-8397-d5eb97187165 6 2024-09-18T00:17:37.512 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:37.507+0000 7f73db1b8180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-18T00:17:37.532 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:37.527+0000 7f2ee31c2180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-18T00:17:37.616 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:37.611+0000 7f2ee31c2180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-18T00:17:37.664 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:37.659+0000 7f73db1b8180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-18T00:17:37.670 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:37.663+0000 7f2ee31c2180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-18T00:17:37.695 INFO:teuthology.orchestra.run.smithi114.stdout:6 2024-09-18T00:17:37.711 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new b6d774ee-fe6d-462c-afa4-7ffd5d7343a9 7 2024-09-18T00:17:37.728 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:37.723+0000 7f73db1b8180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-18T00:17:37.825 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:37.819+0000 7f2ee31c2180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-18T00:17:37.888 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:37.883+0000 7f2ee31c2180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-18T00:17:37.955 INFO:teuthology.orchestra.run.smithi114.stdout:7 2024-09-18T00:17:37.971 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-18T00:17:37.971 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-18T00:17:37.973 INFO:tasks.ceph.osd.0:Started 2024-09-18T00:17:37.973 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-18T00:17:37.974 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-18T00:17:37.976 INFO:tasks.ceph.osd.1:Started 2024-09-18T00:17:37.976 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-18T00:17:37.976 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-18T00:17:37.978 INFO:tasks.ceph.osd.2:Started 2024-09-18T00:17:37.979 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-18T00:17:37.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-18T00:17:37.981 INFO:tasks.ceph.osd.3:Started 2024-09-18T00:17:37.981 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-18T00:17:37.981 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-18T00:17:37.983 INFO:tasks.ceph.osd.4:Started 2024-09-18T00:17:37.983 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-18T00:17:37.983 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-18T00:17:37.984 INFO:tasks.ceph.osd.5:Started 2024-09-18T00:17:37.985 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-18T00:17:37.985 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-18T00:17:37.986 INFO:tasks.ceph.osd.6:Started 2024-09-18T00:17:37.987 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-18T00:17:37.987 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-18T00:17:37.989 INFO:tasks.ceph.osd.7:Started 2024-09-18T00:17:37.989 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T00:17:38.208 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:17:38.208 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":10,"fsid":"b993f56a-c4ef-4602-bb41-f32ae406b45a","created":"2024-09-18T00:17:35.658788+0000","modified":"2024-09-18T00:17:37.952065+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T00:17:37.952065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bfda7950-3d83-435d-8313-2f14345096ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"35949061-aa30-4aff-b39d-90aebeffb115","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"31b3e1c1-7648-44fe-9931-758ffe9a53ca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d7daebbc-0e33-42ab-97ee-a6c1b449ae8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d55ca69b-5000-45ff-acc4-00f7c4763cad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"196a329b-bd93-4cc2-85bb-64ed81d3918b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dc9712f0-8858-467a-8397-d5eb97187165","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b6d774ee-fe6d-462c-afa4-7ffd5d7343a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T00:17:38.224 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-18T00:17:38.224 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-18T00:17:38.348 INFO:tasks.ceph.mgr.y.smithi097.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-18T00:17:38.348 INFO:tasks.ceph.mgr.y.smithi097.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-18T00:17:38.348 INFO:tasks.ceph.mgr.y.smithi097.stderr: from numpy import show_config as show_numpy_config 2024-09-18T00:17:38.434 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:38.431+0000 7f73db1b8180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-18T00:17:38.519 INFO:tasks.ceph.mgr.x.smithi114.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-18T00:17:38.520 INFO:tasks.ceph.mgr.x.smithi114.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-18T00:17:38.520 INFO:tasks.ceph.mgr.x.smithi114.stderr: from numpy import show_config as show_numpy_config 2024-09-18T00:17:38.569 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:38.563+0000 7f2ee31c2180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-18T00:17:38.603 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:38.599+0000 7f73db1b8180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-18T00:17:38.743 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:38.739+0000 7f2ee31c2180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-18T00:17:38.777 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:38.775+0000 7f73db1b8180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-18T00:17:38.840 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:38.835+0000 7f73db1b8180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-18T00:17:38.920 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:38.915+0000 7f2ee31c2180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-18T00:17:38.976 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:38.971+0000 7f73db1b8180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-18T00:17:38.989 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:38.983+0000 7f2ee31c2180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-18T00:17:39.029 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T00:17:39.124 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:39.119+0000 7f2ee31c2180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-18T00:17:39.169 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:17:39.167+0000 7f2b940777c0 -1 Falling back to public interface 2024-09-18T00:17:39.193 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:17:39.191+0000 7fc68bb927c0 -1 Falling back to public interface 2024-09-18T00:17:39.213 INFO:teuthology.misc.health.smithi097.stdout: 2024-09-18T00:17:39.213 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":10,"fsid":"b993f56a-c4ef-4602-bb41-f32ae406b45a","created":"2024-09-18T00:17:35.658788+0000","modified":"2024-09-18T00:17:37.952065+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T00:17:37.952065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bfda7950-3d83-435d-8313-2f14345096ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"35949061-aa30-4aff-b39d-90aebeffb115","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"31b3e1c1-7648-44fe-9931-758ffe9a53ca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d7daebbc-0e33-42ab-97ee-a6c1b449ae8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d55ca69b-5000-45ff-acc4-00f7c4763cad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"196a329b-bd93-4cc2-85bb-64ed81d3918b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dc9712f0-8858-467a-8397-d5eb97187165","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b6d774ee-fe6d-462c-afa4-7ffd5d7343a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T00:17:39.227 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-18T00:17:39.392 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:17:39.387+0000 7fafbe2847c0 -1 Falling back to public interface 2024-09-18T00:17:39.393 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:17:39.387+0000 7f945742b7c0 -1 Falling back to public interface 2024-09-18T00:17:39.411 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:17:39.407+0000 7f6e700517c0 -1 Falling back to public interface 2024-09-18T00:17:39.418 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:17:39.415+0000 7f8f40d377c0 -1 Falling back to public interface 2024-09-18T00:17:39.427 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:17:39.423+0000 7f18c89147c0 -1 Falling back to public interface 2024-09-18T00:17:39.429 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:17:39.427+0000 7f20dfefb7c0 -1 Falling back to public interface 2024-09-18T00:17:39.581 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:39.579+0000 7f73db1b8180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-18T00:17:39.656 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:39.651+0000 7f73db1b8180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-18T00:17:39.724 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:39.719+0000 7f2ee31c2180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-18T00:17:39.799 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:39.795+0000 7f2ee31c2180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-18T00:17:39.840 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:39.835+0000 7f73db1b8180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-18T00:17:39.913 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:39.911+0000 7f73db1b8180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-18T00:17:39.979 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:39.971+0000 7f2ee31c2180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-18T00:17:40.054 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:40.047+0000 7f2ee31c2180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-18T00:17:40.205 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:40.203+0000 7f73db1b8180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-18T00:17:40.266 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:40.263+0000 7f73db1b8180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-18T00:17:40.344 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:40.339+0000 7f2ee31c2180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-18T00:17:40.406 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-18T00:17:40.399+0000 7f2ee31c2180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-18T00:17:41.305 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:41.303+0000 7f7381cf0640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-18T00:17:41.306 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-09-18T00:17:41.303+0000 7f7381cf0640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-18T00:17:43.244 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:17:43.239+0000 7fc68bb927c0 -1 osd.3 0 log_to_monitors true 2024-09-18T00:17:43.281 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:17:43.279+0000 7f2b940777c0 -1 osd.0 0 log_to_monitors true 2024-09-18T00:17:43.420 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:17:43.415+0000 7fafbe2847c0 -1 osd.4 0 log_to_monitors true 2024-09-18T00:17:43.492 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:17:43.487+0000 7f20dfefb7c0 -1 osd.1 0 log_to_monitors true 2024-09-18T00:17:43.518 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:17:43.511+0000 7f18c89147c0 -1 osd.6 0 log_to_monitors true 2024-09-18T00:17:43.723 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:17:43.719+0000 7f6e700517c0 -1 osd.7 0 log_to_monitors true 2024-09-18T00:17:43.725 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:17:43.723+0000 7f8f40d377c0 -1 osd.2 0 log_to_monitors true 2024-09-18T00:17:43.774 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:17:43.767+0000 7f945742b7c0 -1 osd.5 0 log_to_monitors true 2024-09-18T00:17:46.034 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T00:17:46.383 INFO:teuthology.misc.health.smithi097.stdout: 2024-09-18T00:17:46.383 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":13,"fsid":"b993f56a-c4ef-4602-bb41-f32ae406b45a","created":"2024-09-18T00:17:35.658788+0000","modified":"2024-09-18T00:17:45.302239+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T00:17:37.952065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bfda7950-3d83-435d-8313-2f14345096ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"35949061-aa30-4aff-b39d-90aebeffb115","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"31b3e1c1-7648-44fe-9931-758ffe9a53ca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d7daebbc-0e33-42ab-97ee-a6c1b449ae8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d55ca69b-5000-45ff-acc4-00f7c4763cad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"196a329b-bd93-4cc2-85bb-64ed81d3918b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dc9712f0-8858-467a-8397-d5eb97187165","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b6d774ee-fe6d-462c-afa4-7ffd5d7343a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T00:17:46.408 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-18T00:17:47.365 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:17:47.363+0000 7f2b90020640 -1 osd.0 0 waiting for initial osdmap 2024-09-18T00:17:47.398 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:17:47.395+0000 7f2b87e55640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:17:47.581 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:17:47.579+0000 7fc687b3b640 -1 osd.3 0 waiting for initial osdmap 2024-09-18T00:17:47.601 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:17:47.599+0000 7fc67f970640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:17:48.408 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:17:48.407+0000 7f20dc6b6640 -1 osd.1 0 waiting for initial osdmap 2024-09-18T00:17:48.437 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:17:48.435+0000 7f20d3cd9640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:17:48.438 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:17:48.435+0000 7f6e6bffa640 -1 osd.7 0 waiting for initial osdmap 2024-09-18T00:17:48.471 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:17:48.467+0000 7f6e63e2f640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:17:48.473 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:17:48.471+0000 7f8f3cce0640 -1 osd.2 0 waiting for initial osdmap 2024-09-18T00:17:48.497 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:17:48.495+0000 7f8f34b15640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:17:48.531 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:17:48.527+0000 7fafba22d640 -1 osd.4 0 waiting for initial osdmap 2024-09-18T00:17:48.543 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:17:48.539+0000 7f9453be6640 -1 osd.5 0 waiting for initial osdmap 2024-09-18T00:17:48.563 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:17:48.559+0000 7fafb2062640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:17:48.567 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:17:48.563+0000 7f944b209640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:17:48.571 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:17:48.567+0000 7f18c48bd640 -1 osd.6 0 waiting for initial osdmap 2024-09-18T00:17:48.587 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:17:48.583+0000 7f18bc6f2640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:17:53.213 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T00:17:53.522 INFO:teuthology.misc.health.smithi097.stdout: 2024-09-18T00:17:53.522 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":18,"fsid":"b993f56a-c4ef-4602-bb41-f32ae406b45a","created":"2024-09-18T00:17:35.658788+0000","modified":"2024-09-18T00:17:52.432866+0000","last_up_change":"2024-09-18T00:17:49.411529+0000","last_in_change":"2024-09-18T00:17:37.952065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T00:17:50.375367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bfda7950-3d83-435d-8313-2f14345096ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6801","nonce":40468291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6803","nonce":40468291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6807","nonce":40468291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6805","nonce":40468291}]},"public_addr":"172.21.15.97:6801/40468291","cluster_addr":"172.21.15.97:6803/40468291","heartbeat_back_addr":"172.21.15.97:6807/40468291","heartbeat_front_addr":"172.21.15.97:6805/40468291","state":["exists","up"]},{"osd":1,"uuid":"35949061-aa30-4aff-b39d-90aebeffb115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2443452537},{"type":"v1","addr":"172.21.15.97:6825","nonce":2443452537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2443452537},{"type":"v1","addr":"172.21.15.97:6827","nonce":2443452537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2443452537},{"type":"v1","addr":"172.21.15.97:6831","nonce":2443452537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2443452537},{"type":"v1","addr":"172.21.15.97:6829","nonce":2443452537}]},"public_addr":"172.21.15.97:6825/2443452537","cluster_addr":"172.21.15.97:6827/2443452537","heartbeat_back_addr":"172.21.15.97:6831/2443452537","heartbeat_front_addr":"172.21.15.97:6829/2443452537","state":["exists","up"]},{"osd":2,"uuid":"31b3e1c1-7648-44fe-9931-758ffe9a53ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6817","nonce":2425060925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6819","nonce":2425060925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6823","nonce":2425060925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6821","nonce":2425060925}]},"public_addr":"172.21.15.97:6817/2425060925","cluster_addr":"172.21.15.97:6819/2425060925","heartbeat_back_addr":"172.21.15.97:6823/2425060925","heartbeat_front_addr":"172.21.15.97:6821/2425060925","state":["exists","up"]},{"osd":3,"uuid":"d7daebbc-0e33-42ab-97ee-a6c1b449ae8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6809","nonce":3512666164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6811","nonce":3512666164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6815","nonce":3512666164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6813","nonce":3512666164}]},"public_addr":"172.21.15.97:6809/3512666164","cluster_addr":"172.21.15.97:6811/3512666164","heartbeat_back_addr":"172.21.15.97:6815/3512666164","heartbeat_front_addr":"172.21.15.97:6813/3512666164","state":["exists","up"]},{"osd":4,"uuid":"d55ca69b-5000-45ff-acc4-00f7c4763cad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6804","nonce":3191744356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6808","nonce":3191744356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6815","nonce":3191744356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6812","nonce":3191744356}]},"public_addr":"172.21.15.114:6804/3191744356","cluster_addr":"172.21.15.114:6808/3191744356","heartbeat_back_addr":"172.21.15.114:6815/3191744356","heartbeat_front_addr":"172.21.15.114:6812/3191744356","state":["exists","up"]},{"osd":5,"uuid":"196a329b-bd93-4cc2-85bb-64ed81d3918b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2340331281},{"type":"v1","addr":"172.21.15.114:6801","nonce":2340331281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6803","nonce":2340331281},{"type":"v1","addr":"172.21.15.114:6805","nonce":2340331281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6811","nonce":2340331281},{"type":"v1","addr":"172.21.15.114:6813","nonce":2340331281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6807","nonce":2340331281},{"type":"v1","addr":"172.21.15.114:6809","nonce":2340331281}]},"public_addr":"172.21.15.114:6801/2340331281","cluster_addr":"172.21.15.114:6805/2340331281","heartbeat_back_addr":"172.21.15.114:6813/2340331281","heartbeat_front_addr":"172.21.15.114:6809/2340331281","state":["exists","up"]},{"osd":6,"uuid":"dc9712f0-8858-467a-8397-d5eb97187165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6825","nonce":1148348470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6827","nonce":1148348470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6831","nonce":1148348470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6829","nonce":1148348470}]},"public_addr":"172.21.15.114:6825/1148348470","cluster_addr":"172.21.15.114:6827/1148348470","heartbeat_back_addr":"172.21.15.114:6831/1148348470","heartbeat_front_addr":"172.21.15.114:6829/1148348470","state":["exists","up"]},{"osd":7,"uuid":"b6d774ee-fe6d-462c-afa4-7ffd5d7343a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6817","nonce":2133497352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6819","nonce":2133497352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6823","nonce":2133497352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6821","nonce":2133497352}]},"public_addr":"172.21.15.114:6817/2133497352","cluster_addr":"172.21.15.114:6819/2133497352","heartbeat_back_addr":"172.21.15.114:6823/2133497352","heartbeat_front_addr":"172.21.15.114:6821/2133497352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.237241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.504107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.690795+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.197056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.459497+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.805108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.484317+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.741853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T00:17:53.539 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-18T00:17:53.539 INFO:tasks.ceph:Creating RBD pool 2024-09-18T00:17:53.540 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-18T00:17:54.453 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'rbd' created 2024-09-18T00:17:54.479 DEBUG:teuthology.orchestra.run.smithi097:> rbd --cluster ceph pool init rbd 2024-09-18T00:17:57.496 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-18T00:17:57.497 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-18T00:17:57.497 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-18T00:17:57.497 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-18T00:17:57.497 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-18T00:17:57.850 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:17:57.868 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1187789583},{"type":"v1","addr":"172.21.15.97:6833","nonce":1187789583}]},"active_addr":"172.21.15.97:6833/1187789583","active_change":"2024-09-18T00:17:40.287440+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":538316231}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":726437758}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":409574565}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3063010014}]}]} 2024-09-18T00:17:57.873 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-18T00:17:57.873 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T00:17:57.873 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T00:17:58.171 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:17:58.171 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":22,"fsid":"b993f56a-c4ef-4602-bb41-f32ae406b45a","created":"2024-09-18T00:17:35.658788+0000","modified":"2024-09-18T00:17:57.470286+0000","last_up_change":"2024-09-18T00:17:49.411529+0000","last_in_change":"2024-09-18T00:17:37.952065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T00:17:50.375367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T00:17:53.896718+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bfda7950-3d83-435d-8313-2f14345096ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6801","nonce":40468291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6803","nonce":40468291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6807","nonce":40468291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6805","nonce":40468291}]},"public_addr":"172.21.15.97:6801/40468291","cluster_addr":"172.21.15.97:6803/40468291","heartbeat_back_addr":"172.21.15.97:6807/40468291","heartbeat_front_addr":"172.21.15.97:6805/40468291","state":["exists","up"]},{"osd":1,"uuid":"35949061-aa30-4aff-b39d-90aebeffb115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2443452537},{"type":"v1","addr":"172.21.15.97:6825","nonce":2443452537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2443452537},{"type":"v1","addr":"172.21.15.97:6827","nonce":2443452537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2443452537},{"type":"v1","addr":"172.21.15.97:6831","nonce":2443452537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2443452537},{"type":"v1","addr":"172.21.15.97:6829","nonce":2443452537}]},"public_addr":"172.21.15.97:6825/2443452537","cluster_addr":"172.21.15.97:6827/2443452537","heartbeat_back_addr":"172.21.15.97:6831/2443452537","heartbeat_front_addr":"172.21.15.97:6829/2443452537","state":["exists","up"]},{"osd":2,"uuid":"31b3e1c1-7648-44fe-9931-758ffe9a53ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6817","nonce":2425060925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6819","nonce":2425060925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6823","nonce":2425060925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6821","nonce":2425060925}]},"public_addr":"172.21.15.97:6817/2425060925","cluster_addr":"172.21.15.97:6819/2425060925","heartbeat_back_addr":"172.21.15.97:6823/2425060925","heartbeat_front_addr":"172.21.15.97:6821/2425060925","state":["exists","up"]},{"osd":3,"uuid":"d7daebbc-0e33-42ab-97ee-a6c1b449ae8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6809","nonce":3512666164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6811","nonce":3512666164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6815","nonce":3512666164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6813","nonce":3512666164}]},"public_addr":"172.21.15.97:6809/3512666164","cluster_addr":"172.21.15.97:6811/3512666164","heartbeat_back_addr":"172.21.15.97:6815/3512666164","heartbeat_front_addr":"172.21.15.97:6813/3512666164","state":["exists","up"]},{"osd":4,"uuid":"d55ca69b-5000-45ff-acc4-00f7c4763cad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6804","nonce":3191744356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6808","nonce":3191744356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6815","nonce":3191744356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6812","nonce":3191744356}]},"public_addr":"172.21.15.114:6804/3191744356","cluster_addr":"172.21.15.114:6808/3191744356","heartbeat_back_addr":"172.21.15.114:6815/3191744356","heartbeat_front_addr":"172.21.15.114:6812/3191744356","state":["exists","up"]},{"osd":5,"uuid":"196a329b-bd93-4cc2-85bb-64ed81d3918b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2340331281},{"type":"v1","addr":"172.21.15.114:6801","nonce":2340331281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6803","nonce":2340331281},{"type":"v1","addr":"172.21.15.114:6805","nonce":2340331281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6811","nonce":2340331281},{"type":"v1","addr":"172.21.15.114:6813","nonce":2340331281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6807","nonce":2340331281},{"type":"v1","addr":"172.21.15.114:6809","nonce":2340331281}]},"public_addr":"172.21.15.114:6801/2340331281","cluster_addr":"172.21.15.114:6805/2340331281","heartbeat_back_addr":"172.21.15.114:6813/2340331281","heartbeat_front_addr":"172.21.15.114:6809/2340331281","state":["exists","up"]},{"osd":6,"uuid":"dc9712f0-8858-467a-8397-d5eb97187165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6825","nonce":1148348470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6827","nonce":1148348470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6831","nonce":1148348470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6829","nonce":1148348470}]},"public_addr":"172.21.15.114:6825/1148348470","cluster_addr":"172.21.15.114:6827/1148348470","heartbeat_back_addr":"172.21.15.114:6831/1148348470","heartbeat_front_addr":"172.21.15.114:6829/1148348470","state":["exists","up"]},{"osd":7,"uuid":"b6d774ee-fe6d-462c-afa4-7ffd5d7343a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6817","nonce":2133497352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6819","nonce":2133497352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6823","nonce":2133497352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6821","nonce":2133497352}]},"public_addr":"172.21.15.114:6817/2133497352","cluster_addr":"172.21.15.114:6819/2133497352","heartbeat_back_addr":"172.21.15.114:6823/2133497352","heartbeat_front_addr":"172.21.15.114:6821/2133497352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.237241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.504107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.690795+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.197056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.459497+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.805108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.484317+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.741853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T00:17:58.190 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T00:17:58.190 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T00:17:58.487 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:17:58.487 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":22,"fsid":"b993f56a-c4ef-4602-bb41-f32ae406b45a","created":"2024-09-18T00:17:35.658788+0000","modified":"2024-09-18T00:17:57.470286+0000","last_up_change":"2024-09-18T00:17:49.411529+0000","last_in_change":"2024-09-18T00:17:37.952065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T00:17:50.375367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T00:17:53.896718+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bfda7950-3d83-435d-8313-2f14345096ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6801","nonce":40468291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6803","nonce":40468291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6807","nonce":40468291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6805","nonce":40468291}]},"public_addr":"172.21.15.97:6801/40468291","cluster_addr":"172.21.15.97:6803/40468291","heartbeat_back_addr":"172.21.15.97:6807/40468291","heartbeat_front_addr":"172.21.15.97:6805/40468291","state":["exists","up"]},{"osd":1,"uuid":"35949061-aa30-4aff-b39d-90aebeffb115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2443452537},{"type":"v1","addr":"172.21.15.97:6825","nonce":2443452537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2443452537},{"type":"v1","addr":"172.21.15.97:6827","nonce":2443452537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":2443452537},{"type":"v1","addr":"172.21.15.97:6831","nonce":2443452537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":2443452537},{"type":"v1","addr":"172.21.15.97:6829","nonce":2443452537}]},"public_addr":"172.21.15.97:6825/2443452537","cluster_addr":"172.21.15.97:6827/2443452537","heartbeat_back_addr":"172.21.15.97:6831/2443452537","heartbeat_front_addr":"172.21.15.97:6829/2443452537","state":["exists","up"]},{"osd":2,"uuid":"31b3e1c1-7648-44fe-9931-758ffe9a53ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6817","nonce":2425060925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6819","nonce":2425060925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6823","nonce":2425060925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6821","nonce":2425060925}]},"public_addr":"172.21.15.97:6817/2425060925","cluster_addr":"172.21.15.97:6819/2425060925","heartbeat_back_addr":"172.21.15.97:6823/2425060925","heartbeat_front_addr":"172.21.15.97:6821/2425060925","state":["exists","up"]},{"osd":3,"uuid":"d7daebbc-0e33-42ab-97ee-a6c1b449ae8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6809","nonce":3512666164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6811","nonce":3512666164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6815","nonce":3512666164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6813","nonce":3512666164}]},"public_addr":"172.21.15.97:6809/3512666164","cluster_addr":"172.21.15.97:6811/3512666164","heartbeat_back_addr":"172.21.15.97:6815/3512666164","heartbeat_front_addr":"172.21.15.97:6813/3512666164","state":["exists","up"]},{"osd":4,"uuid":"d55ca69b-5000-45ff-acc4-00f7c4763cad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6804","nonce":3191744356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6808","nonce":3191744356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6815","nonce":3191744356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6812","nonce":3191744356}]},"public_addr":"172.21.15.114:6804/3191744356","cluster_addr":"172.21.15.114:6808/3191744356","heartbeat_back_addr":"172.21.15.114:6815/3191744356","heartbeat_front_addr":"172.21.15.114:6812/3191744356","state":["exists","up"]},{"osd":5,"uuid":"196a329b-bd93-4cc2-85bb-64ed81d3918b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2340331281},{"type":"v1","addr":"172.21.15.114:6801","nonce":2340331281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6803","nonce":2340331281},{"type":"v1","addr":"172.21.15.114:6805","nonce":2340331281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6811","nonce":2340331281},{"type":"v1","addr":"172.21.15.114:6813","nonce":2340331281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6807","nonce":2340331281},{"type":"v1","addr":"172.21.15.114:6809","nonce":2340331281}]},"public_addr":"172.21.15.114:6801/2340331281","cluster_addr":"172.21.15.114:6805/2340331281","heartbeat_back_addr":"172.21.15.114:6813/2340331281","heartbeat_front_addr":"172.21.15.114:6809/2340331281","state":["exists","up"]},{"osd":6,"uuid":"dc9712f0-8858-467a-8397-d5eb97187165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6825","nonce":1148348470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6827","nonce":1148348470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6831","nonce":1148348470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6829","nonce":1148348470}]},"public_addr":"172.21.15.114:6825/1148348470","cluster_addr":"172.21.15.114:6827/1148348470","heartbeat_back_addr":"172.21.15.114:6831/1148348470","heartbeat_front_addr":"172.21.15.114:6829/1148348470","state":["exists","up"]},{"osd":7,"uuid":"b6d774ee-fe6d-462c-afa4-7ffd5d7343a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6817","nonce":2133497352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6819","nonce":2133497352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6823","nonce":2133497352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6821","nonce":2133497352}]},"public_addr":"172.21.15.114:6817/2133497352","cluster_addr":"172.21.15.114:6819/2133497352","heartbeat_back_addr":"172.21.15.114:6823/2133497352","heartbeat_front_addr":"172.21.15.114:6821/2133497352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.237241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.504107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.690795+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.197056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.459497+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.805108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.484317+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.741853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T00:17:58.507 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T00:17:58.507 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T00:17:58.508 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T00:17:58.508 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T00:17:58.508 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T00:17:58.509 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T00:17:58.509 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-18T00:17:58.509 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-18T00:17:58.743 INFO:teuthology.orchestra.run.smithi097.stdout:64424509443 2024-09-18T00:17:58.744 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T00:17:58.776 INFO:teuthology.orchestra.run.smithi097.stdout:60129542149 2024-09-18T00:17:58.777 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T00:17:58.777 INFO:teuthology.orchestra.run.smithi097.stdout:64424509445 2024-09-18T00:17:58.778 INFO:teuthology.orchestra.run.smithi097.stdout:64424509444 2024-09-18T00:17:58.778 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T00:17:58.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T00:17:58.780 INFO:teuthology.orchestra.run.smithi097.stdout:64424509445 2024-09-18T00:17:58.780 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T00:17:58.781 INFO:teuthology.orchestra.run.smithi097.stdout:60129542148 2024-09-18T00:17:58.781 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T00:17:58.790 INFO:teuthology.orchestra.run.smithi097.stdout:64424509446 2024-09-18T00:17:58.790 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T00:17:58.792 INFO:teuthology.orchestra.run.smithi097.stdout:64424509446 2024-09-18T00:17:58.792 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T00:17:59.238 INFO:teuthology.orchestra.run.smithi097.stdout:64424509442 2024-09-18T00:17:59.263 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.7 2024-09-18T00:17:59.283 INFO:teuthology.orchestra.run.smithi097.stdout:64424509444 2024-09-18T00:17:59.303 INFO:teuthology.orchestra.run.smithi097.stdout:64424509445 2024-09-18T00:17:59.309 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.4 2024-09-18T00:17:59.314 INFO:teuthology.orchestra.run.smithi097.stdout:64424509444 2024-09-18T00:17:59.316 INFO:teuthology.orchestra.run.smithi097.stdout:60129542148 2024-09-18T00:17:59.319 INFO:teuthology.orchestra.run.smithi097.stdout:60129542147 2024-09-18T00:17:59.332 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.1 2024-09-18T00:17:59.340 INFO:teuthology.orchestra.run.smithi097.stdout:64424509443 2024-09-18T00:17:59.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.3 2024-09-18T00:17:59.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.5 2024-09-18T00:17:59.345 INFO:teuthology.orchestra.run.smithi097.stdout:64424509445 2024-09-18T00:17:59.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.0 2024-09-18T00:17:59.360 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.6 2024-09-18T00:17:59.364 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.2 2024-09-18T00:18:00.265 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T00:18:00.310 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T00:18:00.333 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T00:18:00.344 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T00:18:00.344 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T00:18:00.350 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T00:18:00.360 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T00:18:00.366 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T00:18:00.770 INFO:teuthology.orchestra.run.smithi097.stdout:64424509443 2024-09-18T00:18:00.796 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.7 2024-09-18T00:18:00.797 DEBUG:teuthology.parallel:result is None 2024-09-18T00:18:00.827 INFO:teuthology.orchestra.run.smithi097.stdout:64424509445 2024-09-18T00:18:00.846 INFO:teuthology.orchestra.run.smithi097.stdout:60129542150 2024-09-18T00:18:00.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.4 2024-09-18T00:18:00.852 DEBUG:teuthology.parallel:result is None 2024-09-18T00:18:00.857 INFO:teuthology.orchestra.run.smithi097.stdout:64424509444 2024-09-18T00:18:00.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.0 2024-09-18T00:18:00.867 DEBUG:teuthology.parallel:result is None 2024-09-18T00:18:00.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.6 2024-09-18T00:18:00.877 DEBUG:teuthology.parallel:result is None 2024-09-18T00:18:00.878 INFO:teuthology.orchestra.run.smithi097.stdout:60129542149 2024-09-18T00:18:00.882 INFO:teuthology.orchestra.run.smithi097.stdout:64424509445 2024-09-18T00:18:00.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.3 2024-09-18T00:18:00.905 DEBUG:teuthology.parallel:result is None 2024-09-18T00:18:00.906 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.5 2024-09-18T00:18:00.907 DEBUG:teuthology.parallel:result is None 2024-09-18T00:18:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:64424509446 2024-09-18T00:18:00.932 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.1 2024-09-18T00:18:00.932 DEBUG:teuthology.parallel:result is None 2024-09-18T00:18:00.946 INFO:teuthology.orchestra.run.smithi097.stdout:64424509446 2024-09-18T00:18:00.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.2 2024-09-18T00:18:00.963 DEBUG:teuthology.parallel:result is None 2024-09-18T00:18:00.963 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T00:18:00.963 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:18:01.274 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:18:01.274 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-18T00:18:01.291 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-18T00:18:00.299944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":222260,"kb_used_data":1848,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754752460,"statfs":{"total":773094113280,"available":772866519040,"internally_reserved":0,"allocated":1892352,"data_stored":1777574,"data_compressed":12342,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.835671"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.535439+0000","last_change":"2024-09-18T00:17:57.535928+0000","last_active":"2024-09-18T00:17:57.535439+0000","last_peered":"2024-09-18T00:17:57.535439+0000","last_clean":"2024-09-18T00:17:57.535439+0000","last_became_active":"2024-09-18T00:17:55.481587+0000","last_became_peered":"2024-09-18T00:17:55.481587+0000","last_unstale":"2024-09-18T00:17:57.535439+0000","last_undegraded":"2024-09-18T00:17:57.535439+0000","last_fullsized":"2024-09-18T00:17:57.535439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:48:20.128718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089554000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.479678+0000","last_change":"2024-09-18T00:17:57.480235+0000","last_active":"2024-09-18T00:17:57.479678+0000","last_peered":"2024-09-18T00:17:57.479678+0000","last_clean":"2024-09-18T00:17:57.479678+0000","last_became_active":"2024-09-18T00:17:55.478616+0000","last_became_peered":"2024-09-18T00:17:55.478616+0000","last_unstale":"2024-09-18T00:17:57.479678+0000","last_undegraded":"2024-09-18T00:17:57.479678+0000","last_fullsized":"2024-09-18T00:17:57.479678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:12:32.092294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097420299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.733728+0000","last_change":"2024-09-18T00:17:57.734136+0000","last_active":"2024-09-18T00:17:57.733728+0000","last_peered":"2024-09-18T00:17:57.733728+0000","last_clean":"2024-09-18T00:17:57.733728+0000","last_became_active":"2024-09-18T00:17:55.548455+0000","last_became_peered":"2024-09-18T00:17:55.548455+0000","last_unstale":"2024-09-18T00:17:57.733728+0000","last_undegraded":"2024-09-18T00:17:57.733728+0000","last_fullsized":"2024-09-18T00:17:57.733728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:22:42.276618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000768164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.479518+0000","last_change":"2024-09-18T00:17:57.479917+0000","last_active":"2024-09-18T00:17:57.479518+0000","last_peered":"2024-09-18T00:17:57.479518+0000","last_clean":"2024-09-18T00:17:57.479518+0000","last_became_active":"2024-09-18T00:17:55.549824+0000","last_became_peered":"2024-09-18T00:17:55.549824+0000","last_unstale":"2024-09-18T00:17:57.479518+0000","last_undegraded":"2024-09-18T00:17:57.479518+0000","last_fullsized":"2024-09-18T00:17:57.479518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:48.269029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.478618+0000","last_change":"2024-09-18T00:17:57.478618+0000","last_active":"2024-09-18T00:17:57.478618+0000","last_peered":"2024-09-18T00:17:57.478618+0000","last_clean":"2024-09-18T00:17:57.478618+0000","last_became_active":"2024-09-18T00:17:55.477945+0000","last_became_peered":"2024-09-18T00:17:55.477945+0000","last_unstale":"2024-09-18T00:17:57.478618+0000","last_undegraded":"2024-09-18T00:17:57.478618+0000","last_fullsized":"2024-09-18T00:17:57.478618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:27.096993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088189300000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.476969+0000","last_change":"2024-09-18T00:17:57.477633+0000","last_active":"2024-09-18T00:17:57.476969+0000","last_peered":"2024-09-18T00:17:57.476969+0000","last_clean":"2024-09-18T00:17:57.476969+0000","last_became_active":"2024-09-18T00:17:55.631277+0000","last_became_peered":"2024-09-18T00:17:55.631277+0000","last_unstale":"2024-09-18T00:17:57.476969+0000","last_undegraded":"2024-09-18T00:17:57.476969+0000","last_fullsized":"2024-09-18T00:17:57.476969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:55.421684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001141204,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.733871+0000","last_change":"2024-09-18T00:17:57.734248+0000","last_active":"2024-09-18T00:17:57.733871+0000","last_peered":"2024-09-18T00:17:57.733871+0000","last_clean":"2024-09-18T00:17:57.733871+0000","last_became_active":"2024-09-18T00:17:55.478064+0000","last_became_peered":"2024-09-18T00:17:55.478064+0000","last_unstale":"2024-09-18T00:17:57.733871+0000","last_undegraded":"2024-09-18T00:17:57.733871+0000","last_fullsized":"2024-09-18T00:17:57.733871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:49:15.359212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085153599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.478175+0000","last_change":"2024-09-18T00:17:57.478704+0000","last_active":"2024-09-18T00:17:57.478175+0000","last_peered":"2024-09-18T00:17:57.478175+0000","last_clean":"2024-09-18T00:17:57.478175+0000","last_became_active":"2024-09-18T00:17:55.477618+0000","last_became_peered":"2024-09-18T00:17:55.477618+0000","last_unstale":"2024-09-18T00:17:57.478175+0000","last_undegraded":"2024-09-18T00:17:57.478175+0000","last_fullsized":"2024-09-18T00:17:57.478175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:11.961859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089022700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.733592+0000","last_change":"2024-09-18T00:17:51.481781+0000","last_active":"2024-09-18T00:17:57.733592+0000","last_peered":"2024-09-18T00:17:57.733592+0000","last_clean":"2024-09-18T00:17:57.733592+0000","last_became_active":"2024-09-18T00:17:51.476635+0000","last_became_peered":"2024-09-18T00:17:51.476635+0000","last_unstale":"2024-09-18T00:17:57.733592+0000","last_undegraded":"2024-09-18T00:17:57.733592+0000","last_fullsized":"2024-09-18T00:17:57.733592+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_clean_scrub_stamp":"2024-09-18T00:17:50.414591+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:09:43.936255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12342,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":664968,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":664968,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:18:01.292 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:18:01.572 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:18:01.572 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-18T00:18:01.589 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-18T00:18:00.299944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":222260,"kb_used_data":1848,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754752460,"statfs":{"total":773094113280,"available":772866519040,"internally_reserved":0,"allocated":1892352,"data_stored":1777574,"data_compressed":12342,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.835671"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.535439+0000","last_change":"2024-09-18T00:17:57.535928+0000","last_active":"2024-09-18T00:17:57.535439+0000","last_peered":"2024-09-18T00:17:57.535439+0000","last_clean":"2024-09-18T00:17:57.535439+0000","last_became_active":"2024-09-18T00:17:55.481587+0000","last_became_peered":"2024-09-18T00:17:55.481587+0000","last_unstale":"2024-09-18T00:17:57.535439+0000","last_undegraded":"2024-09-18T00:17:57.535439+0000","last_fullsized":"2024-09-18T00:17:57.535439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:48:20.128718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089554000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.479678+0000","last_change":"2024-09-18T00:17:57.480235+0000","last_active":"2024-09-18T00:17:57.479678+0000","last_peered":"2024-09-18T00:17:57.479678+0000","last_clean":"2024-09-18T00:17:57.479678+0000","last_became_active":"2024-09-18T00:17:55.478616+0000","last_became_peered":"2024-09-18T00:17:55.478616+0000","last_unstale":"2024-09-18T00:17:57.479678+0000","last_undegraded":"2024-09-18T00:17:57.479678+0000","last_fullsized":"2024-09-18T00:17:57.479678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:12:32.092294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097420299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.733728+0000","last_change":"2024-09-18T00:17:57.734136+0000","last_active":"2024-09-18T00:17:57.733728+0000","last_peered":"2024-09-18T00:17:57.733728+0000","last_clean":"2024-09-18T00:17:57.733728+0000","last_became_active":"2024-09-18T00:17:55.548455+0000","last_became_peered":"2024-09-18T00:17:55.548455+0000","last_unstale":"2024-09-18T00:17:57.733728+0000","last_undegraded":"2024-09-18T00:17:57.733728+0000","last_fullsized":"2024-09-18T00:17:57.733728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:22:42.276618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000768164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.479518+0000","last_change":"2024-09-18T00:17:57.479917+0000","last_active":"2024-09-18T00:17:57.479518+0000","last_peered":"2024-09-18T00:17:57.479518+0000","last_clean":"2024-09-18T00:17:57.479518+0000","last_became_active":"2024-09-18T00:17:55.549824+0000","last_became_peered":"2024-09-18T00:17:55.549824+0000","last_unstale":"2024-09-18T00:17:57.479518+0000","last_undegraded":"2024-09-18T00:17:57.479518+0000","last_fullsized":"2024-09-18T00:17:57.479518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:48.269029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.478618+0000","last_change":"2024-09-18T00:17:57.478618+0000","last_active":"2024-09-18T00:17:57.478618+0000","last_peered":"2024-09-18T00:17:57.478618+0000","last_clean":"2024-09-18T00:17:57.478618+0000","last_became_active":"2024-09-18T00:17:55.477945+0000","last_became_peered":"2024-09-18T00:17:55.477945+0000","last_unstale":"2024-09-18T00:17:57.478618+0000","last_undegraded":"2024-09-18T00:17:57.478618+0000","last_fullsized":"2024-09-18T00:17:57.478618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:27.096993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088189300000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.476969+0000","last_change":"2024-09-18T00:17:57.477633+0000","last_active":"2024-09-18T00:17:57.476969+0000","last_peered":"2024-09-18T00:17:57.476969+0000","last_clean":"2024-09-18T00:17:57.476969+0000","last_became_active":"2024-09-18T00:17:55.631277+0000","last_became_peered":"2024-09-18T00:17:55.631277+0000","last_unstale":"2024-09-18T00:17:57.476969+0000","last_undegraded":"2024-09-18T00:17:57.476969+0000","last_fullsized":"2024-09-18T00:17:57.476969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:55.421684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001141204,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.733871+0000","last_change":"2024-09-18T00:17:57.734248+0000","last_active":"2024-09-18T00:17:57.733871+0000","last_peered":"2024-09-18T00:17:57.733871+0000","last_clean":"2024-09-18T00:17:57.733871+0000","last_became_active":"2024-09-18T00:17:55.478064+0000","last_became_peered":"2024-09-18T00:17:55.478064+0000","last_unstale":"2024-09-18T00:17:57.733871+0000","last_undegraded":"2024-09-18T00:17:57.733871+0000","last_fullsized":"2024-09-18T00:17:57.733871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:49:15.359212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085153599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.478175+0000","last_change":"2024-09-18T00:17:57.478704+0000","last_active":"2024-09-18T00:17:57.478175+0000","last_peered":"2024-09-18T00:17:57.478175+0000","last_clean":"2024-09-18T00:17:57.478175+0000","last_became_active":"2024-09-18T00:17:55.477618+0000","last_became_peered":"2024-09-18T00:17:55.477618+0000","last_unstale":"2024-09-18T00:17:57.478175+0000","last_undegraded":"2024-09-18T00:17:57.478175+0000","last_fullsized":"2024-09-18T00:17:57.478175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:11.961859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089022700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.733592+0000","last_change":"2024-09-18T00:17:51.481781+0000","last_active":"2024-09-18T00:17:57.733592+0000","last_peered":"2024-09-18T00:17:57.733592+0000","last_clean":"2024-09-18T00:17:57.733592+0000","last_became_active":"2024-09-18T00:17:51.476635+0000","last_became_peered":"2024-09-18T00:17:51.476635+0000","last_unstale":"2024-09-18T00:17:57.733592+0000","last_undegraded":"2024-09-18T00:17:57.733592+0000","last_fullsized":"2024-09-18T00:17:57.733592+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_clean_scrub_stamp":"2024-09-18T00:17:50.414591+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:09:43.936255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12342,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":664968,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":664968,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:18:01.590 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T00:18:01.590 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-18T00:18:01.590 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-18T00:18:01.590 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-18T00:18:01.899 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:18:01.899 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-18T00:18:01.917 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-18T00:18:01.917 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-18T00:18:01.926 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-18T00:18:01.926 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7'}} 2024-09-18T00:18:01.926 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-18T00:18:01.926 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-18T00:18:02.227 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-18T00:18:02.227 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T00:18:02.227 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:18:02.524 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:18:02.525 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-18T00:18:02.541 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-18T00:18:02.300463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":222260,"kb_used_data":1848,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754752460,"statfs":{"total":773094113280,"available":772866519040,"internally_reserved":0,"allocated":1892352,"data_stored":1777574,"data_compressed":12342,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.836191"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.535439+0000","last_change":"2024-09-18T00:17:57.535928+0000","last_active":"2024-09-18T00:17:57.535439+0000","last_peered":"2024-09-18T00:17:57.535439+0000","last_clean":"2024-09-18T00:17:57.535439+0000","last_became_active":"2024-09-18T00:17:55.481587+0000","last_became_peered":"2024-09-18T00:17:55.481587+0000","last_unstale":"2024-09-18T00:17:57.535439+0000","last_undegraded":"2024-09-18T00:17:57.535439+0000","last_fullsized":"2024-09-18T00:17:57.535439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:48:20.128718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089554000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.479678+0000","last_change":"2024-09-18T00:17:57.480235+0000","last_active":"2024-09-18T00:17:57.479678+0000","last_peered":"2024-09-18T00:17:57.479678+0000","last_clean":"2024-09-18T00:17:57.479678+0000","last_became_active":"2024-09-18T00:17:55.478616+0000","last_became_peered":"2024-09-18T00:17:55.478616+0000","last_unstale":"2024-09-18T00:17:57.479678+0000","last_undegraded":"2024-09-18T00:17:57.479678+0000","last_fullsized":"2024-09-18T00:17:57.479678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:12:32.092294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097420299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.733728+0000","last_change":"2024-09-18T00:17:57.734136+0000","last_active":"2024-09-18T00:17:57.733728+0000","last_peered":"2024-09-18T00:17:57.733728+0000","last_clean":"2024-09-18T00:17:57.733728+0000","last_became_active":"2024-09-18T00:17:55.548455+0000","last_became_peered":"2024-09-18T00:17:55.548455+0000","last_unstale":"2024-09-18T00:17:57.733728+0000","last_undegraded":"2024-09-18T00:17:57.733728+0000","last_fullsized":"2024-09-18T00:17:57.733728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:22:42.276618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000768164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.479518+0000","last_change":"2024-09-18T00:17:57.479917+0000","last_active":"2024-09-18T00:17:57.479518+0000","last_peered":"2024-09-18T00:17:57.479518+0000","last_clean":"2024-09-18T00:17:57.479518+0000","last_became_active":"2024-09-18T00:17:55.549824+0000","last_became_peered":"2024-09-18T00:17:55.549824+0000","last_unstale":"2024-09-18T00:17:57.479518+0000","last_undegraded":"2024-09-18T00:17:57.479518+0000","last_fullsized":"2024-09-18T00:17:57.479518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:48.269029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.478618+0000","last_change":"2024-09-18T00:17:57.478618+0000","last_active":"2024-09-18T00:17:57.478618+0000","last_peered":"2024-09-18T00:17:57.478618+0000","last_clean":"2024-09-18T00:17:57.478618+0000","last_became_active":"2024-09-18T00:17:55.477945+0000","last_became_peered":"2024-09-18T00:17:55.477945+0000","last_unstale":"2024-09-18T00:17:57.478618+0000","last_undegraded":"2024-09-18T00:17:57.478618+0000","last_fullsized":"2024-09-18T00:17:57.478618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:27.096993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088189300000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.476969+0000","last_change":"2024-09-18T00:17:57.477633+0000","last_active":"2024-09-18T00:17:57.476969+0000","last_peered":"2024-09-18T00:17:57.476969+0000","last_clean":"2024-09-18T00:17:57.476969+0000","last_became_active":"2024-09-18T00:17:55.631277+0000","last_became_peered":"2024-09-18T00:17:55.631277+0000","last_unstale":"2024-09-18T00:17:57.476969+0000","last_undegraded":"2024-09-18T00:17:57.476969+0000","last_fullsized":"2024-09-18T00:17:57.476969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:55.421684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001141204,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.733871+0000","last_change":"2024-09-18T00:17:57.734248+0000","last_active":"2024-09-18T00:17:57.733871+0000","last_peered":"2024-09-18T00:17:57.733871+0000","last_clean":"2024-09-18T00:17:57.733871+0000","last_became_active":"2024-09-18T00:17:55.478064+0000","last_became_peered":"2024-09-18T00:17:55.478064+0000","last_unstale":"2024-09-18T00:17:57.733871+0000","last_undegraded":"2024-09-18T00:17:57.733871+0000","last_fullsized":"2024-09-18T00:17:57.733871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:49:15.359212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085153599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.478175+0000","last_change":"2024-09-18T00:17:57.478704+0000","last_active":"2024-09-18T00:17:57.478175+0000","last_peered":"2024-09-18T00:17:57.478175+0000","last_clean":"2024-09-18T00:17:57.478175+0000","last_became_active":"2024-09-18T00:17:55.477618+0000","last_became_peered":"2024-09-18T00:17:55.477618+0000","last_unstale":"2024-09-18T00:17:57.478175+0000","last_undegraded":"2024-09-18T00:17:57.478175+0000","last_fullsized":"2024-09-18T00:17:57.478175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:11.961859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089022700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.733592+0000","last_change":"2024-09-18T00:17:51.481781+0000","last_active":"2024-09-18T00:17:57.733592+0000","last_peered":"2024-09-18T00:17:57.733592+0000","last_clean":"2024-09-18T00:17:57.733592+0000","last_became_active":"2024-09-18T00:17:51.476635+0000","last_became_peered":"2024-09-18T00:17:51.476635+0000","last_unstale":"2024-09-18T00:17:57.733592+0000","last_undegraded":"2024-09-18T00:17:57.733592+0000","last_fullsized":"2024-09-18T00:17:57.733592+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_clean_scrub_stamp":"2024-09-18T00:17:50.414591+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:09:43.936255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12342,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":664968,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":664968,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:18:02.542 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:18:02.822 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:18:02.822 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-18T00:18:02.838 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-18T00:18:02.300463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":222260,"kb_used_data":1848,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754752460,"statfs":{"total":773094113280,"available":772866519040,"internally_reserved":0,"allocated":1892352,"data_stored":1777574,"data_compressed":12342,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.836191"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.535439+0000","last_change":"2024-09-18T00:17:57.535928+0000","last_active":"2024-09-18T00:17:57.535439+0000","last_peered":"2024-09-18T00:17:57.535439+0000","last_clean":"2024-09-18T00:17:57.535439+0000","last_became_active":"2024-09-18T00:17:55.481587+0000","last_became_peered":"2024-09-18T00:17:55.481587+0000","last_unstale":"2024-09-18T00:17:57.535439+0000","last_undegraded":"2024-09-18T00:17:57.535439+0000","last_fullsized":"2024-09-18T00:17:57.535439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:48:20.128718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089554000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.479678+0000","last_change":"2024-09-18T00:17:57.480235+0000","last_active":"2024-09-18T00:17:57.479678+0000","last_peered":"2024-09-18T00:17:57.479678+0000","last_clean":"2024-09-18T00:17:57.479678+0000","last_became_active":"2024-09-18T00:17:55.478616+0000","last_became_peered":"2024-09-18T00:17:55.478616+0000","last_unstale":"2024-09-18T00:17:57.479678+0000","last_undegraded":"2024-09-18T00:17:57.479678+0000","last_fullsized":"2024-09-18T00:17:57.479678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:12:32.092294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097420299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.733728+0000","last_change":"2024-09-18T00:17:57.734136+0000","last_active":"2024-09-18T00:17:57.733728+0000","last_peered":"2024-09-18T00:17:57.733728+0000","last_clean":"2024-09-18T00:17:57.733728+0000","last_became_active":"2024-09-18T00:17:55.548455+0000","last_became_peered":"2024-09-18T00:17:55.548455+0000","last_unstale":"2024-09-18T00:17:57.733728+0000","last_undegraded":"2024-09-18T00:17:57.733728+0000","last_fullsized":"2024-09-18T00:17:57.733728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:22:42.276618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000768164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.479518+0000","last_change":"2024-09-18T00:17:57.479917+0000","last_active":"2024-09-18T00:17:57.479518+0000","last_peered":"2024-09-18T00:17:57.479518+0000","last_clean":"2024-09-18T00:17:57.479518+0000","last_became_active":"2024-09-18T00:17:55.549824+0000","last_became_peered":"2024-09-18T00:17:55.549824+0000","last_unstale":"2024-09-18T00:17:57.479518+0000","last_undegraded":"2024-09-18T00:17:57.479518+0000","last_fullsized":"2024-09-18T00:17:57.479518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:48.269029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.478618+0000","last_change":"2024-09-18T00:17:57.478618+0000","last_active":"2024-09-18T00:17:57.478618+0000","last_peered":"2024-09-18T00:17:57.478618+0000","last_clean":"2024-09-18T00:17:57.478618+0000","last_became_active":"2024-09-18T00:17:55.477945+0000","last_became_peered":"2024-09-18T00:17:55.477945+0000","last_unstale":"2024-09-18T00:17:57.478618+0000","last_undegraded":"2024-09-18T00:17:57.478618+0000","last_fullsized":"2024-09-18T00:17:57.478618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:27.096993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088189300000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.476969+0000","last_change":"2024-09-18T00:17:57.477633+0000","last_active":"2024-09-18T00:17:57.476969+0000","last_peered":"2024-09-18T00:17:57.476969+0000","last_clean":"2024-09-18T00:17:57.476969+0000","last_became_active":"2024-09-18T00:17:55.631277+0000","last_became_peered":"2024-09-18T00:17:55.631277+0000","last_unstale":"2024-09-18T00:17:57.476969+0000","last_undegraded":"2024-09-18T00:17:57.476969+0000","last_fullsized":"2024-09-18T00:17:57.476969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:55.421684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001141204,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.733871+0000","last_change":"2024-09-18T00:17:57.734248+0000","last_active":"2024-09-18T00:17:57.733871+0000","last_peered":"2024-09-18T00:17:57.733871+0000","last_clean":"2024-09-18T00:17:57.733871+0000","last_became_active":"2024-09-18T00:17:55.478064+0000","last_became_peered":"2024-09-18T00:17:55.478064+0000","last_unstale":"2024-09-18T00:17:57.733871+0000","last_undegraded":"2024-09-18T00:17:57.733871+0000","last_fullsized":"2024-09-18T00:17:57.733871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:49:15.359212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085153599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.478175+0000","last_change":"2024-09-18T00:17:57.478704+0000","last_active":"2024-09-18T00:17:57.478175+0000","last_peered":"2024-09-18T00:17:57.478175+0000","last_clean":"2024-09-18T00:17:57.478175+0000","last_became_active":"2024-09-18T00:17:55.477618+0000","last_became_peered":"2024-09-18T00:17:55.477618+0000","last_unstale":"2024-09-18T00:17:57.478175+0000","last_undegraded":"2024-09-18T00:17:57.478175+0000","last_fullsized":"2024-09-18T00:17:57.478175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:11.961859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089022700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-18T00:17:57.733592+0000","last_change":"2024-09-18T00:17:51.481781+0000","last_active":"2024-09-18T00:17:57.733592+0000","last_peered":"2024-09-18T00:17:57.733592+0000","last_clean":"2024-09-18T00:17:57.733592+0000","last_became_active":"2024-09-18T00:17:51.476635+0000","last_became_peered":"2024-09-18T00:17:51.476635+0000","last_unstale":"2024-09-18T00:17:57.733592+0000","last_undegraded":"2024-09-18T00:17:57.733592+0000","last_fullsized":"2024-09-18T00:17:57.733592+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_clean_scrub_stamp":"2024-09-18T00:17:50.414591+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:09:43.936255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12342,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":664968,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":664968,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:18:02.839 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T00:18:02.839 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-18T00:18:03.128 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 22 2024-09-18T00:18:03.128 INFO:teuthology.orchestra.run.smithi097.stdout:fsid b993f56a-c4ef-4602-bb41-f32ae406b45a 2024-09-18T00:18:03.128 INFO:teuthology.orchestra.run.smithi097.stdout:created 2024-09-18T00:17:35.658788+0000 2024-09-18T00:18:03.128 INFO:teuthology.orchestra.run.smithi097.stdout:modified 2024-09-18T00:17:57.470286+0000 2024-09-18T00:18:03.128 INFO:teuthology.orchestra.run.smithi097.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-18T00:18:03.128 INFO:teuthology.orchestra.run.smithi097.stdout:crush_version 5 2024-09-18T00:18:03.128 INFO:teuthology.orchestra.run.smithi097.stdout:full_ratio 0.95 2024-09-18T00:18:03.128 INFO:teuthology.orchestra.run.smithi097.stdout:backfillfull_ratio 0.9 2024-09-18T00:18:03.128 INFO:teuthology.orchestra.run.smithi097.stdout:nearfull_ratio 0.85 2024-09-18T00:18:03.129 INFO:teuthology.orchestra.run.smithi097.stdout:require_min_compat_client luminous 2024-09-18T00:18:03.129 INFO:teuthology.orchestra.run.smithi097.stdout:min_compat_client jewel 2024-09-18T00:18:03.129 INFO:teuthology.orchestra.run.smithi097.stdout:require_osd_release squid 2024-09-18T00:18:03.129 INFO:teuthology.orchestra.run.smithi097.stdout:stretch_mode_enabled false 2024-09-18T00:18:03.129 INFO:teuthology.orchestra.run.smithi097.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-18T00:18:03.129 INFO:teuthology.orchestra.run.smithi097.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 22 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-18T00:18:03.129 INFO:teuthology.orchestra.run.smithi097.stdout: removed_snaps_queue [2~1] 2024-09-18T00:18:03.129 INFO:teuthology.orchestra.run.smithi097.stdout:max_osd 8 2024-09-18T00:18:03.129 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6800/40468291,v1:172.21.15.97:6801/40468291] [v2:172.21.15.97:6802/40468291,v1:172.21.15.97:6803/40468291] exists,up bfda7950-3d83-435d-8313-2f14345096ec 2024-09-18T00:18:03.129 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6824/2443452537,v1:172.21.15.97:6825/2443452537] [v2:172.21.15.97:6826/2443452537,v1:172.21.15.97:6827/2443452537] exists,up 35949061-aa30-4aff-b39d-90aebeffb115 2024-09-18T00:18:03.129 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6816/2425060925,v1:172.21.15.97:6817/2425060925] [v2:172.21.15.97:6818/2425060925,v1:172.21.15.97:6819/2425060925] exists,up 31b3e1c1-7648-44fe-9931-758ffe9a53ca 2024-09-18T00:18:03.129 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6808/3512666164,v1:172.21.15.97:6809/3512666164] [v2:172.21.15.97:6810/3512666164,v1:172.21.15.97:6811/3512666164] exists,up d7daebbc-0e33-42ab-97ee-a6c1b449ae8d 2024-09-18T00:18:03.129 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6802/3191744356,v1:172.21.15.114:6804/3191744356] [v2:172.21.15.114:6806/3191744356,v1:172.21.15.114:6808/3191744356] exists,up d55ca69b-5000-45ff-acc4-00f7c4763cad 2024-09-18T00:18:03.129 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6800/2340331281,v1:172.21.15.114:6801/2340331281] [v2:172.21.15.114:6803/2340331281,v1:172.21.15.114:6805/2340331281] exists,up 196a329b-bd93-4cc2-85bb-64ed81d3918b 2024-09-18T00:18:03.130 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6824/1148348470,v1:172.21.15.114:6825/1148348470] [v2:172.21.15.114:6826/1148348470,v1:172.21.15.114:6827/1148348470] exists,up dc9712f0-8858-467a-8397-d5eb97187165 2024-09-18T00:18:03.130 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6816/2133497352,v1:172.21.15.114:6817/2133497352] [v2:172.21.15.114:6818/2133497352,v1:172.21.15.114:6819/2133497352] exists,up b6d774ee-fe6d-462c-afa4-7ffd5d7343a9 2024-09-18T00:18:03.144 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6800/40468291,v1:172.21.15.97:6801/40468291] [v2:172.21.15.97:6802/40468291,v1:172.21.15.97:6803/40468291] exists,up bfda7950-3d83-435d-8313-2f14345096ec', 'osd.1 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6824/2443452537,v1:172.21.15.97:6825/2443452537] [v2:172.21.15.97:6826/2443452537,v1:172.21.15.97:6827/2443452537] exists,up 35949061-aa30-4aff-b39d-90aebeffb115', 'osd.2 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6816/2425060925,v1:172.21.15.97:6817/2425060925] [v2:172.21.15.97:6818/2425060925,v1:172.21.15.97:6819/2425060925] exists,up 31b3e1c1-7648-44fe-9931-758ffe9a53ca', 'osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6808/3512666164,v1:172.21.15.97:6809/3512666164] [v2:172.21.15.97:6810/3512666164,v1:172.21.15.97:6811/3512666164] exists,up d7daebbc-0e33-42ab-97ee-a6c1b449ae8d', 'osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6802/3191744356,v1:172.21.15.114:6804/3191744356] [v2:172.21.15.114:6806/3191744356,v1:172.21.15.114:6808/3191744356] exists,up d55ca69b-5000-45ff-acc4-00f7c4763cad', 'osd.5 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6800/2340331281,v1:172.21.15.114:6801/2340331281] [v2:172.21.15.114:6803/2340331281,v1:172.21.15.114:6805/2340331281] exists,up 196a329b-bd93-4cc2-85bb-64ed81d3918b', 'osd.6 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6824/1148348470,v1:172.21.15.114:6825/1148348470] [v2:172.21.15.114:6826/1148348470,v1:172.21.15.114:6827/1148348470] exists,up dc9712f0-8858-467a-8397-d5eb97187165', 'osd.7 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6816/2133497352,v1:172.21.15.114:6817/2133497352] [v2:172.21.15.114:6818/2133497352,v1:172.21.15.114:6819/2133497352] exists,up b6d774ee-fe6d-462c-afa4-7ffd5d7343a9'] 2024-09-18T00:18:03.145 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-18T00:18:03.259 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mon.a", 2024-09-18T00:18:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster": "ceph", 2024-09-18T00:18:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-18T00:18:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "admin_socket_mode": "", 2024-09-18T00:18:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-18T00:18:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_client_required": "cephx, none", 2024-09-18T00:18:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_cluster_required": "cephx", 2024-09-18T00:18:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_debug": "true", 2024-09-18T00:18:03.259 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-18T00:18:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-18T00:18:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_service_required": "cephx", 2024-09-18T00:18:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-18T00:18:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_supported": "cephx", 2024-09-18T00:18:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_aio": "true", 2024-09-18T00:18:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-18T00:18:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_aio_poll_ms": "250", 2024-09-18T00:18:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_aio_reap_max": "16", 2024-09-18T00:18:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_async_discard": "false", 2024-09-18T00:18:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_block_size": "4096", 2024-09-18T00:18:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_debug_aio": "false", 2024-09-18T00:18:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-18T00:18:03.260 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-18T00:18:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_debug_inflight_ios": "false", 2024-09-18T00:18:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_enable_discard": "false", 2024-09-18T00:18:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_flock_retry": "3", 2024-09-18T00:18:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-18T00:18:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_inject_crash": "0", 2024-09-18T00:18:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-18T00:18:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_ioring": "false", 2024-09-18T00:18:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_ioring_hipri": "false", 2024-09-18T00:18:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-18T00:18:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-18T00:18:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-18T00:18:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-18T00:18:03.261 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-18T00:18:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-18T00:18:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "bdev_type": "", 2024-09-18T00:18:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_alloc_size": "1048576", 2024-09-18T00:18:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_allocator": "hybrid", 2024-09-18T00:18:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_buffered_io": "true", 2024-09-18T00:18:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_check_for_zeros": "false", 2024-09-18T00:18:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-18T00:18:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-18T00:18:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_compact_log_sync": "false", 2024-09-18T00:18:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-18T00:18:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-18T00:18:03.262 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-18T00:18:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-18T00:18:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_max_log_runway": "4194304", 2024-09-18T00:18:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_max_prefetch": "1048576", 2024-09-18T00:18:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_min_flush_size": "524288", 2024-09-18T00:18:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_min_log_runway": "1048576", 2024-09-18T00:18:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_replay_recovery": "false", 2024-09-18T00:18:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-18T00:18:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-18T00:18:03.263 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_sync_write": "false", 2024-09-18T00:18:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-18T00:18:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-18T00:18:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-18T00:18:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_allocation_from_file": "true", 2024-09-18T00:18:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_allocator": "hybrid", 2024-09-18T00:18:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-18T00:18:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-18T00:18:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-18T00:18:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-18T00:18:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_bdev_label_multi": "true", 2024-09-18T00:18:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-18T00:18:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-18T00:18:03.264 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-18T00:18:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-18T00:18:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-18T00:18:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_block_create": "true", 2024-09-18T00:18:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_block_db_create": "false", 2024-09-18T00:18:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_block_db_path": "", 2024-09-18T00:18:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_block_db_size": "0", 2024-09-18T00:18:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_block_path": "", 2024-09-18T00:18:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_block_preallocate_file": "false", 2024-09-18T00:18:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_block_size": "107374182400", 2024-09-18T00:18:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_block_wal_create": "false", 2024-09-18T00:18:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_block_wal_path": "", 2024-09-18T00:18:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_block_wal_size": "100663296", 2024-09-18T00:18:03.265 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_bluefs": "true", 2024-09-18T00:18:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-18T00:18:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-18T00:18:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-18T00:18:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-18T00:18:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T00:18:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T00:18:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-18T00:18:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T00:18:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_autotune": "true", 2024-09-18T00:18:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-18T00:18:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-18T00:18:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-18T00:18:03.266 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-18T00:18:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_size": "0", 2024-09-18T00:18:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-18T00:18:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-18T00:18:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-18T00:18:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-18T00:18:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_type": "2q", 2024-09-18T00:18:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-18T00:18:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_clone_cow": "true", 2024-09-18T00:18:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-18T00:18:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-18T00:18:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-18T00:18:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-18T00:18:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-18T00:18:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-18T00:18:03.267 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-18T00:18:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_compression_mode": "none", 2024-09-18T00:18:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-18T00:18:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_csum_type": "crc32c", 2024-09-18T00:18:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-18T00:18:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_debug_freelist": "false", 2024-09-18T00:18:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-18T00:18:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-18T00:18:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-18T00:18:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-18T00:18:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-18T00:18:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-18T00:18:03.268 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-18T00:18:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-18T00:18:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-18T00:18:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-18T00:18:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-18T00:18:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-18T00:18:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_debug_small_allocations": "0", 2024-09-18T00:18:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-18T00:18:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_default_buffered_read": "true", 2024-09-18T00:18:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_default_buffered_write": "false", 2024-09-18T00:18:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-18T00:18:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-18T00:18:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-18T00:18:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-18T00:18:03.269 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-18T00:18:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-18T00:18:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-18T00:18:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-18T00:18:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-18T00:18:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fail_eio": "false", 2024-09-18T00:18:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-18T00:18:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-18T00:18:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-18T00:18:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-18T00:18:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-18T00:18:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-18T00:18:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_on_mount": "false", 2024-09-18T00:18:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-18T00:18:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_on_umount": "false", 2024-09-18T00:18:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-18T00:18:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-18T00:18:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-18T00:18:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-18T00:18:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-18T00:18:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-18T00:18:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-18T00:18:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-18T00:18:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_ignore_data_csum": "false", 2024-09-18T00:18:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-18T00:18:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-18T00:18:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-18T00:18:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-18T00:18:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_log_op_age": "5.000000", 2024-09-18T00:18:03.271 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_max_alloc_size": "0", 2024-09-18T00:18:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_max_blob_size": "0", 2024-09-18T00:18:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-18T00:18:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-18T00:18:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-18T00:18:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_max_deferred_txc": "32", 2024-09-18T00:18:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_min_alloc_size": "0", 2024-09-18T00:18:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-18T00:18:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-18T00:18:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_nid_prealloc": "1024", 2024-09-18T00:18:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-18T00:18:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-18T00:18:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-18T00:18:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-18T00:18:03.272 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_retry_disk_reads": "3", 2024-09-18T00:18:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_rocksdb_cf": "true", 2024-09-18T00:18:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-18T00:18:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-18T00:18:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-18T00:18:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-18T00:18:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-18T00:18:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-18T00:18:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-18T00:18:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-18T00:18:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_spdk_mem": "512", 2024-09-18T00:18:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-18T00:18:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-18T00:18:03.273 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-18T00:18:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-18T00:18:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-18T00:18:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-18T00:18:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-18T00:18:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_tracing": "false", 2024-09-18T00:18:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-18T00:18:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-18T00:18:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-18T00:18:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-18T00:18:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-18T00:18:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-18T00:18:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-18T00:18:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-18T00:18:03.274 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-18T00:18:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_zero_block_detection": "false", 2024-09-18T00:18:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-18T00:18:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-18T00:18:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-18T00:18:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-18T00:18:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-18T00:18:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-18T00:18:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-18T00:18:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-18T00:18:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-18T00:18:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-18T00:18:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-18T00:18:03.275 INFO:teuthology.orchestra.run.smithi097.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-18T00:18:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-18T00:18:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-18T00:18:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "cephx_cluster_require_signatures": "false", 2024-09-18T00:18:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "cephx_cluster_require_version": "2", 2024-09-18T00:18:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "cephx_require_signatures": "false", 2024-09-18T00:18:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "cephx_require_version": "2", 2024-09-18T00:18:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "cephx_service_require_signatures": "false", 2024-09-18T00:18:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "cephx_service_require_version": "2", 2024-09-18T00:18:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "cephx_sign_messages": "true", 2024-09-18T00:18:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "chdir": "", 2024-09-18T00:18:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "client_acl_type": "", 2024-09-18T00:18:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "client_asio_thread_count": "2", 2024-09-18T00:18:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "client_cache_mid": "0.750000", 2024-09-18T00:18:03.276 INFO:teuthology.orchestra.run.smithi097.stdout: "client_cache_size": "16384", 2024-09-18T00:18:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_caps_release_delay": "5", 2024-09-18T00:18:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_check_pool_perm": "true", 2024-09-18T00:18:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-18T00:18:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_debug_force_sync_read": "false", 2024-09-18T00:18:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_debug_getattr_caps": "false", 2024-09-18T00:18:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_debug_inject_features": "", 2024-09-18T00:18:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_debug_inject_tick_delay": "0", 2024-09-18T00:18:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-18T00:18:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_die_on_failed_remount": "false", 2024-09-18T00:18:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_dirsize_rbytes": "true", 2024-09-18T00:18:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_force_lazyio": "false", 2024-09-18T00:18:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_fs": "", 2024-09-18T00:18:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-18T00:18:03.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_inject_release_failure": "false", 2024-09-18T00:18:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_max_inline_size": "4096", 2024-09-18T00:18:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-18T00:18:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_mds_namespace": "", 2024-09-18T00:18:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_metadata": "", 2024-09-18T00:18:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_mount_gid": "-1", 2024-09-18T00:18:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_mount_timeout": "300", 2024-09-18T00:18:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_mount_uid": "-1", 2024-09-18T00:18:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_mountpoint": "/", 2024-09-18T00:18:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_notify_timeout": "10", 2024-09-18T00:18:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_oc": "true", 2024-09-18T00:18:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_oc_max_dirty": "104857600", 2024-09-18T00:18:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-18T00:18:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_oc_max_objects": "1000", 2024-09-18T00:18:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client_oc_size": "209715200", 2024-09-18T00:18:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client_oc_target_dirty": "8388608", 2024-09-18T00:18:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client_permissions": "true", 2024-09-18T00:18:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client_quota": "true", 2024-09-18T00:18:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client_quota_df": "true", 2024-09-18T00:18:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client_readahead_max_bytes": "0", 2024-09-18T00:18:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client_readahead_max_periods": "4", 2024-09-18T00:18:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client_readahead_min": "131072", 2024-09-18T00:18:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client_reconnect_stale": "false", 2024-09-18T00:18:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client_shutdown_timeout": "30", 2024-09-18T00:18:03.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client_snapdir": ".snap", 2024-09-18T00:18:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "client_tick_interval": "1", 2024-09-18T00:18:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "client_trace": "", 2024-09-18T00:18:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "client_try_dentry_invalidate": "false", 2024-09-18T00:18:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "client_use_faked_inos": "false", 2024-09-18T00:18:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "client_use_random_mds": "false", 2024-09-18T00:18:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "clog_to_graylog": "false", 2024-09-18T00:18:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-18T00:18:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "clog_to_graylog_port": "12201", 2024-09-18T00:18:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "clog_to_monitors": "default=true", 2024-09-18T00:18:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "clog_to_syslog": "false", 2024-09-18T00:18:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-18T00:18:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "clog_to_syslog_level": "info", 2024-09-18T00:18:03.280 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addr": "-", 2024-09-18T00:18:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_network": "", 2024-09-18T00:18:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_network_interface": "", 2024-09-18T00:18:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "compressor_zlib_isal": "false", 2024-09-18T00:18:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "compressor_zlib_level": "5", 2024-09-18T00:18:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "compressor_zlib_winsize": "-15", 2024-09-18T00:18:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "compressor_zstd_level": "1", 2024-09-18T00:18:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-18T00:18:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-18T00:18:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "crimson_alien_op_num_threads": "6", 2024-09-18T00:18:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-18T00:18:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-18T00:18:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-18T00:18:03.281 INFO:teuthology.orchestra.run.smithi097.stdout: "crimson_osd_stat_interval": "0", 2024-09-18T00:18:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "crimson_seastar_cpu_cores": "", 2024-09-18T00:18:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "crimson_seastar_num_threads": "0", 2024-09-18T00:18:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_location": "", 2024-09-18T00:18:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_location_hook": "", 2024-09-18T00:18:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_location_hook_timeout": "10", 2024-09-18T00:18:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "daemonize": "false", 2024-09-18T00:18:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "daos_pool": "tank", 2024-09-18T00:18:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-18T00:18:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-18T00:18:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-18T00:18:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_alienstore": "0/5", 2024-09-18T00:18:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_allow_any_pool_priority": "false", 2024-09-18T00:18:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_asok": "1/5", 2024-09-18T00:18:03.282 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_asok_assert_abort": "false", 2024-09-18T00:18:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_asserts_on_shutdown": "true", 2024-09-18T00:18:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_auth": "20/20", 2024-09-18T00:18:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_bdev": "1/3", 2024-09-18T00:18:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_bluefs": "1/5", 2024-09-18T00:18:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_bluestore": "1/5", 2024-09-18T00:18:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_buffer": "0/1", 2024-09-18T00:18:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_ceph_exporter": "1/5", 2024-09-18T00:18:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_cephfs_mirror": "0/5", 2024-09-18T00:18:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_cephsqlite": "0/5", 2024-09-18T00:18:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_client": "0/5", 2024-09-18T00:18:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_compressor": "1/5", 2024-09-18T00:18:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_context": "0/1", 2024-09-18T00:18:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_crimson_interrupt": "0/5", 2024-09-18T00:18:03.283 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_crush": "1/1", 2024-09-18T00:18:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_crypto": "1/5", 2024-09-18T00:18:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_cyanstore": "0/5", 2024-09-18T00:18:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_deliberately_leak_memory": "false", 2024-09-18T00:18:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_disable_randomized_ping": "false", 2024-09-18T00:18:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_dpdk": "1/5", 2024-09-18T00:18:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_eventtrace": "1/5", 2024-09-18T00:18:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_filer": "0/1", 2024-09-18T00:18:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_filestore": "1/3", 2024-09-18T00:18:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_finisher": "1/1", 2024-09-18T00:18:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_fuse": "1/5", 2024-09-18T00:18:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_heartbeat_testing_span": "0", 2024-09-18T00:18:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_heartbeatmap": "1/5", 2024-09-18T00:18:03.284 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-18T00:18:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_javaclient": "1/5", 2024-09-18T00:18:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_journal": "1/3", 2024-09-18T00:18:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_journaler": "0/5", 2024-09-18T00:18:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_kstore": "1/5", 2024-09-18T00:18:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_lockdep": "0/1", 2024-09-18T00:18:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_mclock": "1/5", 2024-09-18T00:18:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_mds": "1/5", 2024-09-18T00:18:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_mds_balancer": "1/5", 2024-09-18T00:18:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_mds_locker": "1/5", 2024-09-18T00:18:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_mds_log": "1/5", 2024-09-18T00:18:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_mds_log_expire": "1/5", 2024-09-18T00:18:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_mds_migrator": "1/5", 2024-09-18T00:18:03.285 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_mds_quiesce": "3/5", 2024-09-18T00:18:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_memstore": "1/5", 2024-09-18T00:18:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_mgr": "2/5", 2024-09-18T00:18:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_mgrc": "1/5", 2024-09-18T00:18:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_mon": "20/20", 2024-09-18T00:18:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_monc": "0/10", 2024-09-18T00:18:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_ms": "1/1", 2024-09-18T00:18:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_none": "0/5", 2024-09-18T00:18:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_objclass": "0/5", 2024-09-18T00:18:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_objectcacher": "0/5", 2024-09-18T00:18:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_objecter": "0/1", 2024-09-18T00:18:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_optracker": "0/5", 2024-09-18T00:18:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_osd": "1/5", 2024-09-18T00:18:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_paxos": "20/20", 2024-09-18T00:18:03.286 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_perfcounter": "1/5", 2024-09-18T00:18:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_prioritycache": "1/5", 2024-09-18T00:18:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_rados": "0/5", 2024-09-18T00:18:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_rbd": "0/5", 2024-09-18T00:18:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_rbd_mirror": "0/5", 2024-09-18T00:18:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_rbd_pwl": "0/5", 2024-09-18T00:18:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_rbd_replay": "0/5", 2024-09-18T00:18:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_refs": "0/0", 2024-09-18T00:18:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_reserver": "1/1", 2024-09-18T00:18:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_rgw": "1/5", 2024-09-18T00:18:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_rgw_access": "1/5", 2024-09-18T00:18:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_rgw_datacache": "1/5", 2024-09-18T00:18:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_rgw_dbstore": "1/5", 2024-09-18T00:18:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_rgw_flight": "1/5", 2024-09-18T00:18:03.287 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-18T00:18:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_rgw_sync": "1/5", 2024-09-18T00:18:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_rocksdb": "4/5", 2024-09-18T00:18:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_seastore": "0/5", 2024-09-18T00:18:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_seastore_backref": "0/5", 2024-09-18T00:18:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_seastore_cache": "0/5", 2024-09-18T00:18:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_seastore_cleaner": "0/5", 2024-09-18T00:18:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_seastore_device": "0/5", 2024-09-18T00:18:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_seastore_epm": "0/5", 2024-09-18T00:18:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-18T00:18:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_seastore_journal": "0/5", 2024-09-18T00:18:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_seastore_lba": "0/5", 2024-09-18T00:18:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_seastore_odata": "0/5", 2024-09-18T00:18:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_seastore_omap": "0/5", 2024-09-18T00:18:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_seastore_onode": "0/5", 2024-09-18T00:18:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_seastore_t": "0/5", 2024-09-18T00:18:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_seastore_tm": "0/5", 2024-09-18T00:18:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_striper": "0/1", 2024-09-18T00:18:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_test": "0/5", 2024-09-18T00:18:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_throttle": "1/1", 2024-09-18T00:18:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_timer": "0/1", 2024-09-18T00:18:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_tp": "0/5", 2024-09-18T00:18:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "debug_trace": "1/5", 2024-09-18T00:18:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-18T00:18:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "device_failure_prediction_mode": "none", 2024-09-18T00:18:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-18T00:18:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-18T00:18:03.289 INFO:teuthology.orchestra.run.smithi097.stdout: "err_to_graylog": "false", 2024-09-18T00:18:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "err_to_journald": "false", 2024-09-18T00:18:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "err_to_stderr": "true", 2024-09-18T00:18:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "err_to_syslog": "false", 2024-09-18T00:18:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event_tracing": "false", 2024-09-18T00:18:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "exporter_addr": "0.0.0.0", 2024-09-18T00:18:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "exporter_http_port": "9926", 2024-09-18T00:18:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "exporter_prio_limit": "5", 2024-09-18T00:18:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-18T00:18:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "exporter_sort_metrics": "true", 2024-09-18T00:18:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "exporter_stats_period": "5", 2024-09-18T00:18:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "fake_statfs_for_testing": "0", 2024-09-18T00:18:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "fatal_signal_handlers": "true", 2024-09-18T00:18:03.290 INFO:teuthology.orchestra.run.smithi097.stdout: "filer_max_purge_ops": "10", 2024-09-18T00:18:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: "filer_max_truncate_ops": "128", 2024-09-18T00:18:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_apply_finisher_threads": "1", 2024-09-18T00:18:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_blackhole": "false", 2024-09-18T00:18:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_btrfs_clone_range": "true", 2024-09-18T00:18:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_btrfs_snap": "true", 2024-09-18T00:18:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_caller_concurrency": "10", 2024-09-18T00:18:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_collect_device_partition_information": "true", 2024-09-18T00:18:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_commit_timeout": "600.000000", 2024-09-18T00:18:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_debug_inject_read_err": "false", 2024-09-18T00:18:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_debug_omap_check": "false", 2024-09-18T00:18:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-18T00:18:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_debug_verify_split": "false", 2024-09-18T00:18:03.291 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_dump_file": "", 2024-09-18T00:18:03.292 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-18T00:18:03.292 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-18T00:18:03.292 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_fadvise": "true", 2024-09-18T00:18:03.292 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_fail_eio": "true", 2024-09-18T00:18:03.292 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_fd_cache_shards": "16", 2024-09-18T00:18:03.292 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_fd_cache_size": "128", 2024-09-18T00:18:03.292 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_fiemap": "false", 2024-09-18T00:18:03.292 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_fiemap_threshold": "4096", 2024-09-18T00:18:03.292 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-18T00:18:03.292 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-18T00:18:03.292 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_inject_stall": "0", 2024-09-18T00:18:03.292 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_journal_parallel": "false", 2024-09-18T00:18:03.292 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_journal_trailing": "false", 2024-09-18T00:18:03.292 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_journal_writeahead": "false", 2024-09-18T00:18:03.293 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_kill_at": "0", 2024-09-18T00:18:03.293 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-18T00:18:03.293 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-18T00:18:03.293 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-18T00:18:03.293 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-18T00:18:03.293 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-18T00:18:03.293 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_max_inline_xattrs": "0", 2024-09-18T00:18:03.293 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-18T00:18:03.293 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-18T00:18:03.293 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-18T00:18:03.293 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-18T00:18:03.293 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_max_xattr_value_size": "0", 2024-09-18T00:18:03.294 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-18T00:18:03.294 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-18T00:18:03.294 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-18T00:18:03.294 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_merge_threshold": "-10", 2024-09-18T00:18:03.294 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-18T00:18:03.294 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_odsync_write": "false", 2024-09-18T00:18:03.294 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_omap_backend": "rocksdb", 2024-09-18T00:18:03.294 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_omap_backend_path": "", 2024-09-18T00:18:03.294 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-18T00:18:03.294 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-18T00:18:03.294 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-18T00:18:03.294 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_op_thread_timeout": "60", 2024-09-18T00:18:03.294 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_op_threads": "2", 2024-09-18T00:18:03.294 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_punch_hole": "false", 2024-09-18T00:18:03.295 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-18T00:18:03.295 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-18T00:18:03.295 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-18T00:18:03.295 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-18T00:18:03.295 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-18T00:18:03.295 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-18T00:18:03.295 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-18T00:18:03.295 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-18T00:18:03.295 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-18T00:18:03.295 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_queue_max_ops": "50", 2024-09-18T00:18:03.295 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-18T00:18:03.295 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_seek_data_hole": "false", 2024-09-18T00:18:03.295 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_sloppy_crc": "false", 2024-09-18T00:18:03.295 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-18T00:18:03.296 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_splice": "false", 2024-09-18T00:18:03.296 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_split_multiple": "2", 2024-09-18T00:18:03.296 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_split_rand_factor": "20", 2024-09-18T00:18:03.296 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_update_to": "1000", 2024-09-18T00:18:03.296 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-18T00:18:03.296 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-18T00:18:03.296 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-18T00:18:03.296 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-18T00:18:03.296 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-18T00:18:03.296 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-18T00:18:03.296 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_wbthrottle_enable": "true", 2024-09-18T00:18:03.296 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-18T00:18:03.296 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-18T00:18:03.297 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-18T00:18:03.297 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-18T00:18:03.297 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-18T00:18:03.297 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-18T00:18:03.297 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_xfs_extsize": "false", 2024-09-18T00:18:03.297 INFO:teuthology.orchestra.run.smithi097.stdout: "filestore_zfs_snap": "false", 2024-09-18T00:18:03.297 INFO:teuthology.orchestra.run.smithi097.stdout: "fio_dir": "/tmp/fio", 2024-09-18T00:18:03.297 INFO:teuthology.orchestra.run.smithi097.stdout: "fsid": "b993f56a-c4ef-4602-bb41-f32ae406b45a", 2024-09-18T00:18:03.297 INFO:teuthology.orchestra.run.smithi097.stdout: "fuse_allow_other": "true", 2024-09-18T00:18:03.297 INFO:teuthology.orchestra.run.smithi097.stdout: "fuse_atomic_o_trunc": "true", 2024-09-18T00:18:03.297 INFO:teuthology.orchestra.run.smithi097.stdout: "fuse_big_writes": "true", 2024-09-18T00:18:03.297 INFO:teuthology.orchestra.run.smithi097.stdout: "fuse_debug": "false", 2024-09-18T00:18:03.297 INFO:teuthology.orchestra.run.smithi097.stdout: "fuse_default_permissions": "false", 2024-09-18T00:18:03.298 INFO:teuthology.orchestra.run.smithi097.stdout: "fuse_disable_pagecache": "false", 2024-09-18T00:18:03.298 INFO:teuthology.orchestra.run.smithi097.stdout: "fuse_max_write": "0", 2024-09-18T00:18:03.298 INFO:teuthology.orchestra.run.smithi097.stdout: "fuse_multithreaded": "true", 2024-09-18T00:18:03.298 INFO:teuthology.orchestra.run.smithi097.stdout: "fuse_require_active_mds": "true", 2024-09-18T00:18:03.298 INFO:teuthology.orchestra.run.smithi097.stdout: "fuse_set_user_groups": "true", 2024-09-18T00:18:03.298 INFO:teuthology.orchestra.run.smithi097.stdout: "fuse_splice_move": "true", 2024-09-18T00:18:03.298 INFO:teuthology.orchestra.run.smithi097.stdout: "fuse_splice_read": "true", 2024-09-18T00:18:03.298 INFO:teuthology.orchestra.run.smithi097.stdout: "fuse_splice_write": "true", 2024-09-18T00:18:03.298 INFO:teuthology.orchestra.run.smithi097.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-18T00:18:03.298 INFO:teuthology.orchestra.run.smithi097.stdout: "fuse_use_invalidate_cb": "true", 2024-09-18T00:18:03.298 INFO:teuthology.orchestra.run.smithi097.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-18T00:18:03.298 INFO:teuthology.orchestra.run.smithi097.stdout: "gss_target_name": "ceph", 2024-09-18T00:18:03.298 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_file": "", 2024-09-18T00:18:03.299 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_inject_failure": "0", 2024-09-18T00:18:03.299 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_interval": "5", 2024-09-18T00:18:03.299 INFO:teuthology.orchestra.run.smithi097.stdout: "host": "smithi097", 2024-09-18T00:18:03.299 INFO:teuthology.orchestra.run.smithi097.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-18T00:18:03.299 INFO:teuthology.orchestra.run.smithi097.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-18T00:18:03.299 INFO:teuthology.orchestra.run.smithi097.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-18T00:18:03.299 INFO:teuthology.orchestra.run.smithi097.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-18T00:18:03.299 INFO:teuthology.orchestra.run.smithi097.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-18T00:18:03.299 INFO:teuthology.orchestra.run.smithi097.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-18T00:18:03.299 INFO:teuthology.orchestra.run.smithi097.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-18T00:18:03.299 INFO:teuthology.orchestra.run.smithi097.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-18T00:18:03.299 INFO:teuthology.orchestra.run.smithi097.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-18T00:18:03.299 INFO:teuthology.orchestra.run.smithi097.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-18T00:18:03.299 INFO:teuthology.orchestra.run.smithi097.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-18T00:18:03.300 INFO:teuthology.orchestra.run.smithi097.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-18T00:18:03.300 INFO:teuthology.orchestra.run.smithi097.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-18T00:18:03.300 INFO:teuthology.orchestra.run.smithi097.stdout: "inject_early_sigterm": "false", 2024-09-18T00:18:03.300 INFO:teuthology.orchestra.run.smithi097.stdout: "jaeger_agent_port": "6799", 2024-09-18T00:18:03.300 INFO:teuthology.orchestra.run.smithi097.stdout: "jaeger_tracing_enable": "false", 2024-09-18T00:18:03.300 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_aio": "true", 2024-09-18T00:18:03.300 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_align_min_size": "65536", 2024-09-18T00:18:03.300 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_block_align": "true", 2024-09-18T00:18:03.300 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_block_size": "4096", 2024-09-18T00:18:03.300 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_dio": "true", 2024-09-18T00:18:03.300 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_discard": "false", 2024-09-18T00:18:03.300 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_force_aio": "false", 2024-09-18T00:18:03.300 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_ignore_corruption": "false", 2024-09-18T00:18:03.301 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_max_write_bytes": "10485760", 2024-09-18T00:18:03.301 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_max_write_entries": "100", 2024-09-18T00:18:03.301 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_replay_from": "0", 2024-09-18T00:18:03.301 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-18T00:18:03.301 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-18T00:18:03.301 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-18T00:18:03.301 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-18T00:18:03.301 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_write_header_frequency": "0", 2024-09-18T00:18:03.301 INFO:teuthology.orchestra.run.smithi097.stdout: "journal_zero_on_create": "false", 2024-09-18T00:18:03.301 INFO:teuthology.orchestra.run.smithi097.stdout: "journaler_prefetch_periods": "10", 2024-09-18T00:18:03.301 INFO:teuthology.orchestra.run.smithi097.stdout: "journaler_prezero_periods": "5", 2024-09-18T00:18:03.301 INFO:teuthology.orchestra.run.smithi097.stdout: "journaler_write_head_interval": "15", 2024-09-18T00:18:03.301 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "", 2024-09-18T00:18:03.301 INFO:teuthology.orchestra.run.smithi097.stdout: "keyfile": "", 2024-09-18T00:18:03.302 INFO:teuthology.orchestra.run.smithi097.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-18T00:18:03.302 INFO:teuthology.orchestra.run.smithi097.stdout: "kstore_backend": "rocksdb", 2024-09-18T00:18:03.302 INFO:teuthology.orchestra.run.smithi097.stdout: "kstore_default_stripe_size": "65536", 2024-09-18T00:18:03.302 INFO:teuthology.orchestra.run.smithi097.stdout: "kstore_fsck_on_mount": "false", 2024-09-18T00:18:03.302 INFO:teuthology.orchestra.run.smithi097.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-18T00:18:03.302 INFO:teuthology.orchestra.run.smithi097.stdout: "kstore_max_bytes": "67108864", 2024-09-18T00:18:03.302 INFO:teuthology.orchestra.run.smithi097.stdout: "kstore_max_ops": "512", 2024-09-18T00:18:03.302 INFO:teuthology.orchestra.run.smithi097.stdout: "kstore_nid_prealloc": "1024", 2024-09-18T00:18:03.302 INFO:teuthology.orchestra.run.smithi097.stdout: "kstore_onode_map_size": "1024", 2024-09-18T00:18:03.302 INFO:teuthology.orchestra.run.smithi097.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-18T00:18:03.302 INFO:teuthology.orchestra.run.smithi097.stdout: "kstore_sync_submit_transaction": "false", 2024-09-18T00:18:03.302 INFO:teuthology.orchestra.run.smithi097.stdout: "kstore_sync_transaction": "false", 2024-09-18T00:18:03.302 INFO:teuthology.orchestra.run.smithi097.stdout: "librados_thread_count": "2", 2024-09-18T00:18:03.303 INFO:teuthology.orchestra.run.smithi097.stdout: "lockdep": "false", 2024-09-18T00:18:03.303 INFO:teuthology.orchestra.run.smithi097.stdout: "lockdep_force_backtrace": "false", 2024-09-18T00:18:03.303 INFO:teuthology.orchestra.run.smithi097.stdout: "log_coarse_timestamps": "true", 2024-09-18T00:18:03.303 INFO:teuthology.orchestra.run.smithi097.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-18T00:18:03.303 INFO:teuthology.orchestra.run.smithi097.stdout: "log_flush_on_exit": "false", 2024-09-18T00:18:03.303 INFO:teuthology.orchestra.run.smithi097.stdout: "log_graylog_host": "127.0.0.1", 2024-09-18T00:18:03.303 INFO:teuthology.orchestra.run.smithi097.stdout: "log_graylog_port": "12201", 2024-09-18T00:18:03.303 INFO:teuthology.orchestra.run.smithi097.stdout: "log_max_new": "1000", 2024-09-18T00:18:03.303 INFO:teuthology.orchestra.run.smithi097.stdout: "log_max_recent": "10000", 2024-09-18T00:18:03.303 INFO:teuthology.orchestra.run.smithi097.stdout: "log_stderr_prefix": "", 2024-09-18T00:18:03.303 INFO:teuthology.orchestra.run.smithi097.stdout: "log_stop_at_utilization": "0.970000", 2024-09-18T00:18:03.303 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": "true", 2024-09-18T00:18:03.303 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_graylog": "false", 2024-09-18T00:18:03.303 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_journald": "false", 2024-09-18T00:18:03.304 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_stderr": "false", 2024-09-18T00:18:03.304 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_syslog": "false", 2024-09-18T00:18:03.304 INFO:teuthology.orchestra.run.smithi097.stdout: "max_rotating_auth_attempts": "10", 2024-09-18T00:18:03.304 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-18T00:18:03.304 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_action_on_write_error": "1", 2024-09-18T00:18:03.304 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_alternate_name_max": "8192", 2024-09-18T00:18:03.304 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_asio_thread_count": "2", 2024-09-18T00:18:03.304 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_export_pin": "true", 2024-09-18T00:18:03.304 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_fragment_dirs": "true", 2024-09-18T00:18:03.304 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-18T00:18:03.304 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_fragment_interval": "5", 2024-09-18T00:18:03.304 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-18T00:18:03.304 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-18T00:18:03.305 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_interval": "10", 2024-09-18T00:18:03.305 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_max": "-1", 2024-09-18T00:18:03.305 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_max_until": "-1", 2024-09-18T00:18:03.305 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_merge_size": "50", 2024-09-18T00:18:03.305 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_midchunk": "0.300000", 2024-09-18T00:18:03.305 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-18T00:18:03.305 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_min_start": "0.200000", 2024-09-18T00:18:03.305 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_minchunk": "0.001000", 2024-09-18T00:18:03.305 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_mode": "0", 2024-09-18T00:18:03.305 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_need_max": "1.200000", 2024-09-18T00:18:03.305 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_need_min": "0.800000", 2024-09-18T00:18:03.305 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_overload_epochs": "2", 2024-09-18T00:18:03.305 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-18T00:18:03.306 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-18T00:18:03.306 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_split_bits": "3", 2024-09-18T00:18:03.306 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-18T00:18:03.306 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_split_size": "10000", 2024-09-18T00:18:03.306 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-18T00:18:03.306 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_target_decay": "10.000000", 2024-09-18T00:18:03.306 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-18T00:18:03.306 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_beacon_grace": "15.000000", 2024-09-18T00:18:03.306 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_beacon_interval": "4.000000", 2024-09-18T00:18:03.306 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-18T00:18:03.306 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-18T00:18:03.306 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache_mid": "0.700000", 2024-09-18T00:18:03.306 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-18T00:18:03.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache_quiesce_delay": "0", 2024-09-18T00:18:03.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-18T00:18:03.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-18T00:18:03.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-18T00:18:03.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache_release_free_interval": "10", 2024-09-18T00:18:03.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache_reservation": "0.050000", 2024-09-18T00:18:03.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-18T00:18:03.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache_trim_interval": "1", 2024-09-18T00:18:03.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache_trim_threshold": "262144", 2024-09-18T00:18:03.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-18T00:18:03.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-18T00:18:03.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-18T00:18:03.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_client_prealloc_inos": "1000", 2024-09-18T00:18:03.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-18T00:18:03.308 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_connect_bootstrapping": "false", 2024-09-18T00:18:03.308 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_damage_table_max_entries": "10000", 2024-09-18T00:18:03.308 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-18T00:18:03.308 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_debug_auth_pins": "false", 2024-09-18T00:18:03.308 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_debug_frag": "false", 2024-09-18T00:18:03.308 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_debug_scatterstat": "false", 2024-09-18T00:18:03.308 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_debug_subtrees": "false", 2024-09-18T00:18:03.308 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_decay_halflife": "5.000000", 2024-09-18T00:18:03.308 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_default_dir_hash": "2", 2024-09-18T00:18:03.308 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_defer_session_stale": "true", 2024-09-18T00:18:03.308 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_deny_all_reconnect": "false", 2024-09-18T00:18:03.308 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_dir_keys_per_op": "16384", 2024-09-18T00:18:03.308 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_dir_max_commit_size": "10", 2024-09-18T00:18:03.309 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_dir_max_entries": "0", 2024-09-18T00:18:03.309 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_dir_prefetch": "true", 2024-09-18T00:18:03.309 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-18T00:18:03.309 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-18T00:18:03.309 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_dump_cache_on_map": "false", 2024-09-18T00:18:03.309 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-18T00:18:03.309 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-18T00:18:03.309 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_early_reply": "true", 2024-09-18T00:18:03.309 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_enable_op_tracker": "true", 2024-09-18T00:18:03.309 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_enforce_unique_name": "true", 2024-09-18T00:18:03.309 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-18T00:18:03.309 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-18T00:18:03.309 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_export_ephemeral_random": "true", 2024-09-18T00:18:03.310 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-18T00:18:03.310 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-18T00:18:03.310 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-18T00:18:03.310 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-18T00:18:03.310 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-18T00:18:03.310 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-18T00:18:03.310 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-18T00:18:03.310 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-18T00:18:03.310 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_health_summarize_threshold": "10", 2024-09-18T00:18:03.310 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-18T00:18:03.310 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-18T00:18:03.310 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_inject_health_dummy": "false", 2024-09-18T00:18:03.310 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-18T00:18:03.310 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_inject_migrator_session_race": "false", 2024-09-18T00:18:03.311 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-18T00:18:03.311 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-18T00:18:03.311 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-18T00:18:03.311 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_join_fs": "", 2024-09-18T00:18:03.311 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_journal_format": "1", 2024-09-18T00:18:03.311 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-18T00:18:03.311 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_kill_create_at": "0", 2024-09-18T00:18:03.311 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_kill_export_at": "0", 2024-09-18T00:18:03.311 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_kill_import_at": "0", 2024-09-18T00:18:03.311 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_kill_journal_at": "0", 2024-09-18T00:18:03.311 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_kill_journal_expire_at": "0", 2024-09-18T00:18:03.311 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_kill_journal_replay_at": "0", 2024-09-18T00:18:03.311 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_kill_link_at": "0", 2024-09-18T00:18:03.312 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_kill_mdstable_at": "0", 2024-09-18T00:18:03.312 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_kill_openc_at": "0", 2024-09-18T00:18:03.312 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_kill_rename_at": "0", 2024-09-18T00:18:03.312 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_kill_shutdown_at": "0", 2024-09-18T00:18:03.312 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log_event_large_threshold": "524288", 2024-09-18T00:18:03.312 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log_events_per_segment": "1024", 2024-09-18T00:18:03.312 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-18T00:18:03.312 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log_max_events": "-1", 2024-09-18T00:18:03.312 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log_max_segments": "128", 2024-09-18T00:18:03.312 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log_pause": "false", 2024-09-18T00:18:03.312 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log_segment_size": "0", 2024-09-18T00:18:03.312 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-18T00:18:03.312 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-18T00:18:03.313 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-18T00:18:03.313 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log_trim_threshold": "128", 2024-09-18T00:18:03.313 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-18T00:18:03.313 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log_warn_factor": "2.000000", 2024-09-18T00:18:03.313 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_max_caps_per_client": "1048576", 2024-09-18T00:18:03.313 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_max_completed_flushes": "100000", 2024-09-18T00:18:03.313 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_max_completed_requests": "100000", 2024-09-18T00:18:03.313 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_max_export_size": "20971520", 2024-09-18T00:18:03.313 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_max_file_recover": "32", 2024-09-18T00:18:03.313 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_max_purge_files": "64", 2024-09-18T00:18:03.313 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_max_purge_ops": "8192", 2024-09-18T00:18:03.313 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-18T00:18:03.313 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-18T00:18:03.314 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_max_snaps_per_dir": "100", 2024-09-18T00:18:03.314 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_metrics_update_interval": "2", 2024-09-18T00:18:03.314 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_min_caps_per_client": "100", 2024-09-18T00:18:03.314 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_min_caps_working_set": "10000", 2024-09-18T00:18:03.314 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-18T00:18:03.314 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_numa_node": "-1", 2024-09-18T00:18:03.314 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-18T00:18:03.314 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_op_complaint_time": "30.000000", 2024-09-18T00:18:03.314 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_op_history_duration": "600", 2024-09-18T00:18:03.314 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_op_history_size": "20", 2024-09-18T00:18:03.314 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_op_history_slow_op_size": "20", 2024-09-18T00:18:03.314 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-18T00:18:03.314 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_op_log_threshold": "5", 2024-09-18T00:18:03.314 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_ping_grace": "15", 2024-09-18T00:18:03.315 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_ping_interval": "5", 2024-09-18T00:18:03.315 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-18T00:18:03.315 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-18T00:18:03.315 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_recall_max_caps": "30000", 2024-09-18T00:18:03.315 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-18T00:18:03.315 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-18T00:18:03.315 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-18T00:18:03.315 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_recall_warning_threshold": "262144", 2024-09-18T00:18:03.315 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-18T00:18:03.315 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_replay_interval": "1.000000", 2024-09-18T00:18:03.315 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-18T00:18:03.315 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-18T00:18:03.315 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_root_ino_gid": "0", 2024-09-18T00:18:03.315 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_root_ino_uid": "0", 2024-09-18T00:18:03.316 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-18T00:18:03.316 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-18T00:18:03.316 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-18T00:18:03.316 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-18T00:18:03.316 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-18T00:18:03.316 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-18T00:18:03.316 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-18T00:18:03.316 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-18T00:18:03.316 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-18T00:18:03.316 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-18T00:18:03.316 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_shutdown_check": "0", 2024-09-18T00:18:03.316 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_skip_ino": "0", 2024-09-18T00:18:03.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-18T00:18:03.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_snap_max_uid": "4294967294", 2024-09-18T00:18:03.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_snap_min_uid": "0", 2024-09-18T00:18:03.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_snap_rstat": "false", 2024-09-18T00:18:03.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_standby_replay_damaged": "false", 2024-09-18T00:18:03.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_symlink_recovery": "true", 2024-09-18T00:18:03.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-18T00:18:03.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_thrash_exports": "0", 2024-09-18T00:18:03.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_thrash_fragments": "0", 2024-09-18T00:18:03.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_tick_interval": "5.000000", 2024-09-18T00:18:03.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_valgrind_exit": "false", 2024-09-18T00:18:03.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_verify_backtrace": "1", 2024-09-18T00:18:03.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_verify_scatter": "false", 2024-09-18T00:18:03.317 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-18T00:18:03.318 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_wipe_sessions": "false", 2024-09-18T00:18:03.318 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool_debug": "false", 2024-09-18T00:18:03.318 INFO:teuthology.orchestra.run.smithi097.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-18T00:18:03.318 INFO:teuthology.orchestra.run.smithi097.stdout: "memstore_device_bytes": "1073741824", 2024-09-18T00:18:03.318 INFO:teuthology.orchestra.run.smithi097.stdout: "memstore_page_set": "false", 2024-09-18T00:18:03.318 INFO:teuthology.orchestra.run.smithi097.stdout: "memstore_page_size": "65536", 2024-09-18T00:18:03.318 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_client_bytes": "134217728", 2024-09-18T00:18:03.318 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_client_messages": "512", 2024-09-18T00:18:03.318 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-18T00:18:03.318 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-18T00:18:03.318 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-18T00:18:03.318 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-18T00:18:03.318 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_disabled_modules": "", 2024-09-18T00:18:03.318 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_enable_op_tracker": "true", 2024-09-18T00:18:03.319 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-18T00:18:03.319 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_max_pg_num_change": "128", 2024-09-18T00:18:03.319 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_mds_bytes": "134217728", 2024-09-18T00:18:03.319 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_mds_messages": "128", 2024-09-18T00:18:03.319 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-18T00:18:03.319 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_mon_bytes": "134217728", 2024-09-18T00:18:03.319 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_mon_messages": "128", 2024-09-18T00:18:03.319 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-18T00:18:03.319 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-18T00:18:03.319 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_op_history_duration": "600", 2024-09-18T00:18:03.319 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_op_history_size": "20", 2024-09-18T00:18:03.319 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-18T00:18:03.319 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-18T00:18:03.319 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_op_log_threshold": "5", 2024-09-18T00:18:03.320 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_osd_bytes": "536870912", 2024-09-18T00:18:03.320 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_osd_messages": "8192", 2024-09-18T00:18:03.320 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_pool": "true", 2024-09-18T00:18:03.320 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-18T00:18:03.320 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_standby_modules": "true", 2024-09-18T00:18:03.320 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_stats_period": "5", 2024-09-18T00:18:03.320 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_stats_threshold": "5", 2024-09-18T00:18:03.320 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_tick_period": "2", 2024-09-18T00:18:03.320 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-18T00:18:03.320 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-18T00:18:03.320 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_allow_pool_delete": "true", 2024-09-18T00:18:03.320 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_allow_pool_size_one": "true", 2024-09-18T00:18:03.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_auth_validate_all_caps": "true", 2024-09-18T00:18:03.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-18T00:18:03.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-18T00:18:03.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_client_bytes": "104857600", 2024-09-18T00:18:03.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_client_directed_command_retry": "5", 2024-09-18T00:18:03.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-18T00:18:03.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-18T00:18:03.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-18T00:18:03.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-18T00:18:03.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_client_hunt_parallel": "3", 2024-09-18T00:18:03.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_client_log_interval": "1.000000", 2024-09-18T00:18:03.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-18T00:18:03.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_client_ping_interval": "10.000000", 2024-09-18T00:18:03.321 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-18T00:18:03.322 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_client_target_rank": "-1", 2024-09-18T00:18:03.322 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-18T00:18:03.322 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-18T00:18:03.322 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-18T00:18:03.322 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_cluster_log_file_level": "debug", 2024-09-18T00:18:03.322 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_cluster_log_to_file": "true", 2024-09-18T00:18:03.322 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-18T00:18:03.322 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-18T00:18:03.322 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-18T00:18:03.322 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_cluster_log_to_journald": "false", 2024-09-18T00:18:03.322 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-18T00:18:03.322 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-18T00:18:03.322 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-18T00:18:03.322 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-18T00:18:03.323 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_compact_on_bootstrap": "false", 2024-09-18T00:18:03.323 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_compact_on_start": "false", 2024-09-18T00:18:03.323 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_compact_on_trim": "true", 2024-09-18T00:18:03.323 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-18T00:18:03.323 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-18T00:18:03.323 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-18T00:18:03.323 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_cpu_threads": "4", 2024-09-18T00:18:03.323 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_crush_min_required_version": "hammer", 2024-09-18T00:18:03.323 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_daemon_bytes": "419430400", 2024-09-18T00:18:03.323 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-18T00:18:03.323 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_data_avail_crit": "5", 2024-09-18T00:18:03.323 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_data_avail_warn": "5", 2024-09-18T00:18:03.323 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_data_size_warn": "16106127360", 2024-09-18T00:18:03.324 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-18T00:18:03.324 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-18T00:18:03.324 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_debug_dump_json": "false", 2024-09-18T00:18:03.324 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-18T00:18:03.324 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_debug_dump_transactions": "false", 2024-09-18T00:18:03.324 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_debug_extra_checks": "false", 2024-09-18T00:18:03.324 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-18T00:18:03.324 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-18T00:18:03.324 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_debug_no_require_reef": "false", 2024-09-18T00:18:03.324 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_debug_no_require_squid": "false", 2024-09-18T00:18:03.324 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-18T00:18:03.324 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-18T00:18:03.324 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-18T00:18:03.324 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_down_mkfs_grace": "120", 2024-09-18T00:18:03.325 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_down_uptime_grace": "60", 2024-09-18T00:18:03.325 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_election_default_strategy": "1", 2024-09-18T00:18:03.325 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_election_timeout": "5.000000", 2024-09-18T00:18:03.325 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-18T00:18:03.325 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_elector_ping_divisor": "2", 2024-09-18T00:18:03.325 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-18T00:18:03.325 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_enable_op_tracker": "true", 2024-09-18T00:18:03.325 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_fake_pool_delete": "false", 2024-09-18T00:18:03.325 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_force_quorum_join": "false", 2024-09-18T00:18:03.325 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-18T00:18:03.325 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_globalid_prealloc": "10000", 2024-09-18T00:18:03.325 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_health_detail_to_clog": "false", 2024-09-18T00:18:03.325 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_health_log_update_period": "5", 2024-09-18T00:18:03.325 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_health_max_detail": "50", 2024-09-18T00:18:03.326 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_health_to_clog": "true", 2024-09-18T00:18:03.326 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_health_to_clog_interval": "600", 2024-09-18T00:18:03.326 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-18T00:18:03.326 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_host": "172.21.15.97,[v2:172.21.15.97:3301,v1:172.21.15.97:6790],172.21.15.114", 2024-09-18T00:18:03.326 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_host_override": "", 2024-09-18T00:18:03.326 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_initial_members": "", 2024-09-18T00:18:03.326 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-18T00:18:03.326 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-18T00:18:03.326 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-18T00:18:03.326 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-18T00:18:03.326 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-18T00:18:03.326 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_lease": "5.000000", 2024-09-18T00:18:03.326 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-18T00:18:03.326 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-18T00:18:03.327 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_log_full_interval": "50", 2024-09-18T00:18:03.327 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_log_max": "10000", 2024-09-18T00:18:03.327 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_log_max_summary": "50", 2024-09-18T00:18:03.327 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-18T00:18:03.327 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_max_log_epochs": "500", 2024-09-18T00:18:03.327 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-18T00:18:03.327 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-18T00:18:03.327 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_max_osd": "10000", 2024-09-18T00:18:03.327 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_max_pg_per_osd": "10000", 2024-09-18T00:18:03.327 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_max_pool_pg_num": "65536", 2024-09-18T00:18:03.327 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-18T00:18:03.327 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-18T00:18:03.327 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_mds_force_trim_to": "0", 2024-09-18T00:18:03.327 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_mds_skip_sanity": "false", 2024-09-18T00:18:03.328 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_memory_autotune": "true", 2024-09-18T00:18:03.328 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_memory_target": "2147483648", 2024-09-18T00:18:03.328 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_mgr_beacon_grace": "30", 2024-09-18T00:18:03.328 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-18T00:18:03.328 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_mgr_digest_period": "5", 2024-09-18T00:18:03.328 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_mgr_inactive_grace": "60", 2024-09-18T00:18:03.328 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-18T00:18:03.328 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-18T00:18:03.328 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_min_osdmap_epochs": "500", 2024-09-18T00:18:03.328 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_op_complaint_time": "30", 2024-09-18T00:18:03.328 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_op_history_duration": "600", 2024-09-18T00:18:03.328 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_op_history_size": "20", 2024-09-18T00:18:03.328 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_op_history_slow_op_size": "20", 2024-09-18T00:18:03.329 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-18T00:18:03.329 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_op_log_threshold": "5", 2024-09-18T00:18:03.329 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-18T00:18:03.329 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-18T00:18:03.329 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-18T00:18:03.329 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_auto_mark_in": "false", 2024-09-18T00:18:03.329 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-18T00:18:03.329 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-18T00:18:03.329 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-18T00:18:03.329 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_cache_size": "500", 2024-09-18T00:18:03.329 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-18T00:18:03.329 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-18T00:18:03.329 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-18T00:18:03.329 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_down_out_interval": "600", 2024-09-18T00:18:03.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-18T00:18:03.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-18T00:18:03.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_force_trim_to": "0", 2024-09-18T00:18:03.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-18T00:18:03.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-18T00:18:03.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-18T00:18:03.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-18T00:18:03.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-18T00:18:03.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-18T00:18:03.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-18T00:18:03.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-18T00:18:03.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_min_down_reporters": "2", 2024-09-18T00:18:03.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-18T00:18:03.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-18T00:18:03.331 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-18T00:18:03.331 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-18T00:18:03.331 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-18T00:18:03.331 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-18T00:18:03.331 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_report_timeout": "900", 2024-09-18T00:18:03.331 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-18T00:18:03.331 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-18T00:18:03.331 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-18T00:18:03.331 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-18T00:18:03.331 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-18T00:18:03.331 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-18T00:18:03.331 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-18T00:18:03.331 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-18T00:18:03.331 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-18T00:18:03.332 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_pg_stuck_threshold": "60", 2024-09-18T00:18:03.332 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-18T00:18:03.332 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-18T00:18:03.332 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-18T00:18:03.332 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-18T00:18:03.332 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-18T00:18:03.332 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-18T00:18:03.332 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_probe_timeout": "2.000000", 2024-09-18T00:18:03.332 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_reweight_max_change": "0.050000", 2024-09-18T00:18:03.332 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_reweight_max_osds": "4", 2024-09-18T00:18:03.332 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-18T00:18:03.332 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-18T00:18:03.332 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-18T00:18:03.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-18T00:18:03.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-18T00:18:03.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_scrub_interval": "86400", 2024-09-18T00:18:03.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_scrub_max_keys": "100", 2024-09-18T00:18:03.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_scrub_timeout": "300", 2024-09-18T00:18:03.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_session_timeout": "300", 2024-09-18T00:18:03.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_smart_report_timeout": "5", 2024-09-18T00:18:03.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_stat_smooth_intervals": "6", 2024-09-18T00:18:03.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-18T00:18:03.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_stretch_pool_min_size": "2", 2024-09-18T00:18:03.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_stretch_pool_size": "4", 2024-09-18T00:18:03.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-18T00:18:03.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-18T00:18:03.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_sync_debug": "false", 2024-09-18T00:18:03.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-18T00:18:03.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-18T00:18:03.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_sync_provider_kill_at": "0", 2024-09-18T00:18:03.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_sync_requester_kill_at": "0", 2024-09-18T00:18:03.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_sync_timeout": "60.000000", 2024-09-18T00:18:03.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_target_pg_per_osd": "100", 2024-09-18T00:18:03.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_tick_interval": "5", 2024-09-18T00:18:03.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_timecheck_interval": "300.000000", 2024-09-18T00:18:03.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-18T00:18:03.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_use_min_delay_socket": "false", 2024-09-18T00:18:03.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_older_version_delay": "604800", 2024-09-18T00:18:03.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-18T00:18:03.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-18T00:18:03.334 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-18T00:18:03.335 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-18T00:18:03.335 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-18T00:18:03.335 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-18T00:18:03.335 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-18T00:18:03.335 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_misplaced": "false", 2024-09-18T00:18:03.335 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-18T00:18:03.335 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_older_version": "true", 2024-09-18T00:18:03.335 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-18T00:18:03.335 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-18T00:18:03.335 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-18T00:18:03.335 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-18T00:18:03.335 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-18T00:18:03.335 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-18T00:18:03.335 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-18T00:18:03.336 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-18T00:18:03.336 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-18T00:18:03.336 INFO:teuthology.orchestra.run.smithi097.stdout: "monmap": "", 2024-09-18T00:18:03.336 INFO:teuthology.orchestra.run.smithi097.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-18T00:18:03.336 INFO:teuthology.orchestra.run.smithi097.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-18T00:18:03.336 INFO:teuthology.orchestra.run.smithi097.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-18T00:18:03.336 INFO:teuthology.orchestra.run.smithi097.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-18T00:18:03.336 INFO:teuthology.orchestra.run.smithi097.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-18T00:18:03.336 INFO:teuthology.orchestra.run.smithi097.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-18T00:18:03.336 INFO:teuthology.orchestra.run.smithi097.stdout: "motr_tracing_enabled": "false", 2024-09-18T00:18:03.336 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_op_threads": "3", 2024-09-18T00:18:03.336 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-18T00:18:03.336 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_rdma_cm": "false", 2024-09-18T00:18:03.336 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_rdma_device_name": "", 2024-09-18T00:18:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_rdma_dscp": "96", 2024-09-18T00:18:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-18T00:18:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-18T00:18:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_rdma_local_gid": "", 2024-09-18T00:18:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-18T00:18:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_rdma_port_num": "1", 2024-09-18T00:18:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-18T00:18:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-18T00:18:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-18T00:18:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-18T00:18:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_rdma_sl": "3", 2024-09-18T00:18:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_rdma_support_srq": "true", 2024-09-18T00:18:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_rdma_type": "ib", 2024-09-18T00:18:03.337 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_async_reap_threshold": "5", 2024-09-18T00:18:03.338 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_bind_before_connect": "false", 2024-09-18T00:18:03.338 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_bind_ipv4": "true", 2024-09-18T00:18:03.338 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_bind_ipv6": "false", 2024-09-18T00:18:03.338 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_bind_msgr1": "true", 2024-09-18T00:18:03.338 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_bind_msgr2": "true", 2024-09-18T00:18:03.338 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_bind_port_max": "7568", 2024-09-18T00:18:03.338 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_bind_port_min": "6800", 2024-09-18T00:18:03.338 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-18T00:18:03.338 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_bind_retry_count": "3", 2024-09-18T00:18:03.338 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_bind_retry_delay": "5", 2024-09-18T00:18:03.338 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_blackhole_client": "false", 2024-09-18T00:18:03.338 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_blackhole_mds": "false", 2024-09-18T00:18:03.338 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_blackhole_mgr": "false", 2024-09-18T00:18:03.339 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_blackhole_mon": "false", 2024-09-18T00:18:03.339 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_blackhole_osd": "false", 2024-09-18T00:18:03.339 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_client_mode": "crc secure", 2024-09-18T00:18:03.339 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_cluster_mode": "crc secure", 2024-09-18T00:18:03.339 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_cluster_type": "", 2024-09-18T00:18:03.339 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_compress_secure": "false", 2024-09-18T00:18:03.339 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_connection_idle_timeout": "900", 2024-09-18T00:18:03.339 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_connection_ready_timeout": "10", 2024-09-18T00:18:03.339 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_crc_data": "true", 2024-09-18T00:18:03.339 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_crc_header": "true", 2024-09-18T00:18:03.339 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_die_on_bad_msg": "false", 2024-09-18T00:18:03.339 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_die_on_bug": "true", 2024-09-18T00:18:03.339 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_die_on_old_message": "true", 2024-09-18T00:18:03.340 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_die_on_skipped_message": "false", 2024-09-18T00:18:03.340 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-18T00:18:03.340 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-18T00:18:03.340 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dpdk_coremask": "0xF", 2024-09-18T00:18:03.340 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-18T00:18:03.340 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-18T00:18:03.340 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dpdk_enable_tso": "true", 2024-09-18T00:18:03.340 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-18T00:18:03.340 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-18T00:18:03.340 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dpdk_hugepages": "", 2024-09-18T00:18:03.340 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-18T00:18:03.340 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-18T00:18:03.340 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dpdk_lro": "true", 2024-09-18T00:18:03.340 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dpdk_memory_channel": "4", 2024-09-18T00:18:03.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-18T00:18:03.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dpdk_pmd": "", 2024-09-18T00:18:03.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dpdk_port_id": "0", 2024-09-18T00:18:03.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-18T00:18:03.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-18T00:18:03.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_dump_on_send": "false", 2024-09-18T00:18:03.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_initial_backoff": "0.200000", 2024-09-18T00:18:03.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_inject_delay_max": "1.000000", 2024-09-18T00:18:03.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-18T00:18:03.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_inject_delay_type": "", 2024-09-18T00:18:03.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-18T00:18:03.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_inject_network_congestion": "0", 2024-09-18T00:18:03.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_inject_socket_failures": "5000", 2024-09-18T00:18:03.342 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_learn_addr_from_peer": "true", 2024-09-18T00:18:03.342 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_max_accept_failures": "4", 2024-09-18T00:18:03.342 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_max_backoff": "15.000000", 2024-09-18T00:18:03.342 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_mon_client_mode": "secure crc", 2024-09-18T00:18:03.342 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-18T00:18:03.342 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_mon_service_mode": "secure crc", 2024-09-18T00:18:03.342 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_osd_compress_min_size": "1024", 2024-09-18T00:18:03.342 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_osd_compress_mode": "none", 2024-09-18T00:18:03.342 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-18T00:18:03.342 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-18T00:18:03.342 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_pq_min_cost": "65536", 2024-09-18T00:18:03.342 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_public_type": "", 2024-09-18T00:18:03.342 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_service_mode": "crc secure", 2024-09-18T00:18:03.343 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_tcp_listen_backlog": "512", 2024-09-18T00:18:03.343 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_tcp_nodelay": "true", 2024-09-18T00:18:03.343 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-18T00:18:03.343 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_tcp_rcvbuf": "0", 2024-09-18T00:18:03.343 INFO:teuthology.orchestra.run.smithi097.stdout: "ms_type": "async+posix", 2024-09-18T00:18:03.343 INFO:teuthology.orchestra.run.smithi097.stdout: "no_config_file": "false", 2024-09-18T00:18:03.343 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter_completion_locks_per_session": "32", 2024-09-18T00:18:03.343 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-18T00:18:03.343 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-18T00:18:03.343 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter_inflight_ops": "1024", 2024-09-18T00:18:03.343 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-18T00:18:03.343 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-18T00:18:03.343 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter_tick_interval": "5.000000", 2024-09-18T00:18:03.343 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter_timeout": "10.000000", 2024-09-18T00:18:03.344 INFO:teuthology.orchestra.run.smithi097.stdout: "objectstore_blackhole": "false", 2024-09-18T00:18:03.344 INFO:teuthology.orchestra.run.smithi097.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-18T00:18:03.344 INFO:teuthology.orchestra.run.smithi097.stdout: "openssl_engine_opts": "", 2024-09-18T00:18:03.344 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_agent_delay_time": "5.000000", 2024-09-18T00:18:03.344 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_agent_hist_halflife": "1000", 2024-09-18T00:18:03.344 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_agent_max_low_ops": "2", 2024-09-18T00:18:03.344 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_agent_max_ops": "4", 2024-09-18T00:18:03.344 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-18T00:18:03.344 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-18T00:18:03.344 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_agent_slop": "0.020000", 2024-09-18T00:18:03.344 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-18T00:18:03.344 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-18T00:18:03.344 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_asio_thread_count": "2", 2024-09-18T00:18:03.345 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_async_recovery_min_cost": "100", 2024-09-18T00:18:03.345 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-18T00:18:03.345 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-18T00:18:03.345 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_backfill_scan_max": "512", 2024-09-18T00:18:03.345 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_backfill_scan_min": "64", 2024-09-18T00:18:03.345 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_backoff_on_degraded": "false", 2024-09-18T00:18:03.345 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_backoff_on_peering": "false", 2024-09-18T00:18:03.345 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_backoff_on_unfound": "true", 2024-09-18T00:18:03.345 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_beacon_report_interval": "300", 2024-09-18T00:18:03.345 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bench_duration": "30", 2024-09-18T00:18:03.345 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-18T00:18:03.345 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bench_max_block_size": "67108864", 2024-09-18T00:18:03.345 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-18T00:18:03.345 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_blkin_trace_all": "false", 2024-09-18T00:18:03.346 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-18T00:18:03.346 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-18T00:18:03.346 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-18T00:18:03.346 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-18T00:18:03.346 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_check_for_log_corruption": "false", 2024-09-18T00:18:03.346 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-18T00:18:03.346 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-18T00:18:03.346 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-18T00:18:03.346 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-18T00:18:03.346 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_class_update_on_start": "true", 2024-09-18T00:18:03.346 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_client_message_cap": "256", 2024-09-18T00:18:03.346 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_client_message_size_cap": "524288000", 2024-09-18T00:18:03.346 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_client_op_priority": "63", 2024-09-18T00:18:03.347 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_client_watch_timeout": "30", 2024-09-18T00:18:03.347 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_command_max_records": "256", 2024-09-18T00:18:03.347 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_compact_on_start": "false", 2024-09-18T00:18:03.347 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-18T00:18:03.347 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-18T00:18:03.347 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-18T00:18:03.347 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_crush_update_on_start": "true", 2024-09-18T00:18:03.347 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_crush_update_weight_set": "true", 2024-09-18T00:18:03.347 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-18T00:18:03.347 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-18T00:18:03.347 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-18T00:18:03.347 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-18T00:18:03.347 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-18T00:18:03.347 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_feed_pullee": "-1", 2024-09-18T00:18:03.348 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-18T00:18:03.348 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-18T00:18:03.348 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-18T00:18:03.348 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_misdirected_ops": "false", 2024-09-18T00:18:03.348 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_no_acting_change": "false", 2024-09-18T00:18:03.348 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_no_purge_strays": "false", 2024-09-18T00:18:03.348 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_op_order": "false", 2024-09-18T00:18:03.348 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-18T00:18:03.348 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-18T00:18:03.348 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-18T00:18:03.348 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-18T00:18:03.348 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_shutdown": "false", 2024-09-18T00:18:03.348 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-18T00:18:03.349 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-18T00:18:03.349 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_trim_objects": "false", 2024-09-18T00:18:03.349 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-18T00:18:03.349 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-18T00:18:03.349 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_verify_snaps": "false", 2024-09-18T00:18:03.349 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-18T00:18:03.349 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-18T00:18:03.349 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_deep_scrub_keys": "1024", 2024-09-18T00:18:03.349 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-18T00:18:03.349 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-18T00:18:03.349 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-18T00:18:03.349 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_deep_scrub_stride": "524288", 2024-09-18T00:18:03.349 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-18T00:18:03.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-18T00:18:03.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_default_notify_timeout": "30", 2024-09-18T00:18:03.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_delete_sleep": "0.000000", 2024-09-18T00:18:03.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-18T00:18:03.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-18T00:18:03.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-18T00:18:03.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_discard_disconnected_ops": "true", 2024-09-18T00:18:03.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_enable_op_tracker": "true", 2024-09-18T00:18:03.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-18T00:18:03.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-18T00:18:03.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-18T00:18:03.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-18T00:18:03.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_fast_info": "true", 2024-09-18T00:18:03.350 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_fast_shutdown": "true", 2024-09-18T00:18:03.351 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-18T00:18:03.351 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-18T00:18:03.351 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-18T00:18:03.351 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-18T00:18:03.351 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-18T00:18:03.351 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_function_tracing": "false", 2024-09-18T00:18:03.351 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_heartbeat_grace": "20", 2024-09-18T00:18:03.351 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_heartbeat_interval": "6", 2024-09-18T00:18:03.351 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-18T00:18:03.351 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_heartbeat_min_peers": "10", 2024-09-18T00:18:03.351 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_heartbeat_min_size": "2000", 2024-09-18T00:18:03.351 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_heartbeat_stale": "600", 2024-09-18T00:18:03.351 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-18T00:18:03.351 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_hit_set_max_size": "100000", 2024-09-18T00:18:03.352 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_hit_set_min_size": "1000", 2024-09-18T00:18:03.352 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-18T00:18:03.352 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-18T00:18:03.352 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-18T00:18:03.352 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-18T00:18:03.352 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-18T00:18:03.352 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-18T00:18:03.352 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_journal_size": "5120", 2024-09-18T00:18:03.352 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_kill_backfill_at": "0", 2024-09-18T00:18:03.352 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-18T00:18:03.352 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_map_cache_size": "50", 2024-09-18T00:18:03.352 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_map_dedup": "true", 2024-09-18T00:18:03.352 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_map_message_max": "40", 2024-09-18T00:18:03.353 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-18T00:18:03.353 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_map_share_max_epochs": "40", 2024-09-18T00:18:03.353 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_attr_name_len": "100", 2024-09-18T00:18:03.353 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_attr_size": "0", 2024-09-18T00:18:03.353 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_backfills": "1", 2024-09-18T00:18:03.353 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_markdown_count": "5", 2024-09-18T00:18:03.353 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_markdown_period": "600", 2024-09-18T00:18:03.353 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_object_name_len": "2048", 2024-09-18T00:18:03.353 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_object_namespace_len": "256", 2024-09-18T00:18:03.353 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_object_size": "134217728", 2024-09-18T00:18:03.353 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-18T00:18:03.353 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-18T00:18:03.353 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_pg_blocked_by": "16", 2024-09-18T00:18:03.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_pg_log_entries": "10000", 2024-09-18T00:18:03.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-18T00:18:03.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_pgls": "1024", 2024-09-18T00:18:03.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_push_cost": "8388608", 2024-09-18T00:18:03.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_push_objects": "10", 2024-09-18T00:18:03.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_scrubs": "3", 2024-09-18T00:18:03.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-18T00:18:03.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_trimming_pgs": "2", 2024-09-18T00:18:03.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_write_op_reply_len": "64", 2024-09-18T00:18:03.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_max_write_size": "90", 2024-09-18T00:18:03.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-18T00:18:03.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-18T00:18:03.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-18T00:18:03.354 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-18T00:18:03.355 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-18T00:18:03.355 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-18T00:18:03.355 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-18T00:18:03.355 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-18T00:18:03.355 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_profile": "balanced", 2024-09-18T00:18:03.355 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-18T00:18:03.355 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-18T00:18:03.355 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-18T00:18:03.355 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-18T00:18:03.355 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-18T00:18:03.355 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-18T00:18:03.355 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-18T00:18:03.355 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-18T00:18:03.356 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-18T00:18:03.356 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-18T00:18:03.356 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-18T00:18:03.356 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_memory_base": "805306368", 2024-09-18T00:18:03.356 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_memory_cache_min": "134217728", 2024-09-18T00:18:03.356 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-18T00:18:03.356 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-18T00:18:03.356 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_memory_target": "4294967296", 2024-09-18T00:18:03.356 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_memory_target_autotune": "false", 2024-09-18T00:18:03.356 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-18T00:18:03.356 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_min_pg_log_entries": "250", 2024-09-18T00:18:03.356 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_min_recovery_priority": "0", 2024-09-18T00:18:03.356 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-18T00:18:03.357 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-18T00:18:03.357 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mon_report_interval": "5", 2024-09-18T00:18:03.357 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-18T00:18:03.357 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-18T00:18:03.357 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_num_cache_shards": "32", 2024-09-18T00:18:03.357 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_num_op_tracker_shard": "32", 2024-09-18T00:18:03.357 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_numa_auto_affinity": "true", 2024-09-18T00:18:03.357 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_numa_node": "-1", 2024-09-18T00:18:03.357 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_numa_prefer_iface": "true", 2024-09-18T00:18:03.357 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-18T00:18:03.357 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_objecter_finishers": "1", 2024-09-18T00:18:03.357 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_objectstore": "bluestore", 2024-09-18T00:18:03.357 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_objectstore_fuse": "false", 2024-09-18T00:18:03.357 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_objectstore_tracing": "false", 2024-09-18T00:18:03.358 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_complaint_time": "30.000000", 2024-09-18T00:18:03.358 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_history_duration": "600", 2024-09-18T00:18:03.358 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_history_size": "20", 2024-09-18T00:18:03.358 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_history_slow_op_size": "20", 2024-09-18T00:18:03.358 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-18T00:18:03.358 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_log_threshold": "5", 2024-09-18T00:18:03.358 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_num_shards": "0", 2024-09-18T00:18:03.358 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_num_shards_hdd": "5", 2024-09-18T00:18:03.358 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_num_shards_ssd": "8", 2024-09-18T00:18:03.358 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-18T00:18:03.358 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-18T00:18:03.358 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-18T00:18:03.358 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-18T00:18:03.358 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_pq_min_cost": "65536", 2024-09-18T00:18:03.359 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-18T00:18:03.359 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_queue_cut_off": "high", 2024-09-18T00:18:03.359 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-18T00:18:03.359 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_op_thread_timeout": "15", 2024-09-18T00:18:03.359 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_open_classes_on_start": "true", 2024-09-18T00:18:03.359 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_os_flags": "0", 2024-09-18T00:18:03.359 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_peering_op_priority": "255", 2024-09-18T00:18:03.359 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pg_delete_cost": "1048576", 2024-09-18T00:18:03.359 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pg_delete_priority": "5", 2024-09-18T00:18:03.359 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-18T00:18:03.359 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-18T00:18:03.359 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-18T00:18:03.359 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pg_log_trim_max": "10000", 2024-09-18T00:18:03.360 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pg_log_trim_min": "100", 2024-09-18T00:18:03.360 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-18T00:18:03.360 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-18T00:18:03.360 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-18T00:18:03.360 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-18T00:18:03.360 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-18T00:18:03.360 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-18T00:18:03.360 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-18T00:18:03.360 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-18T00:18:03.360 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-18T00:18:03.360 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-18T00:18:03.360 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_crimson": "false", 2024-09-18T00:18:03.360 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-18T00:18:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-18T00:18:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-18T00:18:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-18T00:18:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-18T00:18:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-18T00:18:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-18T00:18:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-18T00:18:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_flags": "0", 2024-09-18T00:18:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-18T00:18:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_min_size": "0", 2024-09-18T00:18:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-18T00:18:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_pg_num": "32", 2024-09-18T00:18:03.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_pgp_num": "0", 2024-09-18T00:18:03.362 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-18T00:18:03.362 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_read_ratio": "70", 2024-09-18T00:18:03.362 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_size": "2", 2024-09-18T00:18:03.362 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_default_type": "replicated", 2024-09-18T00:18:03.362 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-18T00:18:03.362 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-18T00:18:03.362 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_push_per_object_cost": "1000", 2024-09-18T00:18:03.362 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-18T00:18:03.362 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recover_clone_overlap": "true", 2024-09-18T00:18:03.362 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-18T00:18:03.362 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recovery_cost": "20971520", 2024-09-18T00:18:03.362 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-18T00:18:03.362 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recovery_max_active": "0", 2024-09-18T00:18:03.362 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-18T00:18:03.363 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-18T00:18:03.363 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-18T00:18:03.363 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-18T00:18:03.363 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recovery_max_single_start": "1", 2024-09-18T00:18:03.363 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recovery_op_priority": "3", 2024-09-18T00:18:03.363 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-18T00:18:03.363 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recovery_priority": "5", 2024-09-18T00:18:03.363 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-18T00:18:03.363 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recovery_sleep": "0.000000", 2024-09-18T00:18:03.364 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-18T00:18:03.364 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-18T00:18:03.364 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-18T00:18:03.364 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_repair_during_recovery": "false", 2024-09-18T00:18:03.364 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_requested_scrub_priority": "120", 2024-09-18T00:18:03.364 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-18T00:18:03.364 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-18T00:18:03.364 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_auto_repair": "false", 2024-09-18T00:18:03.364 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-18T00:18:03.364 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-18T00:18:03.364 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_begin_hour": "0", 2024-09-18T00:18:03.364 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_begin_week_day": "0", 2024-09-18T00:18:03.364 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_chunk_max": "25", 2024-09-18T00:18:03.365 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_chunk_min": "5", 2024-09-18T00:18:03.365 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_cost": "52428800", 2024-09-18T00:18:03.365 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-18T00:18:03.365 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_during_recovery": "false", 2024-09-18T00:18:03.365 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_end_hour": "0", 2024-09-18T00:18:03.365 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_end_week_day": "0", 2024-09-18T00:18:03.365 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_event_cost": "4096", 2024-09-18T00:18:03.365 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-18T00:18:03.365 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-18T00:18:03.365 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_invalid_stats": "true", 2024-09-18T00:18:03.365 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-18T00:18:03.365 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-18T00:18:03.365 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_max_preemptions": "5", 2024-09-18T00:18:03.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-18T00:18:03.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_priority": "5", 2024-09-18T00:18:03.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_scrub_sleep": "0.000000", 2024-09-18T00:18:03.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-18T00:18:03.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-18T00:18:03.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-18T00:18:03.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_skip_data_digest": "false", 2024-09-18T00:18:03.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_smart_report_timeout": "5", 2024-09-18T00:18:03.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_snap_trim_cost": "1048576", 2024-09-18T00:18:03.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_snap_trim_priority": "5", 2024-09-18T00:18:03.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-18T00:18:03.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-18T00:18:03.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-18T00:18:03.367 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-18T00:18:03.367 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-18T00:18:03.367 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-18T00:18:03.367 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-18T00:18:03.367 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_target_transaction_size": "30", 2024-09-18T00:18:03.367 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-18T00:18:03.367 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-18T00:18:03.367 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-18T00:18:03.367 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-18T00:18:03.367 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-18T00:18:03.367 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-18T00:18:03.367 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-18T00:18:03.367 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-18T00:18:03.368 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-18T00:18:03.368 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-18T00:18:03.368 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_tracing": "false", 2024-09-18T00:18:03.368 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_use_stale_snap": "false", 2024-09-18T00:18:03.368 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-18T00:18:03.368 INFO:teuthology.orchestra.run.smithi097.stdout: "osdc_blkin_trace_all": "false", 2024-09-18T00:18:03.368 INFO:teuthology.orchestra.run.smithi097.stdout: "paxos_kill_at": "0", 2024-09-18T00:18:03.368 INFO:teuthology.orchestra.run.smithi097.stdout: "paxos_max_join_drift": "10", 2024-09-18T00:18:03.368 INFO:teuthology.orchestra.run.smithi097.stdout: "paxos_min": "500", 2024-09-18T00:18:03.368 INFO:teuthology.orchestra.run.smithi097.stdout: "paxos_min_wait": "0.050000", 2024-09-18T00:18:03.368 INFO:teuthology.orchestra.run.smithi097.stdout: "paxos_propose_interval": "1.000000", 2024-09-18T00:18:03.368 INFO:teuthology.orchestra.run.smithi097.stdout: "paxos_service_trim_max": "500", 2024-09-18T00:18:03.368 INFO:teuthology.orchestra.run.smithi097.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-18T00:18:03.369 INFO:teuthology.orchestra.run.smithi097.stdout: "paxos_service_trim_min": "250", 2024-09-18T00:18:03.369 INFO:teuthology.orchestra.run.smithi097.stdout: "paxos_stash_full_interval": "25", 2024-09-18T00:18:03.369 INFO:teuthology.orchestra.run.smithi097.stdout: "paxos_trim_max": "500", 2024-09-18T00:18:03.369 INFO:teuthology.orchestra.run.smithi097.stdout: "paxos_trim_min": "250", 2024-09-18T00:18:03.369 INFO:teuthology.orchestra.run.smithi097.stdout: "perf": "true", 2024-09-18T00:18:03.369 INFO:teuthology.orchestra.run.smithi097.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-18T00:18:03.369 INFO:teuthology.orchestra.run.smithi097.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-18T00:18:03.369 INFO:teuthology.orchestra.run.smithi097.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-18T00:18:03.369 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addr": "-", 2024-09-18T00:18:03.369 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addrv": "", 2024-09-18T00:18:03.369 INFO:teuthology.orchestra.run.smithi097.stdout: "public_bind_addr": "-", 2024-09-18T00:18:03.369 INFO:teuthology.orchestra.run.smithi097.stdout: "public_network": "", 2024-09-18T00:18:03.369 INFO:teuthology.orchestra.run.smithi097.stdout: "public_network_interface": "", 2024-09-18T00:18:03.369 INFO:teuthology.orchestra.run.smithi097.stdout: "qat_compressor_busy_polling": "false", 2024-09-18T00:18:03.370 INFO:teuthology.orchestra.run.smithi097.stdout: "qat_compressor_enabled": "false", 2024-09-18T00:18:03.370 INFO:teuthology.orchestra.run.smithi097.stdout: "qat_compressor_session_max_number": "256", 2024-09-18T00:18:03.370 INFO:teuthology.orchestra.run.smithi097.stdout: "rados_mon_op_timeout": "0", 2024-09-18T00:18:03.370 INFO:teuthology.orchestra.run.smithi097.stdout: "rados_osd_op_timeout": "0", 2024-09-18T00:18:03.370 INFO:teuthology.orchestra.run.smithi097.stdout: "rados_tracing": "false", 2024-09-18T00:18:03.370 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_atime_update_interval": "60", 2024-09-18T00:18:03.370 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-18T00:18:03.370 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_balance_parent_reads": "false", 2024-09-18T00:18:03.370 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_balance_snap_reads": "false", 2024-09-18T00:18:03.370 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_blkin_trace_all": "false", 2024-09-18T00:18:03.370 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-18T00:18:03.370 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-18T00:18:03.370 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_cache": "true", 2024-09-18T00:18:03.370 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-18T00:18:03.371 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-18T00:18:03.371 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-18T00:18:03.371 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-18T00:18:03.371 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_cache_policy": "writearound", 2024-09-18T00:18:03.371 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_cache_size": "33554432", 2024-09-18T00:18:03.371 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-18T00:18:03.371 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-18T00:18:03.371 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_clone_copy_on_read": "false", 2024-09-18T00:18:03.371 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_compression_hint": "none", 2024-09-18T00:18:03.371 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_concurrent_management_ops": "10", 2024-09-18T00:18:03.371 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-18T00:18:03.371 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_default_clone_format": "auto", 2024-09-18T00:18:03.371 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_default_data_pool": "", 2024-09-18T00:18:03.371 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_default_features": "61", 2024-09-18T00:18:03.372 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_default_format": "2", 2024-09-18T00:18:03.372 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_default_map_options": "", 2024-09-18T00:18:03.372 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_default_order": "22", 2024-09-18T00:18:03.372 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_default_pool": "rbd", 2024-09-18T00:18:03.372 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-18T00:18:03.372 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_default_stripe_count": "0", 2024-09-18T00:18:03.372 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_default_stripe_unit": "0", 2024-09-18T00:18:03.372 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-18T00:18:03.372 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-18T00:18:03.372 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-18T00:18:03.372 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_enable_alloc_hint": "true", 2024-09-18T00:18:03.372 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-18T00:18:03.372 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_io_scheduler": "simple", 2024-09-18T00:18:03.373 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-18T00:18:03.373 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-18T00:18:03.373 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-18T00:18:03.373 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-18T00:18:03.373 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-18T00:18:03.373 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-18T00:18:03.373 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-18T00:18:03.373 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-18T00:18:03.373 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-18T00:18:03.373 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_journal_order": "24", 2024-09-18T00:18:03.373 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_journal_pool": "", 2024-09-18T00:18:03.373 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_journal_splay_width": "4", 2024-09-18T00:18:03.373 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_localize_parent_reads": "false", 2024-09-18T00:18:03.373 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_localize_snap_reads": "false", 2024-09-18T00:18:03.374 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-18T00:18:03.374 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-18T00:18:03.374 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-18T00:18:03.374 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-18T00:18:03.374 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-18T00:18:03.374 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-18T00:18:03.374 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-18T00:18:03.374 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-18T00:18:03.374 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-18T00:18:03.374 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-18T00:18:03.374 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-18T00:18:03.374 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-18T00:18:03.374 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-18T00:18:03.375 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-18T00:18:03.375 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-18T00:18:03.375 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-18T00:18:03.375 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-18T00:18:03.375 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-18T00:18:03.375 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-18T00:18:03.375 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-18T00:18:03.375 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-18T00:18:03.375 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-18T00:18:03.375 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-18T00:18:03.375 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-18T00:18:03.375 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-18T00:18:03.375 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-18T00:18:03.375 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-18T00:18:03.376 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-18T00:18:03.376 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-18T00:18:03.376 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-18T00:18:03.376 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-18T00:18:03.376 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_mtime_update_interval": "60", 2024-09-18T00:18:03.376 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_non_blocking_aio": "true", 2024-09-18T00:18:03.376 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_op_thread_timeout": "60", 2024-09-18T00:18:03.376 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_op_threads": "1", 2024-09-18T00:18:03.376 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_parent_cache_enabled": "false", 2024-09-18T00:18:03.376 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-18T00:18:03.376 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-18T00:18:03.376 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-18T00:18:03.376 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_plugins": "", 2024-09-18T00:18:03.377 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_bps_burst": "0", 2024-09-18T00:18:03.377 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-18T00:18:03.377 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_bps_limit": "0", 2024-09-18T00:18:03.377 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_exclude_ops": "0", 2024-09-18T00:18:03.377 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_iops_burst": "0", 2024-09-18T00:18:03.377 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-18T00:18:03.377 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_iops_limit": "0", 2024-09-18T00:18:03.377 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-18T00:18:03.377 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-18T00:18:03.377 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-18T00:18:03.377 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-18T00:18:03.377 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-18T00:18:03.377 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-18T00:18:03.378 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-18T00:18:03.378 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-18T00:18:03.378 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-18T00:18:03.378 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-18T00:18:03.378 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-18T00:18:03.378 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-18T00:18:03.378 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-18T00:18:03.378 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-18T00:18:03.378 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_read_from_replica_policy": "default", 2024-09-18T00:18:03.378 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-18T00:18:03.378 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-18T00:18:03.378 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-18T00:18:03.378 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-18T00:18:03.378 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_skip_partial_discard": "true", 2024-09-18T00:18:03.379 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-18T00:18:03.379 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_tracing": "false", 2024-09-18T00:18:03.379 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_validate_names": "true", 2024-09-18T00:18:03.379 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_validate_pool": "true", 2024-09-18T00:18:03.379 INFO:teuthology.orchestra.run.smithi097.stdout: "restapi_base_url": "", 2024-09-18T00:18:03.379 INFO:teuthology.orchestra.run.smithi097.stdout: "restapi_log_level": "", 2024-09-18T00:18:03.379 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-18T00:18:03.379 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-18T00:18:03.379 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_acl_grants_max_num": "100", 2024-09-18T00:18:03.379 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_admin_entry": "admin", 2024-09-18T00:18:03.379 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-18T00:18:03.379 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_backend_store": "rados", 2024-09-18T00:18:03.379 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_barbican_url": "", 2024-09-18T00:18:03.380 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_beast_enable_async": "true", 2024-09-18T00:18:03.380 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_bucket_counters_cache": "false", 2024-09-18T00:18:03.380 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-18T00:18:03.380 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-18T00:18:03.380 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-18T00:18:03.380 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-18T00:18:03.380 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-18T00:18:03.380 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-18T00:18:03.380 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-18T00:18:03.380 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-18T00:18:03.380 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_cache_enabled": "true", 2024-09-18T00:18:03.380 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_cache_expiry_interval": "900", 2024-09-18T00:18:03.380 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_cache_lru_size": "25000", 2024-09-18T00:18:03.381 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_config_store": "rados", 2024-09-18T00:18:03.381 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_content_length_compat": "false", 2024-09-18T00:18:03.381 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_copy_obj_progress": "true", 2024-09-18T00:18:03.381 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-18T00:18:03.381 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_cors_rules_max_num": "100", 2024-09-18T00:18:03.381 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_cross_domain_policy": "", 2024-09-18T00:18:03.381 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-18T00:18:03.381 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_kmip_addr": "", 2024-09-18T00:18:03.381 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-18T00:18:03.381 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-18T00:18:03.381 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-18T00:18:03.381 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-18T00:18:03.381 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_kmip_password": "", 2024-09-18T00:18:03.381 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-18T00:18:03.382 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_kmip_username": "", 2024-09-18T00:18:03.382 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_require_ssl": "true", 2024-09-18T00:18:03.382 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-18T00:18:03.382 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-18T00:18:03.382 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-18T00:18:03.382 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-18T00:18:03.382 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-18T00:18:03.382 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-18T00:18:03.382 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-18T00:18:03.382 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-18T00:18:03.382 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-18T00:18:03.382 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-18T00:18:03.382 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-18T00:18:03.382 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-18T00:18:03.383 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-18T00:18:03.383 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-18T00:18:03.383 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-18T00:18:03.383 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_vault_addr": "", 2024-09-18T00:18:03.383 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_vault_auth": "token", 2024-09-18T00:18:03.383 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_vault_namespace": "", 2024-09-18T00:18:03.383 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_vault_prefix": "", 2024-09-18T00:18:03.383 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-18T00:18:03.383 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-18T00:18:03.383 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-18T00:18:03.383 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-18T00:18:03.383 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_vault_token_file": "", 2024-09-18T00:18:03.383 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-18T00:18:03.383 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_curl_buffersize": "524288", 2024-09-18T00:18:03.384 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-18T00:18:03.384 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_curl_low_speed_time": "300", 2024-09-18T00:18:03.384 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-18T00:18:03.384 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-18T00:18:03.384 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-18T00:18:03.384 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-18T00:18:03.384 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-18T00:18:03.384 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-18T00:18:03.384 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-18T00:18:03.384 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-18T00:18:03.384 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-18T00:18:03.384 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-18T00:18:03.384 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-18T00:18:03.385 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_d4n_port": "6379", 2024-09-18T00:18:03.385 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-18T00:18:03.385 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_data_log_changes_size": "1000", 2024-09-18T00:18:03.385 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_data_log_num_shards": "128", 2024-09-18T00:18:03.385 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-18T00:18:03.385 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_data_log_window": "30", 2024-09-18T00:18:03.385 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-18T00:18:03.385 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-18T00:18:03.385 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-18T00:18:03.385 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-18T00:18:03.385 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-18T00:18:03.385 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-18T00:18:03.385 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-18T00:18:03.386 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-18T00:18:03.386 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-18T00:18:03.386 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-18T00:18:03.386 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-18T00:18:03.386 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-18T00:18:03.386 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-18T00:18:03.386 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-18T00:18:03.386 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-18T00:18:03.386 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-18T00:18:03.386 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-18T00:18:03.386 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-18T00:18:03.386 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-18T00:18:03.386 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-18T00:18:03.386 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-18T00:18:03.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-18T00:18:03.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-18T00:18:03.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-18T00:18:03.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-18T00:18:03.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_dns_name": "", 2024-09-18T00:18:03.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_dns_s3website_name": "", 2024-09-18T00:18:03.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_dynamic_resharding": "true", 2024-09-18T00:18:03.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-18T00:18:03.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_enable_gc_threads": "true", 2024-09-18T00:18:03.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_enable_lc_threads": "true", 2024-09-18T00:18:03.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_enable_ops_log": "false", 2024-09-18T00:18:03.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_enable_quota_threads": "true", 2024-09-18T00:18:03.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_enable_static_website": "false", 2024-09-18T00:18:03.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_enable_usage_log": "false", 2024-09-18T00:18:03.388 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_enforce_swift_acls": "true", 2024-09-18T00:18:03.388 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_exit_timeout_secs": "120", 2024-09-18T00:18:03.388 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_expose_bucket": "false", 2024-09-18T00:18:03.388 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_extended_http_attrs": "", 2024-09-18T00:18:03.388 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_filter": "none", 2024-09-18T00:18:03.388 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-18T00:18:03.388 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_frontends": "beast port=7480", 2024-09-18T00:18:03.388 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-18T00:18:03.388 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_gc_max_deferred": "50", 2024-09-18T00:18:03.388 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-18T00:18:03.388 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_gc_max_objs": "32", 2024-09-18T00:18:03.388 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-18T00:18:03.388 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-18T00:18:03.389 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-18T00:18:03.389 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-18T00:18:03.389 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_gc_processor_period": "3600", 2024-09-18T00:18:03.389 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-18T00:18:03.389 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-18T00:18:03.389 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-18T00:18:03.389 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-18T00:18:03.389 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_init_timeout": "300", 2024-09-18T00:18:03.389 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-18T00:18:03.389 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-18T00:18:03.389 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_kafka_connection_idle": "300", 2024-09-18T00:18:03.389 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-18T00:18:03.389 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-18T00:18:03.389 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-18T00:18:03.390 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-18T00:18:03.390 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-18T00:18:03.390 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_admin_domain": "", 2024-09-18T00:18:03.390 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_admin_password": "", 2024-09-18T00:18:03.390 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_admin_password_path": "", 2024-09-18T00:18:03.390 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_admin_project": "", 2024-09-18T00:18:03.390 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_admin_tenant": "", 2024-09-18T00:18:03.390 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_admin_token": "", 2024-09-18T00:18:03.390 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_admin_token_path": "", 2024-09-18T00:18:03.390 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_admin_user": "", 2024-09-18T00:18:03.390 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_api_version": "2", 2024-09-18T00:18:03.390 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_barbican_domain": "", 2024-09-18T00:18:03.390 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_barbican_password": "", 2024-09-18T00:18:03.391 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_barbican_project": "", 2024-09-18T00:18:03.391 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-18T00:18:03.391 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_barbican_user": "", 2024-09-18T00:18:03.391 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-18T00:18:03.391 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-18T00:18:03.391 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-18T00:18:03.391 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-18T00:18:03.391 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-18T00:18:03.391 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_url": "", 2024-09-18T00:18:03.391 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-18T00:18:03.391 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_lc_debug_interval": "-1", 2024-09-18T00:18:03.391 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_lc_lock_max_time": "90", 2024-09-18T00:18:03.391 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_lc_max_objs": "32", 2024-09-18T00:18:03.392 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_lc_max_rules": "1000", 2024-09-18T00:18:03.392 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_lc_max_worker": "3", 2024-09-18T00:18:03.392 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-18T00:18:03.392 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_lc_thread_delay": "0", 2024-09-18T00:18:03.392 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-18T00:18:03.392 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_ldap_dnattr": "uid", 2024-09-18T00:18:03.392 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-18T00:18:03.392 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_ldap_searchfilter": "", 2024-09-18T00:18:03.392 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-18T00:18:03.392 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_ldap_uri": "", 2024-09-18T00:18:03.392 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-18T00:18:03.392 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-18T00:18:03.392 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-18T00:18:03.393 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_log_http_headers": "", 2024-09-18T00:18:03.393 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-18T00:18:03.393 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-18T00:18:03.393 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_log_object_name_utc": "false", 2024-09-18T00:18:03.393 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-18T00:18:03.393 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-18T00:18:03.393 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_max_attr_name_len": "0", 2024-09-18T00:18:03.393 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_max_attr_size": "0", 2024-09-18T00:18:03.393 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-18T00:18:03.393 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_max_chunk_size": "4194304", 2024-09-18T00:18:03.393 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-18T00:18:03.393 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-18T00:18:03.393 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-18T00:18:03.394 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_max_listing_results": "1000", 2024-09-18T00:18:03.394 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_max_notify_retries": "10", 2024-09-18T00:18:03.394 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-18T00:18:03.394 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_max_put_param_size": "1048576", 2024-09-18T00:18:03.394 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_max_put_size": "5368709120", 2024-09-18T00:18:03.394 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_max_slo_entries": "1000", 2024-09-18T00:18:03.394 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_md_log_max_shards": "64", 2024-09-18T00:18:03.394 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-18T00:18:03.394 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-18T00:18:03.394 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-18T00:18:03.394 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-18T00:18:03.394 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_mp_lock_max_time": "600", 2024-09-18T00:18:03.394 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-18T00:18:03.394 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-18T00:18:03.395 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-18T00:18:03.395 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-18T00:18:03.395 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-18T00:18:03.395 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-18T00:18:03.395 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-18T00:18:03.395 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-18T00:18:03.395 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_nfs_max_gc": "300", 2024-09-18T00:18:03.395 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-18T00:18:03.395 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-18T00:18:03.395 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-18T00:18:03.395 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-18T00:18:03.395 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-18T00:18:03.395 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-18T00:18:03.395 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-18T00:18:03.396 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_num_async_rados_threads": "32", 2024-09-18T00:18:03.396 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_num_control_oids": "8", 2024-09-18T00:18:03.396 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_numa_node": "-1", 2024-09-18T00:18:03.396 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-18T00:18:03.396 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-18T00:18:03.396 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_objexp_chunk_size": "100", 2024-09-18T00:18:03.396 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_objexp_gc_interval": "600", 2024-09-18T00:18:03.396 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-18T00:18:03.396 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-18T00:18:03.396 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-18T00:18:03.396 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_op_thread_timeout": "600", 2024-09-18T00:18:03.396 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_op_tracing": "false", 2024-09-18T00:18:03.396 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_opa_token": "", 2024-09-18T00:18:03.397 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_opa_url": "", 2024-09-18T00:18:03.397 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_opa_verify_ssl": "true", 2024-09-18T00:18:03.397 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-18T00:18:03.397 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-18T00:18:03.397 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_ops_log_rados": "false", 2024-09-18T00:18:03.397 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_ops_log_socket_path": "", 2024-09-18T00:18:03.397 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-18T00:18:03.397 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-18T00:18:03.397 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-18T00:18:03.397 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_period_push_interval": "2.000000", 2024-09-18T00:18:03.397 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-18T00:18:03.397 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-18T00:18:03.397 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-18T00:18:03.397 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-18T00:18:03.398 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_posix_cache_lanes": "3", 2024-09-18T00:18:03.398 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-18T00:18:03.398 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-18T00:18:03.398 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_posix_cache_partitions": "3", 2024-09-18T00:18:03.398 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-18T00:18:03.398 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_print_continue": "true", 2024-09-18T00:18:03.398 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-18T00:18:03.398 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-18T00:18:03.398 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-18T00:18:03.398 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-18T00:18:03.398 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-18T00:18:03.398 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_rados_tracing": "false", 2024-09-18T00:18:03.398 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_realm": "", 2024-09-18T00:18:03.399 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_realm_id": "", 2024-09-18T00:18:03.399 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-18T00:18:03.399 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_region": "", 2024-09-18T00:18:03.399 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-18T00:18:03.399 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-18T00:18:03.399 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-18T00:18:03.399 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-18T00:18:03.399 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_request_uri": "", 2024-09-18T00:18:03.399 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_reshard_batch_size": "64", 2024-09-18T00:18:03.399 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-18T00:18:03.399 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_reshard_max_aio": "128", 2024-09-18T00:18:03.399 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_reshard_num_logs": "16", 2024-09-18T00:18:03.399 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_reshard_thread_interval": "600", 2024-09-18T00:18:03.399 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_resolve_cname": "false", 2024-09-18T00:18:03.400 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-18T00:18:03.400 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_run_sync_thread": "true", 2024-09-18T00:18:03.400 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-18T00:18:03.400 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-18T00:18:03.400 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-18T00:18:03.400 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-18T00:18:03.400 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-18T00:18:03.400 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-18T00:18:03.400 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-18T00:18:03.400 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-18T00:18:03.400 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-18T00:18:03.400 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_scheduler_type": "throttler", 2024-09-18T00:18:03.400 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_script_uri": "", 2024-09-18T00:18:03.400 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_service_provider_name": "", 2024-09-18T00:18:03.401 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-18T00:18:03.401 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sts_client_id": "", 2024-09-18T00:18:03.401 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sts_client_secret": "", 2024-09-18T00:18:03.401 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sts_entry": "sts", 2024-09-18T00:18:03.401 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sts_key": "sts", 2024-09-18T00:18:03.401 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-18T00:18:03.401 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sts_min_session_duration": "900", 2024-09-18T00:18:03.401 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sts_token_introspection_url": "", 2024-09-18T00:18:03.401 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_swift_account_in_url": "false", 2024-09-18T00:18:03.401 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_swift_auth_entry": "auth", 2024-09-18T00:18:03.401 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_swift_auth_url": "", 2024-09-18T00:18:03.401 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_swift_custom_header": "", 2024-09-18T00:18:03.401 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-18T00:18:03.402 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_swift_need_stats": "true", 2024-09-18T00:18:03.402 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_swift_tenant_name": "", 2024-09-18T00:18:03.402 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_swift_token_expiration": "86400", 2024-09-18T00:18:03.402 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_swift_url": "", 2024-09-18T00:18:03.402 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_swift_url_prefix": "swift", 2024-09-18T00:18:03.402 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-18T00:18:03.402 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-18T00:18:03.402 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-18T00:18:03.402 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sync_lease_period": "120", 2024-09-18T00:18:03.402 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-18T00:18:03.402 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-18T00:18:03.402 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-18T00:18:03.402 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-18T00:18:03.402 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-18T00:18:03.403 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-18T00:18:03.403 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-18T00:18:03.403 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-18T00:18:03.403 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-18T00:18:03.403 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_thread_pool_size": "512", 2024-09-18T00:18:03.403 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-18T00:18:03.403 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-18T00:18:03.403 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-18T00:18:03.403 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-18T00:18:03.403 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_torrent_comment": "", 2024-09-18T00:18:03.403 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_torrent_createby": "", 2024-09-18T00:18:03.403 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_torrent_encoding": "", 2024-09-18T00:18:03.403 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_torrent_flag": "false", 2024-09-18T00:18:03.403 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-18T00:18:03.404 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_torrent_origin": "", 2024-09-18T00:18:03.404 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-18T00:18:03.404 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_torrent_tracker": "", 2024-09-18T00:18:03.404 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_trust_forwarded_https": "false", 2024-09-18T00:18:03.404 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-18T00:18:03.404 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-18T00:18:03.404 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_usage_max_shards": "32", 2024-09-18T00:18:03.404 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_usage_max_user_shards": "1", 2024-09-18T00:18:03.404 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_use_opa_authz": "false", 2024-09-18T00:18:03.404 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_user_counters_cache": "false", 2024-09-18T00:18:03.404 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-18T00:18:03.404 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-18T00:18:03.404 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-18T00:18:03.404 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_user_max_buckets": "1000", 2024-09-18T00:18:03.405 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_user_policies_max_num": "100", 2024-09-18T00:18:03.405 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-18T00:18:03.405 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-18T00:18:03.405 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-18T00:18:03.405 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-18T00:18:03.405 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_user_unique_email": "true", 2024-09-18T00:18:03.405 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_verify_ssl": "true", 2024-09-18T00:18:03.405 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-18T00:18:03.405 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_zone": "", 2024-09-18T00:18:03.405 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_zone_id": "", 2024-09-18T00:18:03.405 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-18T00:18:03.405 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_zonegroup": "", 2024-09-18T00:18:03.405 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_zonegroup_id": "", 2024-09-18T00:18:03.405 INFO:teuthology.orchestra.run.smithi097.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-18T00:18:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "rgwlc_auto_session_clear": "true", 2024-09-18T00:18:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-18T00:18:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_block_size": "4096", 2024-09-18T00:18:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-18T00:18:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-18T00:18:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-18T00:18:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-18T00:18:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-18T00:18:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_cache_size": "536870912", 2024-09-18T00:18:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-18T00:18:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-18T00:18:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-18T00:18:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-18T00:18:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-18T00:18:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-18T00:18:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-18T00:18:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-18T00:18:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_index_type": "binary_search", 2024-09-18T00:18:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-18T00:18:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-18T00:18:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_partition_filters": "false", 2024-09-18T00:18:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_perf": "false", 2024-09-18T00:18:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-18T00:18:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-18T00:18:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-18T00:18:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "run_dir": "/var/run/ceph", 2024-09-18T00:18:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_block_create": "true", 2024-09-18T00:18:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_cache_lru_size": "67108864", 2024-09-18T00:18:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-18T00:18:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-18T00:18:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_default_max_object_size": "16777216", 2024-09-18T00:18:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-18T00:18:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_device_size": "53687091200", 2024-09-18T00:18:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-18T00:18:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_full_integrity_check": "false", 2024-09-18T00:18:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_journal_batch_capacity": "16", 2024-09-18T00:18:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-18T00:18:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-18T00:18:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-18T00:18:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_main_device_type": "SSD", 2024-09-18T00:18:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-18T00:18:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-18T00:18:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-18T00:18:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-18T00:18:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-18T00:18:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-18T00:18:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "seastore_segment_size": "67108864", 2024-09-18T00:18:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "set_keepcaps": "false", 2024-09-18T00:18:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "setgroup": "", 2024-09-18T00:18:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "setuser": "", 2024-09-18T00:18:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "setuser_match_path": "", 2024-09-18T00:18:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-18T00:18:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "thp": "false", 2024-09-18T00:18:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "threadpool_default_timeout": "60", 2024-09-18T00:18:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-18T00:18:03.410 INFO:teuthology.orchestra.run.smithi097.stdout: "throttler_perf_counter": "true" 2024-09-18T00:18:03.410 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:03.412 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-18T00:18:03.555 INFO:teuthology.orchestra.run.smithi097.stdout:mon.a: {} 2024-09-18T00:18:03.556 INFO:teuthology.orchestra.run.smithi097.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T00:18:03.569 INFO:teuthology.orchestra.run.smithi097.stdout:mon.b: {} 2024-09-18T00:18:03.569 INFO:teuthology.orchestra.run.smithi097.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T00:18:03.578 INFO:teuthology.orchestra.run.smithi097.stdout:mon.c: {} 2024-09-18T00:18:03.578 INFO:teuthology.orchestra.run.smithi097.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T00:18:03.594 DEBUG:teuthology.orchestra.run.smithi114:> type ceph-objectstore-tool 2024-09-18T00:18:03.600 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T00:18:03.600 DEBUG:teuthology.orchestra.run.smithi097:> type ceph-objectstore-tool 2024-09-18T00:18:03.605 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T00:18:03.605 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-18T00:18:03.615 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-18T00:18:03.615 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-18T00:18:03.615 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:18:03.615 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:18:03.615 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-18T00:18:03.615 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T00:18:03.616 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T00:18:03.616 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-18T00:18:03.616 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-18T00:18:03.616 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-18T00:18:03.617 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:18:03.617 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-18T00:18:03.617 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:18:03.617 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-18T00:18:03.617 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:18:03.618 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-18T00:18:03.653 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:03.651+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:03.669 INFO:teuthology.orchestra.run.smithi097.stdout:mapped writes DISABLED 2024-09-18T00:18:03.669 INFO:teuthology.orchestra.run.smithi097.stdout:mapped reads DISABLED 2024-09-18T00:18:03.669 INFO:teuthology.orchestra.run.smithi097.stdout:Seed set to 5281 2024-09-18T00:18:03.718 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:03.715+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:03.769 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:03.769 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:03.769 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:03.770 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:03.772 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:03.772 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T00:18:03.772 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:03.783 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:18:03.786 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:03.786 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:03.802 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:18:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:18:03.819 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:03.815+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:03.820 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:03.821 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:03.838 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:03.839 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:03.858 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:03.858 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:03.878 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:18:03.878 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:18:03.891 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:03.891 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:03.898 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:03.898 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:03.904 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:18:03.918 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:03.919 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:03.920 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:03.919+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:04.021 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:04.019+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:04.023 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:04.023 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:04.035 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:18:04.122 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:04.115+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:04.200 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:04.200 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:04.213 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:18:04.222 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:04.219+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:04.323 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:04.319+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:04.332 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:04.332 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:04.345 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:18:04.433 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:04.431+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:04.510 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:04.510 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:04.526 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:04.523+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:04.527 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is set 2024-09-18T00:18:04.529 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:18:04.626 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:04.623+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:04.714 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:04.714 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:04.728 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:18:04.729 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:04.723+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:04.828 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:04.827+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:04.849 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:04.849 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:04.862 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:18:04.930 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:04.923+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:05.026 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:05.026 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:05.031 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:05.027+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:05.040 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:18:05.132 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:05.127+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:05.204 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:05.204 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:05.218 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:18:05.261 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:05.259+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:05.334 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:05.327+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:05.377 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:05.377 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:05.398 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:18:05.434 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:05.431+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:05.520 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:05.520 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:05.534 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:18:05.536 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:05.531+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:05.637 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:05.635+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:05.666 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:05.666 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:05.679 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:18:05.737 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:05.731+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:05.804 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:05.804 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:05.819 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:18:05.863 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:05.859+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:05.935 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:18:05.939 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:05.935+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:05.983 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:05.984 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:05.997 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:18:06.046 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:06.043+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:06.093 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:06.093 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:06.109 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:18:06.109 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:18:06.120 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:06.120 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:06.125 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:06.125 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:06.134 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:18:06.141 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:06.139+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:06.141 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:06.142 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:06.160 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:06.160 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:06.178 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:18:06.178 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:18:06.196 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:06.197 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:06.214 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:06.214 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:06.242 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:06.235+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:06.305 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:06.305 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:06.305 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:06.305 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:06.305 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:06.318 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:18:06.343 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:06.339+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:06.444 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:06.443+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:06.484 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:06.484 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:06.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:06.485 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:06.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.485 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4263.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-18T00:18:06.485 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:55.470436+0000", 2024-09-18T00:18:06.485 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 11.012008284, 2024-09-18T00:18:06.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01376928, 2024-09-18T00:18:06.485 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:06.485 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:06.485 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:06.485 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:06.485 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4263", 2024-09-18T00:18:06.485 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1025497029", 2024-09-18T00:18:06.485 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-18T00:18:06.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.486 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:06.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:06.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.470436+0000", 2024-09-18T00:18:06.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:06.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:06.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.470436+0000", 2024-09-18T00:18:06.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:06.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:06.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.470442+0000", 2024-09-18T00:18:06.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7559999999999996e-06 2024-09-18T00:18:06.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:06.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.470453+0000", 2024-09-18T00:18:06.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.078e-05 2024-09-18T00:18:06.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:06.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.470474+0000", 2024-09-18T00:18:06.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1126999999999999e-05 2024-09-18T00:18:06.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:06.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.470491+0000", 2024-09-18T00:18:06.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7465999999999998e-05 2024-09-18T00:18:06.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:06.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.470674+0000", 2024-09-18T00:18:06.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000182719 2024-09-18T00:18:06.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:18:06.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.470693+0000", 2024-09-18T00:18:06.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9332999999999999e-05 2024-09-18T00:18:06.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:06.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.474770+0000", 2024-09-18T00:18:06.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040765649999999999 2024-09-18T00:18:06.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:18:06.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.474791+0000", 2024-09-18T00:18:06.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1107e-05 2024-09-18T00:18:06.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:06.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.478172+0000", 2024-09-18T00:18:06.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0033807139999999999 2024-09-18T00:18:06.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:06.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.478520+0000", 2024-09-18T00:18:06.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034797700000000002 2024-09-18T00:18:06.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:06.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.479136+0000", 2024-09-18T00:18:06.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061632899999999999 2024-09-18T00:18:06.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:06.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.482703+0000", 2024-09-18T00:18:06.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035675490000000002 2024-09-18T00:18:06.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:06.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.484057+0000", 2024-09-18T00:18:06.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0013534490000000001 2024-09-18T00:18:06.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:06.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.484144+0000", 2024-09-18T00:18:06.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6676999999999997e-05 2024-09-18T00:18:06.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:06.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.484205+0000", 2024-09-18T00:18:06.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1731999999999998e-05 2024-09-18T00:18:06.492 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.492 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:06.492 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.492 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4263.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-18T00:18:06.492 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:55.483882+0000", 2024-09-18T00:18:06.492 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.998562214, 2024-09-18T00:18:06.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033593000000000002, 2024-09-18T00:18:06.493 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:06.493 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:06.493 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:06.493 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:06.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:06.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.483882+0000", 2024-09-18T00:18:06.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:06.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:06.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.483882+0000", 2024-09-18T00:18:06.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:06.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:06.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.483884+0000", 2024-09-18T00:18:06.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5460000000000001e-06 2024-09-18T00:18:06.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:06.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.483894+0000", 2024-09-18T00:18:06.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.006e-05 2024-09-18T00:18:06.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:06.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.483920+0000", 2024-09-18T00:18:06.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5986000000000002e-05 2024-09-18T00:18:06.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:06.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.483935+0000", 2024-09-18T00:18:06.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5840000000000001e-05 2024-09-18T00:18:06.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:06.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.484033+0000", 2024-09-18T00:18:06.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7745999999999996e-05 2024-09-18T00:18:06.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:06.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.484048+0000", 2024-09-18T00:18:06.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4747e-05 2024-09-18T00:18:06.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:06.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.484218+0000", 2024-09-18T00:18:06.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017000500000000001 2024-09-18T00:18:06.496 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.496 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:06.496 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.497 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4263.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-18T00:18:06.497 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:55.488571+0000", 2024-09-18T00:18:06.497 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.993873461, 2024-09-18T00:18:06.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000526159, 2024-09-18T00:18:06.497 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:06.497 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:06.497 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:18:06.497 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:06.497 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4263", 2024-09-18T00:18:06.497 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1025497029", 2024-09-18T00:18:06.497 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2 2024-09-18T00:18:06.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.497 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:06.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:06.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.488571+0000", 2024-09-18T00:18:06.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:06.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:06.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.488571+0000", 2024-09-18T00:18:06.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:06.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:06.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.488573+0000", 2024-09-18T00:18:06.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.396e-06 2024-09-18T00:18:06.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:06.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.488584+0000", 2024-09-18T00:18:06.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.061e-05 2024-09-18T00:18:06.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:06.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.488612+0000", 2024-09-18T00:18:06.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8580000000000001e-05 2024-09-18T00:18:06.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:06.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.488640+0000", 2024-09-18T00:18:06.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7297e-05 2024-09-18T00:18:06.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:06.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.488782+0000", 2024-09-18T00:18:06.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142199 2024-09-18T00:18:06.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:06.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.489097+0000", 2024-09-18T00:18:06.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031507699999999998 2024-09-18T00:18:06.501 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.501 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:06.501 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.501 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4263.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-18T00:18:06.501 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:55.489472+0000", 2024-09-18T00:18:06.501 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.992972405, 2024-09-18T00:18:06.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00038936, 2024-09-18T00:18:06.501 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:06.501 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:06.501 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:18:06.501 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:06.501 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4263", 2024-09-18T00:18:06.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1025497029", 2024-09-18T00:18:06.502 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 3 2024-09-18T00:18:06.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.502 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:06.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:06.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.489472+0000", 2024-09-18T00:18:06.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:06.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:06.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.489472+0000", 2024-09-18T00:18:06.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:06.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:06.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.489474+0000", 2024-09-18T00:18:06.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.767e-06 2024-09-18T00:18:06.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:06.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.489483+0000", 2024-09-18T00:18:06.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9959999999999999e-06 2024-09-18T00:18:06.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:06.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.489504+0000", 2024-09-18T00:18:06.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1257e-05 2024-09-18T00:18:06.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:06.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.489515+0000", 2024-09-18T00:18:06.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.077e-05 2024-09-18T00:18:06.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:06.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.489629+0000", 2024-09-18T00:18:06.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011466199999999999 2024-09-18T00:18:06.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:06.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.489861+0000", 2024-09-18T00:18:06.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023190799999999999 2024-09-18T00:18:06.505 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.505 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:06.505 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.505 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4263.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-18T00:18:06.505 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:56.475935+0000", 2024-09-18T00:18:06.505 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.006509189000001, 2024-09-18T00:18:06.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0088589089999999999, 2024-09-18T00:18:06.505 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:06.505 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:06.505 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:06.506 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:06.506 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4263", 2024-09-18T00:18:06.506 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1025497029", 2024-09-18T00:18:06.506 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5 2024-09-18T00:18:06.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.506 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:06.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:06.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.475935+0000", 2024-09-18T00:18:06.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:06.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:06.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.475935+0000", 2024-09-18T00:18:06.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:06.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:06.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.475938+0000", 2024-09-18T00:18:06.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2899999999999998e-06 2024-09-18T00:18:06.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:06.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.475953+0000", 2024-09-18T00:18:06.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4267000000000001e-05 2024-09-18T00:18:06.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:06.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.475989+0000", 2024-09-18T00:18:06.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999997e-05 2024-09-18T00:18:06.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:06.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.476014+0000", 2024-09-18T00:18:06.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5052999999999999e-05 2024-09-18T00:18:06.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:06.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.476167+0000", 2024-09-18T00:18:06.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000153602 2024-09-18T00:18:06.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:18:06.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.476178+0000", 2024-09-18T00:18:06.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0560000000000001e-05 2024-09-18T00:18:06.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:06.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.481752+0000", 2024-09-18T00:18:06.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055741059999999997 2024-09-18T00:18:06.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:06.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.481969+0000", 2024-09-18T00:18:06.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000216739 2024-09-18T00:18:06.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-18T00:18:06.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.482181+0000", 2024-09-18T00:18:06.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021255199999999999 2024-09-18T00:18:06.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:06.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.483689+0000", 2024-09-18T00:18:06.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0015074509999999999 2024-09-18T00:18:06.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:06.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.484522+0000", 2024-09-18T00:18:06.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083324099999999997 2024-09-18T00:18:06.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:06.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.484678+0000", 2024-09-18T00:18:06.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015571500000000001 2024-09-18T00:18:06.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:06.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.484794+0000", 2024-09-18T00:18:06.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116273 2024-09-18T00:18:06.512 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.512 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:06.512 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.512 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4263.0:5 2.2 e21/19 ondisk, result = 0)", 2024-09-18T00:18:06.512 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:56.484304+0000", 2024-09-18T00:18:06.512 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9981402300000006, 2024-09-18T00:18:06.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00051137000000000003, 2024-09-18T00:18:06.512 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:06.512 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:06.512 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:06.513 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:06.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:06.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.484304+0000", 2024-09-18T00:18:06.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:06.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:06.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.484304+0000", 2024-09-18T00:18:06.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:06.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:06.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.484306+0000", 2024-09-18T00:18:06.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0540000000000002e-06 2024-09-18T00:18:06.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:06.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.484316+0000", 2024-09-18T00:18:06.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0295999999999999e-05 2024-09-18T00:18:06.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:06.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.484345+0000", 2024-09-18T00:18:06.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8949999999999999e-05 2024-09-18T00:18:06.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:06.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.484359+0000", 2024-09-18T00:18:06.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4119e-05 2024-09-18T00:18:06.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:06.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.484477+0000", 2024-09-18T00:18:06.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011802 2024-09-18T00:18:06.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:06.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.484506+0000", 2024-09-18T00:18:06.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8076e-05 2024-09-18T00:18:06.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:06.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.484815+0000", 2024-09-18T00:18:06.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00030985499999999999 2024-09-18T00:18:06.516 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.516 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:06.516 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.516 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4263.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-18T00:18:06.516 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:57.478295+0000", 2024-09-18T00:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.0041496970000008, 2024-09-18T00:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071761079999999996, 2024-09-18T00:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4263", 2024-09-18T00:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1025497029", 2024-09-18T00:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 7 2024-09-18T00:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.478295+0000", 2024-09-18T00:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.478295+0000", 2024-09-18T00:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.478298+0000", 2024-09-18T00:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1470000000000001e-06 2024-09-18T00:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.478313+0000", 2024-09-18T00:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.503e-05 2024-09-18T00:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.478344+0000", 2024-09-18T00:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1099000000000002e-05 2024-09-18T00:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.478385+0000", 2024-09-18T00:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1344000000000003e-05 2024-09-18T00:18:06.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:06.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.478784+0000", 2024-09-18T00:18:06.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039903999999999997 2024-09-18T00:18:06.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:06.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.479047+0000", 2024-09-18T00:18:06.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026309800000000002 2024-09-18T00:18:06.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-18T00:18:06.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.479399+0000", 2024-09-18T00:18:06.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000351904 2024-09-18T00:18:06.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:06.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.481913+0000", 2024-09-18T00:18:06.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0025139070000000001 2024-09-18T00:18:06.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:06.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.485303+0000", 2024-09-18T00:18:06.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003390353 2024-09-18T00:18:06.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:06.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.485399+0000", 2024-09-18T00:18:06.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5408999999999994e-05 2024-09-18T00:18:06.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:06.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.485471+0000", 2024-09-18T00:18:06.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1777000000000001e-05 2024-09-18T00:18:06.523 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.523 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:06.523 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.523 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4263.0:7 2.2 e22/19 ondisk, result = 0)", 2024-09-18T00:18:06.523 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:57.481692+0000", 2024-09-18T00:18:06.523 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.0007520369999998, 2024-09-18T00:18:06.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000233685, 2024-09-18T00:18:06.523 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:06.523 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:06.523 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:06.523 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:06.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:06.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.481692+0000", 2024-09-18T00:18:06.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:06.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:06.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.481692+0000", 2024-09-18T00:18:06.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:06.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.481694+0000", 2024-09-18T00:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2740000000000002e-06 2024-09-18T00:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.481715+0000", 2024-09-18T00:18:06.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0820000000000001e-05 2024-09-18T00:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.481744+0000", 2024-09-18T00:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8326e-05 2024-09-18T00:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.481757+0000", 2024-09-18T00:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3482999999999999e-05 2024-09-18T00:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:06.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.481888+0000", 2024-09-18T00:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013101899999999999 2024-09-18T00:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.481904+0000", 2024-09-18T00:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5903e-05 2024-09-18T00:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:57.481926+0000", 2024-09-18T00:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1860000000000001e-05 2024-09-18T00:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:06.527 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:06.528 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:06.528 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:06.529 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:18:06.545 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:06.543+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:06.548 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:18:06.646 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:06.643+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:06.648 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:06.648 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:06.648 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:18:06.648 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:06.662 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:18:06.746 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:06.743+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:06.780 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:06.780 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:06.780 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:06.780 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:06.780 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:06.794 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:18:06.847 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:06.843+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:06.948 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:06.947+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:06.962 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:06.962 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:06.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:06.962 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [] 2024-09-18T00:18:06.962 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:06.975 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:18:07.049 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:07.043+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:07.095 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:07.096 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:07.096 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:18:07.096 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:07.110 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:18:07.151 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:07.147+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:07.251 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:07.247+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:07.269 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:07.269 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:07.284 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:18:07.352 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:07.347+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:07.394 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is set 2024-09-18T00:18:07.409 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.440942+0000", 2024-09-18T00:18:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.965933961999999, 2024-09-18T00:18:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042441465999999997, 2024-09-18T00:18:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-18T00:18:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.440942+0000", 2024-09-18T00:18:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.440942+0000", 2024-09-18T00:18:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.440945+0000", 2024-09-18T00:18:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5969999999999999e-06 2024-09-18T00:18:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.440958+0000", 2024-09-18T00:18:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3393e-05 2024-09-18T00:18:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.440981+0000", 2024-09-18T00:18:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3054e-05 2024-09-18T00:18:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.441001+0000", 2024-09-18T00:18:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.969e-05 2024-09-18T00:18:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.441125+0000", 2024-09-18T00:18:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012457200000000001 2024-09-18T00:18:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:18:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.441143+0000", 2024-09-18T00:18:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7876999999999998e-05 2024-09-18T00:18:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.477045+0000", 2024-09-18T00:18:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035901699000000002 2024-09-18T00:18:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.477360+0000", 2024-09-18T00:18:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031502400000000001 2024-09-18T00:18:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.477816+0000", 2024-09-18T00:18:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045637 2024-09-18T00:18:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.482239+0000", 2024-09-18T00:18:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044223659999999996 2024-09-18T00:18:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.483228+0000", 2024-09-18T00:18:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00098898600000000008 2024-09-18T00:18:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.483324+0000", 2024-09-18T00:18:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5878999999999995e-05 2024-09-18T00:18:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.483384+0000", 2024-09-18T00:18:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9959e-05 2024-09-18T00:18:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.549389+0000", 2024-09-18T00:18:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.857487099, 2024-09-18T00:18:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014668851, 2024-09-18T00:18:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34 2024-09-18T00:18:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549389+0000", 2024-09-18T00:18:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549389+0000", 2024-09-18T00:18:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549390+0000", 2024-09-18T00:18:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5400000000000001e-07 2024-09-18T00:18:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549395+0000", 2024-09-18T00:18:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5060000000000003e-06 2024-09-18T00:18:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549405+0000", 2024-09-18T00:18:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4399999999999994e-06 2024-09-18T00:18:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549412+0000", 2024-09-18T00:18:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.007e-06 2024-09-18T00:18:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.556213+0000", 2024-09-18T00:18:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068017370000000004 2024-09-18T00:18:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.556465+0000", 2024-09-18T00:18:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025113499999999999 2024-09-18T00:18:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.556837+0000", 2024-09-18T00:18:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037237400000000002 2024-09-18T00:18:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.562324+0000", 2024-09-18T00:18:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054871649999999996 2024-09-18T00:18:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.563789+0000", 2024-09-18T00:18:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0014652210000000001 2024-09-18T00:18:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.563940+0000", 2024-09-18T00:18:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015076299999999999 2024-09-18T00:18:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564058+0000", 2024-09-18T00:18:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000117849 2024-09-18T00:18:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.549430+0000", 2024-09-18T00:18:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.857446335000001, 2024-09-18T00:18:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015014833999999999, 2024-09-18T00:18:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35 2024-09-18T00:18:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549430+0000", 2024-09-18T00:18:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549430+0000", 2024-09-18T00:18:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549431+0000", 2024-09-18T00:18:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0599999999999999e-07 2024-09-18T00:18:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549517+0000", 2024-09-18T00:18:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6159e-05 2024-09-18T00:18:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549530+0000", 2024-09-18T00:18:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3604000000000001e-05 2024-09-18T00:18:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549538+0000", 2024-09-18T00:18:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7959999999999996e-06 2024-09-18T00:18:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.558970+0000", 2024-09-18T00:18:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.009432134 2024-09-18T00:18:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.559209+0000", 2024-09-18T00:18:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023852500000000001 2024-09-18T00:18:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.559572+0000", 2024-09-18T00:18:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00036321999999999998 2024-09-18T00:18:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564262+0000", 2024-09-18T00:18:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046898909999999999 2024-09-18T00:18:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564293+0000", 2024-09-18T00:18:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.074e-05 2024-09-18T00:18:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564373+0000", 2024-09-18T00:18:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9785999999999998e-05 2024-09-18T00:18:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564445+0000", 2024-09-18T00:18:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2072999999999999e-05 2024-09-18T00:18:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.549552+0000", 2024-09-18T00:18:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.857324306000001, 2024-09-18T00:18:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019865803000000001, 2024-09-18T00:18:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 36 2024-09-18T00:18:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549552+0000", 2024-09-18T00:18:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549552+0000", 2024-09-18T00:18:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549553+0000", 2024-09-18T00:18:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4600000000000004e-07 2024-09-18T00:18:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549558+0000", 2024-09-18T00:18:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6099999999999997e-06 2024-09-18T00:18:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549568+0000", 2024-09-18T00:18:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3470000000000005e-06 2024-09-18T00:18:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549576+0000", 2024-09-18T00:18:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0399999999999993e-06 2024-09-18T00:18:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.561348+0000", 2024-09-18T00:18:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011771992 2024-09-18T00:18:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.561491+0000", 2024-09-18T00:18:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014369299999999999 2024-09-18T00:18:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.561712+0000", 2024-09-18T00:18:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000220361 2024-09-18T00:18:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564458+0000", 2024-09-18T00:18:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0027459519999999999 2024-09-18T00:18:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569146+0000", 2024-09-18T00:18:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046883070000000001 2024-09-18T00:18:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569303+0000", 2024-09-18T00:18:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015697199999999999 2024-09-18T00:18:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569418+0000", 2024-09-18T00:18:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000114783 2024-09-18T00:18:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.549587+0000", 2024-09-18T00:18:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.857289366, 2024-09-18T00:18:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.020238056000000001, 2024-09-18T00:18:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 37 2024-09-18T00:18:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549587+0000", 2024-09-18T00:18:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549587+0000", 2024-09-18T00:18:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549587+0000", 2024-09-18T00:18:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3300000000000002e-07 2024-09-18T00:18:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549593+0000", 2024-09-18T00:18:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2700000000000004e-06 2024-09-18T00:18:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549602+0000", 2024-09-18T00:18:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0559999999999994e-06 2024-09-18T00:18:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549609+0000", 2024-09-18T00:18:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1930000000000004e-06 2024-09-18T00:18:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.562442+0000", 2024-09-18T00:18:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012833229 2024-09-18T00:18:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.562723+0000", 2024-09-18T00:18:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028104399999999999 2024-09-18T00:18:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.563094+0000", 2024-09-18T00:18:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037122000000000001 2024-09-18T00:18:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564844+0000", 2024-09-18T00:18:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00174986 2024-09-18T00:18:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569573+0000", 2024-09-18T00:18:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047285829999999997 2024-09-18T00:18:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569727+0000", 2024-09-18T00:18:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000154146 2024-09-18T00:18:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569825+0000", 2024-09-18T00:18:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7822e-05 2024-09-18T00:18:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.677038+0000", 2024-09-18T00:18:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.729838487, 2024-09-18T00:18:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011802852000000001, 2024-09-18T00:18:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 88 2024-09-18T00:18:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677038+0000", 2024-09-18T00:18:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677038+0000", 2024-09-18T00:18:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677039+0000", 2024-09-18T00:18:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.53e-06 2024-09-18T00:18:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677050+0000", 2024-09-18T00:18:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1253000000000001e-05 2024-09-18T00:18:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677070+0000", 2024-09-18T00:18:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9707e-05 2024-09-18T00:18:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677088+0000", 2024-09-18T00:18:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8352999999999999e-05 2024-09-18T00:18:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.685999+0000", 2024-09-18T00:18:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089100550000000001 2024-09-18T00:18:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.686128+0000", 2024-09-18T00:18:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012984199999999999 2024-09-18T00:18:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.686318+0000", 2024-09-18T00:18:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019002900000000001 2024-09-18T00:18:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.687712+0000", 2024-09-18T00:18:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001393799 2024-09-18T00:18:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.688703+0000", 2024-09-18T00:18:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000990875 2024-09-18T00:18:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.688776+0000", 2024-09-18T00:18:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3280000000000003e-05 2024-09-18T00:18:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.688841+0000", 2024-09-18T00:18:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4128999999999995e-05 2024-09-18T00:18:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836576+0000", 2024-09-18T00:18:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.570299758000001, 2024-09-18T00:18:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01473359, 2024-09-18T00:18:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110 2024-09-18T00:18:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836576+0000", 2024-09-18T00:18:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836576+0000", 2024-09-18T00:18:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836577+0000", 2024-09-18T00:18:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4e-07 2024-09-18T00:18:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836586+0000", 2024-09-18T00:18:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3699999999999995e-06 2024-09-18T00:18:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836599+0000", 2024-09-18T00:18:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.343e-05 2024-09-18T00:18:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836608+0000", 2024-09-18T00:18:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2199999999999998e-06 2024-09-18T00:18:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.846689+0000", 2024-09-18T00:18:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010080512 2024-09-18T00:18:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.846824+0000", 2024-09-18T00:18:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000134902 2024-09-18T00:18:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.847020+0000", 2024-09-18T00:18:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019580199999999999 2024-09-18T00:18:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851018+0000", 2024-09-18T00:18:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0039983629999999996 2024-09-18T00:18:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851062+0000", 2024-09-18T00:18:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3769000000000001e-05 2024-09-18T00:18:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851205+0000", 2024-09-18T00:18:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014293199999999999 2024-09-18T00:18:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851310+0000", 2024-09-18T00:18:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010535 2024-09-18T00:18:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836620+0000", 2024-09-18T00:18:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.570256025000001, 2024-09-18T00:18:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016200902, 2024-09-18T00:18:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 111 2024-09-18T00:18:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836620+0000", 2024-09-18T00:18:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836620+0000", 2024-09-18T00:18:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836621+0000", 2024-09-18T00:18:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2699999999999999e-07 2024-09-18T00:18:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836626+0000", 2024-09-18T00:18:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2360000000000004e-06 2024-09-18T00:18:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836636+0000", 2024-09-18T00:18:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6169999999999996e-06 2024-09-18T00:18:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836679+0000", 2024-09-18T00:18:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3415999999999999e-05 2024-09-18T00:18:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.847736+0000", 2024-09-18T00:18:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011057361999999999 2024-09-18T00:18:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.847978+0000", 2024-09-18T00:18:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024185400000000001 2024-09-18T00:18:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.848415+0000", 2024-09-18T00:18:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000436904 2024-09-18T00:18:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851430+0000", 2024-09-18T00:18:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030148369999999998 2024-09-18T00:18:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.852537+0000", 2024-09-18T00:18:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011069910000000001 2024-09-18T00:18:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.852703+0000", 2024-09-18T00:18:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016631600000000001 2024-09-18T00:18:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.852821+0000", 2024-09-18T00:18:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000117642 2024-09-18T00:18:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836714+0000", 2024-09-18T00:18:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.570161962, 2024-09-18T00:18:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017759086, 2024-09-18T00:18:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 112 2024-09-18T00:18:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836714+0000", 2024-09-18T00:18:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836714+0000", 2024-09-18T00:18:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836715+0000", 2024-09-18T00:18:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1030000000000001e-06 2024-09-18T00:18:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836810+0000", 2024-09-18T00:18:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5192000000000006e-05 2024-09-18T00:18:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836839+0000", 2024-09-18T00:18:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8753999999999999e-05 2024-09-18T00:18:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836849+0000", 2024-09-18T00:18:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7860000000000008e-06 2024-09-18T00:18:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.849128+0000", 2024-09-18T00:18:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012278839 2024-09-18T00:18:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.849256+0000", 2024-09-18T00:18:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012836499999999999 2024-09-18T00:18:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.849450+0000", 2024-09-18T00:18:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019406600000000001 2024-09-18T00:18:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.852958+0000", 2024-09-18T00:18:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035073930000000001 2024-09-18T00:18:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854162+0000", 2024-09-18T00:18:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0012046100000000001 2024-09-18T00:18:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854355+0000", 2024-09-18T00:18:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000192449 2024-09-18T00:18:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854473+0000", 2024-09-18T00:18:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000118529 2024-09-18T00:18:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836865+0000", 2024-09-18T00:18:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.570011447000001, 2024-09-18T00:18:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019780444000000001, 2024-09-18T00:18:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 113 2024-09-18T00:18:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836865+0000", 2024-09-18T00:18:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836865+0000", 2024-09-18T00:18:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836866+0000", 2024-09-18T00:18:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7400000000000002e-07 2024-09-18T00:18:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836872+0000", 2024-09-18T00:18:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0260000000000004e-06 2024-09-18T00:18:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836885+0000", 2024-09-18T00:18:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3303e-05 2024-09-18T00:18:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836892+0000", 2024-09-18T00:18:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5700000000000004e-06 2024-09-18T00:18:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851546+0000", 2024-09-18T00:18:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014653497999999999 2024-09-18T00:18:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851760+0000", 2024-09-18T00:18:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021415400000000001 2024-09-18T00:18:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851954+0000", 2024-09-18T00:18:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019398199999999999 2024-09-18T00:18:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856341+0000", 2024-09-18T00:18:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0043870489999999996 2024-09-18T00:18:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856388+0000", 2024-09-18T00:18:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6919999999999998e-05 2024-09-18T00:18:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856530+0000", 2024-09-18T00:18:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142249 2024-09-18T00:18:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856645+0000", 2024-09-18T00:18:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000114819 2024-09-18T00:18:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836911+0000", 2024-09-18T00:18:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.569964736999999, 2024-09-18T00:18:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.021208742999999999, 2024-09-18T00:18:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 114 2024-09-18T00:18:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836911+0000", 2024-09-18T00:18:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836911+0000", 2024-09-18T00:18:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836912+0000", 2024-09-18T00:18:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5600000000000005e-07 2024-09-18T00:18:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836919+0000", 2024-09-18T00:18:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4400000000000002e-06 2024-09-18T00:18:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836928+0000", 2024-09-18T00:18:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.91e-06 2024-09-18T00:18:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836935+0000", 2024-09-18T00:18:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0829999999999998e-06 2024-09-18T00:18:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.853068+0000", 2024-09-18T00:18:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016132082999999998 2024-09-18T00:18:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.853344+0000", 2024-09-18T00:18:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027593799999999999 2024-09-18T00:18:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.853571+0000", 2024-09-18T00:18:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022752800000000001 2024-09-18T00:18:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856732+0000", 2024-09-18T00:18:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031610779999999999 2024-09-18T00:18:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.857856+0000", 2024-09-18T00:18:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001123752 2024-09-18T00:18:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.857998+0000", 2024-09-18T00:18:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142292 2024-09-18T00:18:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.858120+0000", 2024-09-18T00:18:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121983 2024-09-18T00:18:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836969+0000", 2024-09-18T00:18:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.569907018, 2024-09-18T00:18:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.022609931999999999, 2024-09-18T00:18:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 115 2024-09-18T00:18:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836969+0000", 2024-09-18T00:18:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836969+0000", 2024-09-18T00:18:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836970+0000", 2024-09-18T00:18:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1399999999999996e-07 2024-09-18T00:18:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837068+0000", 2024-09-18T00:18:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7868999999999994e-05 2024-09-18T00:18:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837095+0000", 2024-09-18T00:18:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7793e-05 2024-09-18T00:18:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837106+0000", 2024-09-18T00:18:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.028e-05 2024-09-18T00:18:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854556+0000", 2024-09-18T00:18:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017449742000000001 2024-09-18T00:18:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854690+0000", 2024-09-18T00:18:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013440500000000001 2024-09-18T00:18:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854900+0000", 2024-09-18T00:18:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000210306 2024-09-18T00:18:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.858224+0000", 2024-09-18T00:18:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0033242010000000002 2024-09-18T00:18:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859316+0000", 2024-09-18T00:18:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010916109999999999 2024-09-18T00:18:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859450+0000", 2024-09-18T00:18:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013410900000000001 2024-09-18T00:18:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859579+0000", 2024-09-18T00:18:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012890199999999999 2024-09-18T00:18:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.837122+0000", 2024-09-18T00:18:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.569754115, 2024-09-18T00:18:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.022693474000000002, 2024-09-18T00:18:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116 2024-09-18T00:18:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837122+0000", 2024-09-18T00:18:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837122+0000", 2024-09-18T00:18:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837123+0000", 2024-09-18T00:18:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.33e-07 2024-09-18T00:18:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837131+0000", 2024-09-18T00:18:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5029999999999997e-06 2024-09-18T00:18:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837142+0000", 2024-09-18T00:18:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0329999999999999e-05 2024-09-18T00:18:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837149+0000", 2024-09-18T00:18:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.99e-06 2024-09-18T00:18:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856845+0000", 2024-09-18T00:18:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019696074000000001 2024-09-18T00:18:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.857107+0000", 2024-09-18T00:18:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026233799999999998 2024-09-18T00:18:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.857298+0000", 2024-09-18T00:18:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019087600000000001 2024-09-18T00:18:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859604+0000", 2024-09-18T00:18:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023056980000000001 2024-09-18T00:18:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859667+0000", 2024-09-18T00:18:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3696999999999997e-05 2024-09-18T00:18:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859749+0000", 2024-09-18T00:18:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1526000000000005e-05 2024-09-18T00:18:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859816+0000", 2024-09-18T00:18:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6508999999999996e-05 2024-09-18T00:18:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.837160+0000", 2024-09-18T00:18:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.569716043, 2024-09-18T00:18:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023190008000000002, 2024-09-18T00:18:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 117 2024-09-18T00:18:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837160+0000", 2024-09-18T00:18:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837160+0000", 2024-09-18T00:18:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837161+0000", 2024-09-18T00:18:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2399999999999998e-07 2024-09-18T00:18:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837166+0000", 2024-09-18T00:18:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.716e-06 2024-09-18T00:18:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837179+0000", 2024-09-18T00:18:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2063e-05 2024-09-18T00:18:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837186+0000", 2024-09-18T00:18:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1040000000000001e-06 2024-09-18T00:18:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.858312+0000", 2024-09-18T00:18:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.021126147000000001 2024-09-18T00:18:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.858553+0000", 2024-09-18T00:18:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024152099999999999 2024-09-18T00:18:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.858880+0000", 2024-09-18T00:18:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000326461 2024-09-18T00:18:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859939+0000", 2024-09-18T00:18:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001059735 2024-09-18T00:18:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.860212+0000", 2024-09-18T00:18:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027284400000000001 2024-09-18T00:18:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.860292+0000", 2024-09-18T00:18:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9983999999999999e-05 2024-09-18T00:18:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.860350+0000", 2024-09-18T00:18:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7809000000000002e-05 2024-09-18T00:18:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.317506+0000", 2024-09-18T00:18:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.089369883, 2024-09-18T00:18:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011471278999999999, 2024-09-18T00:18:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 135 2024-09-18T00:18:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317506+0000", 2024-09-18T00:18:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317506+0000", 2024-09-18T00:18:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317507+0000", 2024-09-18T00:18:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7000000000000003e-07 2024-09-18T00:18:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317513+0000", 2024-09-18T00:18:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5160000000000002e-06 2024-09-18T00:18:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317522+0000", 2024-09-18T00:18:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2599999999999994e-06 2024-09-18T00:18:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317529+0000", 2024-09-18T00:18:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1570000000000003e-06 2024-09-18T00:18:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320156+0000", 2024-09-18T00:18:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00262737 2024-09-18T00:18:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320285+0000", 2024-09-18T00:18:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012859199999999999 2024-09-18T00:18:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320398+0000", 2024-09-18T00:18:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112553 2024-09-18T00:18:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.324003+0000", 2024-09-18T00:18:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036049300000000001 2024-09-18T00:18:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.328923+0000", 2024-09-18T00:18:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049205899999999999 2024-09-18T00:18:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.328952+0000", 2024-09-18T00:18:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8858999999999999e-05 2024-09-18T00:18:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.328978+0000", 2024-09-18T00:18:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5582e-05 2024-09-18T00:18:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.317540+0000", 2024-09-18T00:18:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.089336533000001, 2024-09-18T00:18:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011513031999999999, 2024-09-18T00:18:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 136 2024-09-18T00:18:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317540+0000", 2024-09-18T00:18:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317540+0000", 2024-09-18T00:18:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317540+0000", 2024-09-18T00:18:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9999999999999997e-07 2024-09-18T00:18:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317545+0000", 2024-09-18T00:18:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1030000000000001e-06 2024-09-18T00:18:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317554+0000", 2024-09-18T00:18:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9430000000000006e-06 2024-09-18T00:18:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317561+0000", 2024-09-18T00:18:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0299999999999996e-06 2024-09-18T00:18:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320658+0000", 2024-09-18T00:18:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030962950000000002 2024-09-18T00:18:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320712+0000", 2024-09-18T00:18:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4159000000000001e-05 2024-09-18T00:18:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320791+0000", 2024-09-18T00:18:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.962e-05 2024-09-18T00:18:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.324006+0000", 2024-09-18T00:18:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032141320000000002 2024-09-18T00:18:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329007+0000", 2024-09-18T00:18:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050010899999999997 2024-09-18T00:18:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329031+0000", 2024-09-18T00:18:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4493000000000001e-05 2024-09-18T00:18:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329053+0000", 2024-09-18T00:18:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1467e-05 2024-09-18T00:18:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.317592+0000", 2024-09-18T00:18:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.089284227, 2024-09-18T00:18:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011529286, 2024-09-18T00:18:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 137 2024-09-18T00:18:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317592+0000", 2024-09-18T00:18:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317592+0000", 2024-09-18T00:18:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317593+0000", 2024-09-18T00:18:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7000000000000003e-07 2024-09-18T00:18:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317750+0000", 2024-09-18T00:18:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015739199999999999 2024-09-18T00:18:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317764+0000", 2024-09-18T00:18:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3845999999999999e-05 2024-09-18T00:18:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317840+0000", 2024-09-18T00:18:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6253000000000003e-05 2024-09-18T00:18:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.321078+0000", 2024-09-18T00:18:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032379560000000002 2024-09-18T00:18:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.321125+0000", 2024-09-18T00:18:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6585999999999997e-05 2024-09-18T00:18:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.321195+0000", 2024-09-18T00:18:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0188e-05 2024-09-18T00:18:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.324011+0000", 2024-09-18T00:18:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0028159410000000002 2024-09-18T00:18:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329075+0000", 2024-09-18T00:18:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050643379999999998 2024-09-18T00:18:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329103+0000", 2024-09-18T00:18:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7616000000000001e-05 2024-09-18T00:18:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329121+0000", 2024-09-18T00:18:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8199999999999999e-05 2024-09-18T00:18:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.317909+0000", 2024-09-18T00:18:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.08896683, 2024-09-18T00:18:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011404755000000001, 2024-09-18T00:18:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 139 2024-09-18T00:18:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317909+0000", 2024-09-18T00:18:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317909+0000", 2024-09-18T00:18:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317910+0000", 2024-09-18T00:18:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2699999999999999e-07 2024-09-18T00:18:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317916+0000", 2024-09-18T00:18:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5230000000000003e-06 2024-09-18T00:18:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317924+0000", 2024-09-18T00:18:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9239999999999996e-06 2024-09-18T00:18:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317932+0000", 2024-09-18T00:18:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0700000000000001e-06 2024-09-18T00:18:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322402+0000", 2024-09-18T00:18:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044705420000000001 2024-09-18T00:18:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322452+0000", 2024-09-18T00:18:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0297999999999999e-05 2024-09-18T00:18:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322533+0000", 2024-09-18T00:18:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1092000000000002e-05 2024-09-18T00:18:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.324722+0000", 2024-09-18T00:18:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0021886010000000001 2024-09-18T00:18:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329227+0000", 2024-09-18T00:18:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045046419999999997 2024-09-18T00:18:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329264+0000", 2024-09-18T00:18:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7577999999999998e-05 2024-09-18T00:18:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329314+0000", 2024-09-18T00:18:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9758e-05 2024-09-18T00:18:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.317962+0000", 2024-09-18T00:18:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.088913746999999, 2024-09-18T00:18:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01213116, 2024-09-18T00:18:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 140 2024-09-18T00:18:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317962+0000", 2024-09-18T00:18:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317962+0000", 2024-09-18T00:18:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317963+0000", 2024-09-18T00:18:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8700000000000004e-07 2024-09-18T00:18:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318086+0000", 2024-09-18T00:18:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000122862 2024-09-18T00:18:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318099+0000", 2024-09-18T00:18:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2459999999999999e-05 2024-09-18T00:18:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318107+0000", 2024-09-18T00:18:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-18T00:18:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322800+0000", 2024-09-18T00:18:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046924649999999998 2024-09-18T00:18:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322849+0000", 2024-09-18T00:18:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9406000000000001e-05 2024-09-18T00:18:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322927+0000", 2024-09-18T00:18:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7763999999999996e-05 2024-09-18T00:18:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.324725+0000", 2024-09-18T00:18:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0017981060000000001 2024-09-18T00:18:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329976+0000", 2024-09-18T00:18:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052507109999999999 2024-09-18T00:18:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.330045+0000", 2024-09-18T00:18:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8998999999999994e-05 2024-09-18T00:18:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.330094+0000", 2024-09-18T00:18:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.905e-05 2024-09-18T00:18:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.318122+0000", 2024-09-18T00:18:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.088754492, 2024-09-18T00:18:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012634492000000001, 2024-09-18T00:18:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 141 2024-09-18T00:18:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318122+0000", 2024-09-18T00:18:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318122+0000", 2024-09-18T00:18:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318122+0000", 2024-09-18T00:18:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9400000000000005e-07 2024-09-18T00:18:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318128+0000", 2024-09-18T00:18:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3800000000000002e-06 2024-09-18T00:18:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318136+0000", 2024-09-18T00:18:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7760000000000003e-06 2024-09-18T00:18:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318143+0000", 2024-09-18T00:18:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-18T00:18:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.323651+0000", 2024-09-18T00:18:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055075269999999999 2024-09-18T00:18:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.323700+0000", 2024-09-18T00:18:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8724999999999999e-05 2024-09-18T00:18:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.323772+0000", 2024-09-18T00:18:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2378000000000004e-05 2024-09-18T00:18:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.325433+0000", 2024-09-18T00:18:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001661278 2024-09-18T00:18:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.330670+0000", 2024-09-18T00:18:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052365689999999999 2024-09-18T00:18:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.330737+0000", 2024-09-18T00:18:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6866999999999995e-05 2024-09-18T00:18:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.330756+0000", 2024-09-18T00:18:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9313999999999998e-05 2024-09-18T00:18:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:07.549 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:07.550 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:07.471+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:07.550 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:18:07.596 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:07.595+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:07.673 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:07.673 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:07.673 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:07.673 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:07.677 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:07.675+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:07.687 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:18:07.778 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:07.771+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:07.846 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:07.846 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:07.846 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:07.847 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:07.847 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:07.859 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:18:07.905 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:07.903+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:07.974 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:07.975 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:07.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:07.975 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:07.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.975 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:34 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'20, mlcod=17'14)", 2024-09-18T00:18:07.975 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:51.558207+0000", 2024-09-18T00:18:07.975 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 16.413841291000001, 2024-09-18T00:18:07.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0046458009999999998, 2024-09-18T00:18:07.975 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:07.975 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:07.975 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.975 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:07.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:07.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.558207+0000", 2024-09-18T00:18:07.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:07.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:07.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.558207+0000", 2024-09-18T00:18:07.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:07.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:07.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.558209+0000", 2024-09-18T00:18:07.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3769999999999999e-06 2024-09-18T00:18:07.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:07.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.558230+0000", 2024-09-18T00:18:07.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1336e-05 2024-09-18T00:18:07.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:07.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.558249+0000", 2024-09-18T00:18:07.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.84e-05 2024-09-18T00:18:07.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.558259+0000", 2024-09-18T00:18:07.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0720000000000001e-05 2024-09-18T00:18:07.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:07.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.558428+0000", 2024-09-18T00:18:07.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000168519 2024-09-18T00:18:07.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:07.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.558457+0000", 2024-09-18T00:18:07.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9456000000000001e-05 2024-09-18T00:18:07.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:07.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.562721+0000", 2024-09-18T00:18:07.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042641739999999999 2024-09-18T00:18:07.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:07.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.562853+0000", 2024-09-18T00:18:07.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013181900000000001 2024-09-18T00:18:07.979 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:07.979 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:07.979 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:07.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.980 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:36 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'22, mlcod=17'18)", 2024-09-18T00:18:07.980 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:51.563151+0000", 2024-09-18T00:18:07.980 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 16.408897432, 2024-09-18T00:18:07.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067203280000000002, 2024-09-18T00:18:07.980 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:07.980 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:07.980 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.980 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:07.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:07.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563151+0000", 2024-09-18T00:18:07.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:07.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:07.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563151+0000", 2024-09-18T00:18:07.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:07.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:07.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563153+0000", 2024-09-18T00:18:07.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.42e-06 2024-09-18T00:18:07.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:07.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563163+0000", 2024-09-18T00:18:07.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.077e-05 2024-09-18T00:18:07.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:07.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563191+0000", 2024-09-18T00:18:07.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.724e-05 2024-09-18T00:18:07.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563204+0000", 2024-09-18T00:18:07.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3266e-05 2024-09-18T00:18:07.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:07.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563319+0000", 2024-09-18T00:18:07.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115343 2024-09-18T00:18:07.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:07.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563371+0000", 2024-09-18T00:18:07.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1922000000000002e-05 2024-09-18T00:18:07.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:07.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.569760+0000", 2024-09-18T00:18:07.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388721 2024-09-18T00:18:07.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:07.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.569872+0000", 2024-09-18T00:18:07.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011164599999999999 2024-09-18T00:18:07.984 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:07.984 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:07.984 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:07.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.984 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:37 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'23, mlcod=17'18)", 2024-09-18T00:18:07.984 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:51.564518+0000", 2024-09-18T00:18:07.984 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 16.407530366, 2024-09-18T00:18:07.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005870994, 2024-09-18T00:18:07.985 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:07.985 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:07.985 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.985 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:07.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:07.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564518+0000", 2024-09-18T00:18:07.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:07.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:07.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564518+0000", 2024-09-18T00:18:07.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:07.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:07.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564520+0000", 2024-09-18T00:18:07.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8300000000000001e-06 2024-09-18T00:18:07.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:07.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564532+0000", 2024-09-18T00:18:07.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1754e-05 2024-09-18T00:18:07.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:07.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564560+0000", 2024-09-18T00:18:07.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8486000000000001e-05 2024-09-18T00:18:07.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564573+0000", 2024-09-18T00:18:07.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3069e-05 2024-09-18T00:18:07.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:07.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564660+0000", 2024-09-18T00:18:07.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6263999999999999e-05 2024-09-18T00:18:07.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:07.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564695+0000", 2024-09-18T00:18:07.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5296000000000002e-05 2024-09-18T00:18:07.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:07.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.570284+0000", 2024-09-18T00:18:07.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055890030000000004 2024-09-18T00:18:07.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:07.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.570389+0000", 2024-09-18T00:18:07.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010529199999999999 2024-09-18T00:18:07.988 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:07.988 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:07.988 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:07.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.989 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:58 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'36, mlcod=17'33)", 2024-09-18T00:18:07.989 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:51.614679+0000", 2024-09-18T00:18:07.989 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 16.357370166999999, 2024-09-18T00:18:07.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043511340000000004, 2024-09-18T00:18:07.989 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:07.989 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:07.989 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.989 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:07.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:07.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.614679+0000", 2024-09-18T00:18:07.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:07.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:07.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.614679+0000", 2024-09-18T00:18:07.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:07.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:07.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.614680+0000", 2024-09-18T00:18:07.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.863e-06 2024-09-18T00:18:07.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:07.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.614797+0000", 2024-09-18T00:18:07.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011681299999999999 2024-09-18T00:18:07.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:07.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.614825+0000", 2024-09-18T00:18:07.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8263e-05 2024-09-18T00:18:07.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.614838+0000", 2024-09-18T00:18:07.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3013000000000001e-05 2024-09-18T00:18:07.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:07.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.614980+0000", 2024-09-18T00:18:07.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014132899999999999 2024-09-18T00:18:07.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:07.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.615019+0000", 2024-09-18T00:18:07.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9360000000000003e-05 2024-09-18T00:18:07.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:07.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.618957+0000", 2024-09-18T00:18:07.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039382729999999999 2024-09-18T00:18:07.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:07.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.619030+0000", 2024-09-18T00:18:07.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2219999999999996e-05 2024-09-18T00:18:07.993 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:07.993 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:07.993 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:07.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.993 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:130 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'88, mlcod=17'85)", 2024-09-18T00:18:07.993 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.319467+0000", 2024-09-18T00:18:07.994 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.652581509000001, 2024-09-18T00:18:07.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047039869999999998, 2024-09-18T00:18:07.994 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:07.994 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:07.994 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.994 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:07.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:07.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319467+0000", 2024-09-18T00:18:07.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:07.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:07.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319467+0000", 2024-09-18T00:18:07.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:07.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:07.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319469+0000", 2024-09-18T00:18:07.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0229999999999999e-06 2024-09-18T00:18:07.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:07.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319512+0000", 2024-09-18T00:18:07.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3232999999999998e-05 2024-09-18T00:18:07.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:07.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319542+0000", 2024-09-18T00:18:07.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0046e-05 2024-09-18T00:18:07.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:07.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319564+0000", 2024-09-18T00:18:07.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1489999999999999e-05 2024-09-18T00:18:07.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:07.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319691+0000", 2024-09-18T00:18:07.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012706600000000001 2024-09-18T00:18:07.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:07.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319726+0000", 2024-09-18T00:18:07.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5206000000000001e-05 2024-09-18T00:18:07.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:07.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324096+0000", 2024-09-18T00:18:07.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043692959999999999 2024-09-18T00:18:07.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:07.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324171+0000", 2024-09-18T00:18:07.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5626999999999999e-05 2024-09-18T00:18:07.998 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:07.998 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:07.998 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:07.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:07.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.998 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:131 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'89, mlcod=17'85)", 2024-09-18T00:18:07.998 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.319608+0000", 2024-09-18T00:18:07.999 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.652440277, 2024-09-18T00:18:07.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0058413620000000001, 2024-09-18T00:18:07.999 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:07.999 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:07.999 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:07.999 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:07.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:07.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:08.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319608+0000", 2024-09-18T00:18:08.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:08.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319608+0000", 2024-09-18T00:18:08.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:08.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319609+0000", 2024-09-18T00:18:08.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3e-07 2024-09-18T00:18:08.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:08.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319676+0000", 2024-09-18T00:18:08.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6889000000000003e-05 2024-09-18T00:18:08.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:08.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319694+0000", 2024-09-18T00:18:08.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7703e-05 2024-09-18T00:18:08.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:08.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319706+0000", 2024-09-18T00:18:08.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2577e-05 2024-09-18T00:18:08.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:08.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320272+0000", 2024-09-18T00:18:08.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056591599999999997 2024-09-18T00:18:08.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:08.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320306+0000", 2024-09-18T00:18:08.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4116000000000003e-05 2024-09-18T00:18:08.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:08.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325386+0000", 2024-09-18T00:18:08.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050793349999999999 2024-09-18T00:18:08.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:08.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325450+0000", 2024-09-18T00:18:08.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4196000000000007e-05 2024-09-18T00:18:08.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.003 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:08.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.003 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:132 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'90, mlcod=17'85)", 2024-09-18T00:18:08.004 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.320067+0000", 2024-09-18T00:18:08.004 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.651981744, 2024-09-18T00:18:08.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0054358590000000004, 2024-09-18T00:18:08.004 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:08.004 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:08.004 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:08.004 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:08.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:08.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320067+0000", 2024-09-18T00:18:08.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:08.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320067+0000", 2024-09-18T00:18:08.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:08.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320068+0000", 2024-09-18T00:18:08.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0100000000000001e-06 2024-09-18T00:18:08.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:08.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320079+0000", 2024-09-18T00:18:08.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1146e-05 2024-09-18T00:18:08.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:08.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320101+0000", 2024-09-18T00:18:08.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1994e-05 2024-09-18T00:18:08.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:08.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320110+0000", 2024-09-18T00:18:08.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:18:08.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:08.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321623+0000", 2024-09-18T00:18:08.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001512981 2024-09-18T00:18:08.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:08.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321656+0000", 2024-09-18T00:18:08.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2539999999999997e-05 2024-09-18T00:18:08.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:08.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325455+0000", 2024-09-18T00:18:08.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0037992880000000001 2024-09-18T00:18:08.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:08.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325503+0000", 2024-09-18T00:18:08.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7567e-05 2024-09-18T00:18:08.008 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.008 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:08.008 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.008 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:133 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'91, mlcod=17'85)", 2024-09-18T00:18:08.008 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.320439+0000", 2024-09-18T00:18:08.008 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.651609601000001, 2024-09-18T00:18:08.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005115858, 2024-09-18T00:18:08.008 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:08.008 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:08.008 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:08.008 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:08.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:08.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320439+0000", 2024-09-18T00:18:08.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:08.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320439+0000", 2024-09-18T00:18:08.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:08.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320440+0000", 2024-09-18T00:18:08.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2999999999999999e-07 2024-09-18T00:18:08.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:08.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320454+0000", 2024-09-18T00:18:08.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4107000000000001e-05 2024-09-18T00:18:08.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:08.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320472+0000", 2024-09-18T00:18:08.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7499999999999998e-05 2024-09-18T00:18:08.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:08.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320480+0000", 2024-09-18T00:18:08.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0970000000000006e-06 2024-09-18T00:18:08.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:08.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322261+0000", 2024-09-18T00:18:08.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001780946 2024-09-18T00:18:08.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:08.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322297+0000", 2024-09-18T00:18:08.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6510000000000001e-05 2024-09-18T00:18:08.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:08.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325507+0000", 2024-09-18T00:18:08.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0032097919999999999 2024-09-18T00:18:08.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:08.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325555+0000", 2024-09-18T00:18:08.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8075999999999998e-05 2024-09-18T00:18:08.012 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.012 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:08.012 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.012 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:134 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'92, mlcod=17'85)", 2024-09-18T00:18:08.012 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.320847+0000", 2024-09-18T00:18:08.012 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.65120187, 2024-09-18T00:18:08.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0068502509999999999, 2024-09-18T00:18:08.013 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:08.013 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:08.013 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:08.013 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:08.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:08.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320847+0000", 2024-09-18T00:18:08.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:08.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320847+0000", 2024-09-18T00:18:08.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:08.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320848+0000", 2024-09-18T00:18:08.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-18T00:18:08.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:08.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320971+0000", 2024-09-18T00:18:08.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012338599999999999 2024-09-18T00:18:08.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:08.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320990+0000", 2024-09-18T00:18:08.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9009999999999999e-05 2024-09-18T00:18:08.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:08.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321000+0000", 2024-09-18T00:18:08.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9399999999999997e-06 2024-09-18T00:18:08.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:08.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322866+0000", 2024-09-18T00:18:08.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001866089 2024-09-18T00:18:08.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:08.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322902+0000", 2024-09-18T00:18:08.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5312999999999999e-05 2024-09-18T00:18:08.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:08.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327608+0000", 2024-09-18T00:18:08.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047063000000000001 2024-09-18T00:18:08.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:08.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327697+0000", 2024-09-18T00:18:08.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9049999999999996e-05 2024-09-18T00:18:08.016 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.016 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:08.016 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.017 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:135 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'93, mlcod=17'85)", 2024-09-18T00:18:08.017 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.321689+0000", 2024-09-18T00:18:08.017 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.65035922, 2024-09-18T00:18:08.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0082046600000000008, 2024-09-18T00:18:08.017 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:08.017 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:08.017 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:08.017 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:08.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:08.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321689+0000", 2024-09-18T00:18:08.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:08.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321689+0000", 2024-09-18T00:18:08.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:08.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321691+0000", 2024-09-18T00:18:08.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0229999999999999e-06 2024-09-18T00:18:08.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:08.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321706+0000", 2024-09-18T00:18:08.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4513e-05 2024-09-18T00:18:08.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:08.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321727+0000", 2024-09-18T00:18:08.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0939999999999999e-05 2024-09-18T00:18:08.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:08.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321745+0000", 2024-09-18T00:18:08.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7986999999999999e-05 2024-09-18T00:18:08.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:08.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324259+0000", 2024-09-18T00:18:08.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025143069999999999 2024-09-18T00:18:08.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:08.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324291+0000", 2024-09-18T00:18:08.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2116999999999998e-05 2024-09-18T00:18:08.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:08.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329824+0000", 2024-09-18T00:18:08.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055326300000000002 2024-09-18T00:18:08.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:08.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329894+0000", 2024-09-18T00:18:08.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0142999999999996e-05 2024-09-18T00:18:08.021 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.021 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:08.022 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.022 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:136 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'94, mlcod=17'85)", 2024-09-18T00:18:08.022 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.322041+0000", 2024-09-18T00:18:08.022 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.650008033000001, 2024-09-18T00:18:08.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0079092060000000002, 2024-09-18T00:18:08.022 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:08.022 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:08.022 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:08.022 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:08.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:08.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322041+0000", 2024-09-18T00:18:08.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:08.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322041+0000", 2024-09-18T00:18:08.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:08.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322042+0000", 2024-09-18T00:18:08.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.22e-06 2024-09-18T00:18:08.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:08.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322053+0000", 2024-09-18T00:18:08.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1584e-05 2024-09-18T00:18:08.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:08.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322073+0000", 2024-09-18T00:18:08.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0023000000000001e-05 2024-09-18T00:18:08.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:08.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322092+0000", 2024-09-18T00:18:08.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8063e-05 2024-09-18T00:18:08.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:08.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324848+0000", 2024-09-18T00:18:08.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027561420000000001 2024-09-18T00:18:08.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:08.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324877+0000", 2024-09-18T00:18:08.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9003e-05 2024-09-18T00:18:08.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:08.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329900+0000", 2024-09-18T00:18:08.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050233650000000001 2024-09-18T00:18:08.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:08.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329950+0000", 2024-09-18T00:18:08.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9805999999999997e-05 2024-09-18T00:18:08.026 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.026 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:08.026 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.026 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:137 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'95, mlcod=17'85)", 2024-09-18T00:18:08.026 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.322465+0000", 2024-09-18T00:18:08.026 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.649583329, 2024-09-18T00:18:08.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0075526680000000002, 2024-09-18T00:18:08.026 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:08.026 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:08.027 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:08.027 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:08.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:08.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322465+0000", 2024-09-18T00:18:08.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:08.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322465+0000", 2024-09-18T00:18:08.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:08.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322467+0000", 2024-09-18T00:18:08.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5999999999999999e-06 2024-09-18T00:18:08.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:08.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322612+0000", 2024-09-18T00:18:08.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014553499999999999 2024-09-18T00:18:08.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:08.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322631+0000", 2024-09-18T00:18:08.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8627e-05 2024-09-18T00:18:08.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:08.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322645+0000", 2024-09-18T00:18:08.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4109e-05 2024-09-18T00:18:08.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:08.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325646+0000", 2024-09-18T00:18:08.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0030007179999999999 2024-09-18T00:18:08.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:08.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325674+0000", 2024-09-18T00:18:08.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7733e-05 2024-09-18T00:18:08.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:08.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329958+0000", 2024-09-18T00:18:08.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042841329999999999 2024-09-18T00:18:08.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:08.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330018+0000", 2024-09-18T00:18:08.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0213000000000001e-05 2024-09-18T00:18:08.030 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.030 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:08.030 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.030 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:138 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'96, mlcod=17'85)", 2024-09-18T00:18:08.031 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.323237+0000", 2024-09-18T00:18:08.031 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.648812182, 2024-09-18T00:18:08.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0068313540000000004, 2024-09-18T00:18:08.031 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:08.031 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:08.031 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:08.031 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:08.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:08.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323237+0000", 2024-09-18T00:18:08.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:08.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323237+0000", 2024-09-18T00:18:08.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:08.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323238+0000", 2024-09-18T00:18:08.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.127e-06 2024-09-18T00:18:08.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:08.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323249+0000", 2024-09-18T00:18:08.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1157e-05 2024-09-18T00:18:08.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:08.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323277+0000", 2024-09-18T00:18:08.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8178999999999999e-05 2024-09-18T00:18:08.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:08.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323287+0000", 2024-09-18T00:18:08.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9699999999999994e-06 2024-09-18T00:18:08.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:08.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327148+0000", 2024-09-18T00:18:08.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038609109999999999 2024-09-18T00:18:08.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:08.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327178+0000", 2024-09-18T00:18:08.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0156000000000001e-05 2024-09-18T00:18:08.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:08.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330022+0000", 2024-09-18T00:18:08.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028444579999999998 2024-09-18T00:18:08.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:08.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330068+0000", 2024-09-18T00:18:08.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5395999999999997e-05 2024-09-18T00:18:08.035 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.035 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:08.035 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.035 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:139 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'86)", 2024-09-18T00:18:08.035 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.323792+0000", 2024-09-18T00:18:08.035 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.648256533, 2024-09-18T00:18:08.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063243850000000001, 2024-09-18T00:18:08.035 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:08.035 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:08.035 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:08.035 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:08.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:08.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323792+0000", 2024-09-18T00:18:08.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:08.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323792+0000", 2024-09-18T00:18:08.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:08.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323793+0000", 2024-09-18T00:18:08.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4399999999999998e-07 2024-09-18T00:18:08.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:08.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323806+0000", 2024-09-18T00:18:08.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3346000000000001e-05 2024-09-18T00:18:08.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:08.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323824+0000", 2024-09-18T00:18:08.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.789e-05 2024-09-18T00:18:08.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:08.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323836+0000", 2024-09-18T00:18:08.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2082999999999999e-05 2024-09-18T00:18:08.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:08.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327773+0000", 2024-09-18T00:18:08.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039365099999999998 2024-09-18T00:18:08.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:08.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327802+0000", 2024-09-18T00:18:08.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8617000000000001e-05 2024-09-18T00:18:08.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:08.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330072+0000", 2024-09-18T00:18:08.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022704219999999998 2024-09-18T00:18:08.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:08.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330117+0000", 2024-09-18T00:18:08.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4573000000000002e-05 2024-09-18T00:18:08.039 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.039 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:08.039 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.039 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:140 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'86)", 2024-09-18T00:18:08.039 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.324263+0000", 2024-09-18T00:18:08.039 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.647786079999999, 2024-09-18T00:18:08.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0066665350000000003, 2024-09-18T00:18:08.039 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:08.040 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:08.040 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:08.040 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:08.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:08.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324263+0000", 2024-09-18T00:18:08.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:08.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324263+0000", 2024-09-18T00:18:08.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:08.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324264+0000", 2024-09-18T00:18:08.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-18T00:18:08.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:08.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324387+0000", 2024-09-18T00:18:08.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012346899999999999 2024-09-18T00:18:08.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:08.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324402+0000", 2024-09-18T00:18:08.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5087e-05 2024-09-18T00:18:08.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324412+0000", 2024-09-18T00:18:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5969999999999998e-06 2024-09-18T00:18:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.328364+0000", 2024-09-18T00:18:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039523559999999998 2024-09-18T00:18:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.328397+0000", 2024-09-18T00:18:08.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2733e-05 2024-09-18T00:18:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330864+0000", 2024-09-18T00:18:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0024676670000000002 2024-09-18T00:18:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330929+0000", 2024-09-18T00:18:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4665999999999994e-05 2024-09-18T00:18:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:141 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'86)", 2024-09-18T00:18:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.325030+0000", 2024-09-18T00:18:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.647019102, 2024-09-18T00:18:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006603462, 2024-09-18T00:18:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325030+0000", 2024-09-18T00:18:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325030+0000", 2024-09-18T00:18:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325031+0000", 2024-09-18T00:18:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.53e-06 2024-09-18T00:18:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325042+0000", 2024-09-18T00:18:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0506000000000001e-05 2024-09-18T00:18:08.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325080+0000", 2024-09-18T00:18:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8300000000000003e-05 2024-09-18T00:18:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325098+0000", 2024-09-18T00:18:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8539999999999999e-05 2024-09-18T00:18:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330164+0000", 2024-09-18T00:18:08.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050653709999999999 2024-09-18T00:18:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330194+0000", 2024-09-18T00:18:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0275999999999999e-05 2024-09-18T00:18:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.331562+0000", 2024-09-18T00:18:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0013680330000000001 2024-09-18T00:18:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.331633+0000", 2024-09-18T00:18:08.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0906000000000002e-05 2024-09-18T00:18:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:142 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'100, mlcod=17'98)", 2024-09-18T00:18:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.332949+0000", 2024-09-18T00:18:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.639099407, 2024-09-18T00:18:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059501479999999997, 2024-09-18T00:18:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:08.048 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.332949+0000", 2024-09-18T00:18:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.332949+0000", 2024-09-18T00:18:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:08.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.332951+0000", 2024-09-18T00:18:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-06 2024-09-18T00:18:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333024+0000", 2024-09-18T00:18:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3112000000000001e-05 2024-09-18T00:18:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333050+0000", 2024-09-18T00:18:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5673e-05 2024-09-18T00:18:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333063+0000", 2024-09-18T00:18:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2644e-05 2024-09-18T00:18:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333228+0000", 2024-09-18T00:18:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016579799999999999 2024-09-18T00:18:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333267+0000", 2024-09-18T00:18:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8189999999999999e-05 2024-09-18T00:18:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338834+0000", 2024-09-18T00:18:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055676679999999996 2024-09-18T00:18:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338899+0000", 2024-09-18T00:18:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5198999999999997e-05 2024-09-18T00:18:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:143 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'101, mlcod=17'98)", 2024-09-18T00:18:08.052 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.333556+0000", 2024-09-18T00:18:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.638492352, 2024-09-18T00:18:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005386053, 2024-09-18T00:18:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333556+0000", 2024-09-18T00:18:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:08.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333556+0000", 2024-09-18T00:18:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333558+0000", 2024-09-18T00:18:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:18:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333677+0000", 2024-09-18T00:18:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119196 2024-09-18T00:18:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333698+0000", 2024-09-18T00:18:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1087e-05 2024-09-18T00:18:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333708+0000", 2024-09-18T00:18:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0125999999999999e-05 2024-09-18T00:18:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334689+0000", 2024-09-18T00:18:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00098056899999999993 2024-09-18T00:18:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334719+0000", 2024-09-18T00:18:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9963000000000001e-05 2024-09-18T00:18:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338907+0000", 2024-09-18T00:18:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041881619999999996 2024-09-18T00:18:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338942+0000", 2024-09-18T00:18:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5660000000000001e-05 2024-09-18T00:18:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:144 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'102, mlcod=17'98)", 2024-09-18T00:18:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.334453+0000", 2024-09-18T00:18:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.637595924999999, 2024-09-18T00:18:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045227619999999996, 2024-09-18T00:18:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:08.057 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334453+0000", 2024-09-18T00:18:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334453+0000", 2024-09-18T00:18:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:08.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334454+0000", 2024-09-18T00:18:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6759999999999999e-06 2024-09-18T00:18:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334507+0000", 2024-09-18T00:18:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2726999999999998e-05 2024-09-18T00:18:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334536+0000", 2024-09-18T00:18:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8385999999999999e-05 2024-09-18T00:18:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:08.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334549+0000", 2024-09-18T00:18:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3237e-05 2024-09-18T00:18:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.335984+0000", 2024-09-18T00:18:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001435609 2024-09-18T00:18:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.336013+0000", 2024-09-18T00:18:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8822999999999998e-05 2024-09-18T00:18:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338944+0000", 2024-09-18T00:18:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029308839999999999 2024-09-18T00:18:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338976+0000", 2024-09-18T00:18:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1420000000000001e-05 2024-09-18T00:18:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T00:18:08.061 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:07.405976+0000", 2024-09-18T00:18:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.56607305100000005, 2024-09-18T00:18:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063813710000000003, 2024-09-18T00:18:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11 2024-09-18T00:18:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.405976+0000", 2024-09-18T00:18:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.405976+0000", 2024-09-18T00:18:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.405979+0000", 2024-09-18T00:18:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9009999999999998e-06 2024-09-18T00:18:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.405998+0000", 2024-09-18T00:18:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9653e-05 2024-09-18T00:18:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.406027+0000", 2024-09-18T00:18:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8442e-05 2024-09-18T00:18:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.406041+0000", 2024-09-18T00:18:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4273e-05 2024-09-18T00:18:08.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.406141+0000", 2024-09-18T00:18:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010008199999999999 2024-09-18T00:18:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:18:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.406147+0000", 2024-09-18T00:18:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9340000000000003e-06 2024-09-18T00:18:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.408986+0000", 2024-09-18T00:18:08.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028392959999999998 2024-09-18T00:18:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.409279+0000", 2024-09-18T00:18:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000292596 2024-09-18T00:18:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.409615+0000", 2024-09-18T00:18:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033606 2024-09-18T00:18:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.411318+0000", 2024-09-18T00:18:08.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017026890000000001 2024-09-18T00:18:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.412269+0000", 2024-09-18T00:18:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00095177200000000003 2024-09-18T00:18:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.412319+0000", 2024-09-18T00:18:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9975999999999997e-05 2024-09-18T00:18:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:08.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.412357+0000", 2024-09-18T00:18:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7697000000000002e-05 2024-09-18T00:18:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:08.068 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:08.068 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:08.070 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:18:08.070 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:08.067+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:08.170 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:08.167+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:08.227 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:08.227 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:08.227 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:08.227 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:08.233 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:18:08.241 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:18:08.285 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:08.283+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:08.372 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:08.367+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:08.375 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:08.393 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:18:08.393 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:18:08.409 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:08.409 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:08.411 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:08.411 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:08.423 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:18:08.426 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:08.426 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:08.446 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:08.447 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:08.467 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:18:08.467 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:18:08.473 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:08.471+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:08.488 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:08.488 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:08.508 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:08.508 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:08.536 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:08.536 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:08.549 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:18:08.592 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:08.591+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:08.675 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:08.671+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:08.701 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:08.702 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:08.714 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:18:08.786 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:18:08.787 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:18:08.787 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:18:08.787 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 7 2024-09-18T00:18:08.805 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:08.803+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:08.876 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:08.871+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:08.880 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:08.880 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:08.893 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:18:08.977 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:08.971+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:09.046 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:09.046 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:09.059 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:18:09.078 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:09.075+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:09.094 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-18T00:18:09.091+0000 7f63a6107640 -1 mon.a@0(leader).osd e26 definitely_dead 0 2024-09-18T00:18:09.176 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:09.177 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:09.179 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:09.175+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:09.191 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:18:09.280 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:09.275+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:09.312 INFO:teuthology.orchestra.run.smithi097.stdout:skipping zero size read 2024-09-18T00:18:09.332 INFO:teuthology.orchestra.run.smithi097.stdout:3 write 0x4eb7e68 thru 0x4eba0f7 (0x2290 bytes) 2024-09-18T00:18:09.347 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:09.347 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:09.355 INFO:teuthology.orchestra.run.smithi097.stdout:4 writesame 0x5b425f8 thru 0x5b4b4fb data_size 0x51(0x8f04 bytes) 2024-09-18T00:18:09.355 INFO:teuthology.orchestra.run.smithi097.stdout:7 writesame 0x212cba3 thru 0x2136c8f data_size 0x1(0xa0ed bytes) 2024-09-18T00:18:09.360 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:18:09.381 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:09.375+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:09.477 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:18:09.487 INFO:teuthology.orchestra.run.smithi097.stdout:8 write 0xc827010 thru 0xc82bf31 (0x4f22 bytes) 2024-09-18T00:18:09.488 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:09.483+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:09.492 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:09.492 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:09.496 INFO:teuthology.orchestra.run.smithi097.stdout:9 writesame 0xc294303 thru 0xc294962 data_size 0x20(0x660 bytes) 2024-09-18T00:18:09.503 INFO:teuthology.orchestra.run.smithi097.stdout:10 read 0x893259 thru 0x8a17d8 (0xe580 bytes) 2024-09-18T00:18:09.503 INFO:teuthology.orchestra.run.smithi097.stdout:11 read 0xa507a8c thru 0xa508f3c (0x14b1 bytes) 2024-09-18T00:18:09.504 INFO:teuthology.orchestra.run.smithi097.stdout:12 read 0x673a47c thru 0x6743592 (0x9117 bytes) 2024-09-18T00:18:09.504 INFO:teuthology.orchestra.run.smithi097.stdout:13 write 0x14e50cc thru 0x14f1b8a (0xcabf bytes) 2024-09-18T00:18:09.506 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:18:09.512 INFO:teuthology.orchestra.run.smithi097.stdout:14 read 0x3bd97bd thru 0x3bdc1fd (0x2a41 bytes) 2024-09-18T00:18:09.512 INFO:teuthology.orchestra.run.smithi097.stdout:15 write 0x94029ee thru 0x940eccc (0xc2df bytes) 2024-09-18T00:18:09.531 INFO:teuthology.orchestra.run.smithi097.stderr:marked out osd.7. 2024-09-18T00:18:09.596 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:09.595+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:09.638 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:09.651 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:18:09.698 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:09.695+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:09.775 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:09.775 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:09.785 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:09.779+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:09.790 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:18:09.886 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:09.883+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:09.955 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:09.955 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:09.969 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:18:09.987 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:09.983+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:10.088 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:10.083+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:10.091 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:10.091 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:10.105 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:18:10.189 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:10.187+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:10.270 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:10.270 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:10.284 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:18:10.330 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:10.327+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:10.384 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is unset 2024-09-18T00:18:10.390 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:10.387+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:10.414 INFO:teuthology.orchestra.run.smithi097.stdout:17 write 0xe8747ec thru 0xe876ddf (0x25f4 bytes) 2024-09-18T00:18:10.420 INFO:teuthology.orchestra.run.smithi097.stdout:18 read 0x284936c thru 0x284dfd0 (0x4c65 bytes) 2024-09-18T00:18:10.421 INFO:teuthology.orchestra.run.smithi097.stdout:21 punch from 0xe03f929 to 0xe04426a, (0x4941 bytes) 2024-09-18T00:18:10.428 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:10.428 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:10.430 INFO:teuthology.orchestra.run.smithi097.stdout:22 write 0xed4de90 thru 0xed4f104 (0x1275 bytes) 2024-09-18T00:18:10.435 INFO:teuthology.orchestra.run.smithi097.stdout:23 write 0xdff7506 thru 0xe000952 (0x944d bytes) 2024-09-18T00:18:10.444 INFO:teuthology.orchestra.run.smithi097.stdout:25 read 0xa664522 thru 0xa6742e1 (0xfdc0 bytes) 2024-09-18T00:18:10.444 INFO:teuthology.orchestra.run.smithi097.stdout:26 writesame 0x62ab487 thru 0x62b800b data_size 0x15(0xcb85 bytes) 2024-09-18T00:18:10.447 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:18:10.453 INFO:teuthology.orchestra.run.smithi097.stdout:27 read 0x63f80d7 thru 0x640441e (0xc348 bytes) 2024-09-18T00:18:10.453 INFO:teuthology.orchestra.run.smithi097.stdout:28 write 0xb296f1a thru 0xb29a4e5 (0x35cc bytes) 2024-09-18T00:18:10.459 INFO:teuthology.orchestra.run.smithi097.stdout:29 read 0x56241e6 thru 0x5631236 (0xd051 bytes) 2024-09-18T00:18:10.459 INFO:teuthology.orchestra.run.smithi097.stdout:30 read 0x55b9c6d thru 0x55bfbfa (0x5f8e bytes) 2024-09-18T00:18:10.459 INFO:teuthology.orchestra.run.smithi097.stdout:31 read 0xd739465 thru 0xd7430bc (0x9c58 bytes) 2024-09-18T00:18:10.459 INFO:teuthology.orchestra.run.smithi097.stdout:32 punch from 0x89b9769 to 0x89b9eb8, (0x74f bytes) 2024-09-18T00:18:10.459 INFO:teuthology.orchestra.run.smithi097.stdout:33 writesame 0xe8d26c4 thru 0xe8d9f34 data_size 0x1(0x7871 bytes) 2024-09-18T00:18:10.490 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:10.487+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:10.525 INFO:teuthology.orchestra.run.smithi097.stdout:truncating to largest ever: 0xe51dfd 2024-09-18T00:18:10.526 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:18:10.560 INFO:teuthology.orchestra.run.smithi097.stdout:34 trunc from 0xed4f105 to 0xe51dfd 2024-09-18T00:18:10.585 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:10.585 INFO:teuthology.orchestra.run.smithi097.stdout:36 punch from 0xccbed1 to 0xccc954, (0xa83 bytes) 2024-09-18T00:18:10.591 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:10.587+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:10.610 INFO:teuthology.orchestra.run.smithi097.stdout:38 write 0x9288dc7 thru 0x9295159 (0xc393 bytes) 2024-09-18T00:18:10.613 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:10.614 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:10.617 INFO:teuthology.orchestra.run.smithi097.stdout:39 punch from 0x6801b61 to 0x6809764, (0x7c03 bytes) 2024-09-18T00:18:10.617 INFO:teuthology.orchestra.run.smithi097.stdout:40 read 0x865c52d thru 0x8662190 (0x5c64 bytes) 2024-09-18T00:18:10.617 INFO:teuthology.orchestra.run.smithi097.stdout:43 read 0x65f5f60 thru 0x6603ded (0xde8e bytes) 2024-09-18T00:18:10.627 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:18:10.632 INFO:teuthology.orchestra.run.smithi097.stdout:44 trunc from 0x929515a to 0xdb34a 2024-09-18T00:18:10.676 INFO:teuthology.orchestra.run.smithi097.stdout:45 write 0xabd40f0 thru 0xabd41e3 (0xf4 bytes) 2024-09-18T00:18:10.680 INFO:teuthology.orchestra.run.smithi097.stdout:46 write 0x8f4dcdf thru 0x8f4e9de (0xd00 bytes) 2024-09-18T00:18:10.683 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:10.684 INFO:teuthology.orchestra.run.smithi097.stdout:50 write 0x41ff20d thru 0x4200b7b (0x196f bytes) 2024-09-18T00:18:10.687 INFO:teuthology.orchestra.run.smithi097.stdout:53 write 0x259ef18 thru 0x25ac1e8 (0xd2d1 bytes) 2024-09-18T00:18:10.690 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:10.692 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:10.687+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:10.708 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:18:10.709 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:18:10.709 INFO:teuthology.orchestra.run.smithi097.stdout:56 trunc from 0xabd41e4 to 0x3813fa 2024-09-18T00:18:10.727 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:10.727 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:10.728 INFO:teuthology.orchestra.run.smithi097.stdout:57 read 0x12d642 thru 0x12e1ed (0xbac bytes) 2024-09-18T00:18:10.728 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:10.728 INFO:teuthology.orchestra.run.smithi097.stdout:truncating to largest ever: 0xe21de30 2024-09-18T00:18:10.745 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:10.748 INFO:teuthology.orchestra.run.smithi097.stdout:59 trunc from 0x3813fa to 0xe21de30 2024-09-18T00:18:10.755 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:10.756 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:10.757 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:10.757 INFO:teuthology.orchestra.run.smithi097.stdout:61 punch from 0x8124924 to 0x8130426, (0xbb02 bytes) 2024-09-18T00:18:10.757 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:10.757 INFO:teuthology.orchestra.run.smithi097.stdout:63 writesame 0xc4f1d2f thru 0xc4fe48c data_size 0x1a(0xc75e bytes) 2024-09-18T00:18:10.766 INFO:teuthology.orchestra.run.smithi097.stdout:64 write 0x21c6c2a thru 0x21cc65b (0x5a32 bytes) 2024-09-18T00:18:10.768 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:10.771 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:18:10.775 INFO:teuthology.orchestra.run.smithi097.stdout:65 write 0x51c0915 thru 0x51cc1b2 (0xb89e bytes) 2024-09-18T00:18:10.784 INFO:teuthology.orchestra.run.smithi097.stdout:66 read 0xc6367e5 thru 0xc643f73 (0xd78f bytes) 2024-09-18T00:18:10.785 INFO:teuthology.orchestra.run.smithi097.stdout:67 read 0xb153858 thru 0xb15744b (0x3bf4 bytes) 2024-09-18T00:18:10.785 INFO:teuthology.orchestra.run.smithi097.stdout:68 write 0xa4f45b5 thru 0xa4f7252 (0x2c9e bytes) 2024-09-18T00:18:10.786 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:18:10.786 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:18:10.788 INFO:teuthology.orchestra.run.smithi097.stdout:69 write 0x9e3416c thru 0x9e37a71 (0x3906 bytes) 2024-09-18T00:18:10.792 INFO:teuthology.orchestra.run.smithi097.stdout:70 write 0xdd55868 thru 0xdd60561 (0xacfa bytes) 2024-09-18T00:18:10.793 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:10.791+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:10.803 INFO:teuthology.orchestra.run.smithi097.stdout:71 read 0x71dc8e0 thru 0x71ea014 (0xd735 bytes) 2024-09-18T00:18:10.804 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:10.805 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:10.814 INFO:teuthology.orchestra.run.smithi097.stdout:74 trunc from 0xe21de30 to 0x67d091c 2024-09-18T00:18:10.825 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:10.826 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:10.830 INFO:teuthology.orchestra.run.smithi097.stdout:75 punch from 0x594f9f2 to 0x595a818, (0xae26 bytes) 2024-09-18T00:18:10.830 INFO:teuthology.orchestra.run.smithi097.stdout:77 punch from 0x4008948 to 0x4009d30, (0x13e8 bytes) 2024-09-18T00:18:10.831 INFO:teuthology.orchestra.run.smithi097.stdout:78 trunc from 0x67d091c to 0x62fb5fb 2024-09-18T00:18:10.838 INFO:teuthology.orchestra.run.smithi097.stdout:79 punch from 0x5ea30a1 to 0x5eb0b06, (0xda65 bytes) 2024-09-18T00:18:10.838 INFO:teuthology.orchestra.run.smithi097.stdout:81 punch from 0x15dfe92 to 0x15e9218, (0x9386 bytes) 2024-09-18T00:18:10.838 INFO:teuthology.orchestra.run.smithi097.stdout:82 read 0x1fbcc47 thru 0x1fc7c83 (0xb03d bytes) 2024-09-18T00:18:10.846 INFO:teuthology.orchestra.run.smithi097.stdout:83 trunc from 0x62fb5fb to 0xa4caa93 2024-09-18T00:18:10.861 INFO:teuthology.orchestra.run.smithi097.stdout:84 trunc from 0xa4caa93 to 0x65a20e9 2024-09-18T00:18:10.868 INFO:teuthology.orchestra.run.smithi097.stdout:85 read 0x2ce6fb6 thru 0x2cef232 (0x827d bytes) 2024-09-18T00:18:10.869 INFO:teuthology.orchestra.run.smithi097.stdout:86 read 0x23f548f thru 0x24010d9 (0xbc4b bytes) 2024-09-18T00:18:10.881 INFO:teuthology.orchestra.run.smithi097.stdout:87 trunc from 0x65a20e9 to 0xc35045d 2024-09-18T00:18:10.888 INFO:teuthology.orchestra.run.smithi097.stdout:89 write 0x7373812 thru 0x737f315 (0xbb04 bytes) 2024-09-18T00:18:10.894 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:10.887+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:10.918 INFO:teuthology.orchestra.run.smithi097.stdout:90 trunc from 0xc35045d to 0x249a17d 2024-09-18T00:18:10.940 INFO:teuthology.orchestra.run.smithi097.stdout:92 punch from 0x15012a to 0x1573dc, (0x72b2 bytes) 2024-09-18T00:18:10.941 INFO:teuthology.orchestra.run.smithi097.stdout:93 read 0x7ed6ef thru 0x7ef2c0 (0x1bd2 bytes) 2024-09-18T00:18:10.945 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:10.945 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:10.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:10.945 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [] 2024-09-18T00:18:10.945 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:94 trunc from 0x249a17d to 0x67c2be6 2024-09-18T00:18:10.959 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:18:10.963 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:10.963 INFO:teuthology.orchestra.run.smithi097.stdout:96 read 0x1efd75b thru 0x1f08f98 (0xb83e bytes) 2024-09-18T00:18:10.963 INFO:teuthology.orchestra.run.smithi097.stdout:97 punch from 0x31fc566 to 0x320804d, (0xbae7 bytes) 2024-09-18T00:18:10.963 INFO:teuthology.orchestra.run.smithi097.stdout:truncating to largest ever: 0xe3506f4 2024-09-18T00:18:10.981 INFO:teuthology.orchestra.run.smithi097.stdout:98 trunc from 0x67c2be6 to 0xe3506f4 2024-09-18T00:18:10.991 INFO:teuthology.orchestra.run.smithi097.stdout:99 read 0xc057932 thru 0xc06549f (0xdb6e bytes) 2024-09-18T00:18:10.991 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:10.991 INFO:teuthology.orchestra.run.smithi097.stdout:101 read 0xaf6220f thru 0xaf668a8 (0x469a bytes) 2024-09-18T00:18:10.992 INFO:teuthology.orchestra.run.smithi097.stdout:103 read 0x6b6c420 thru 0x6b74fc1 (0x8ba2 bytes) 2024-09-18T00:18:10.992 INFO:teuthology.orchestra.run.smithi097.stdout:104 punch from 0xc3e532e to 0xc3ee3fc, (0x90ce bytes) 2024-09-18T00:18:10.992 INFO:teuthology.orchestra.run.smithi097.stdout:105 punch from 0x4d6ecf5 to 0x4d7eb74, (0xfe7f bytes) 2024-09-18T00:18:10.992 INFO:teuthology.orchestra.run.smithi097.stdout:106 writesame 0x4808e90 thru 0x480b3d3 data_size 0xa(0x2544 bytes) 2024-09-18T00:18:10.995 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:10.991+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:10.998 INFO:teuthology.orchestra.run.smithi097.stdout:109 read 0xbfc6803 thru 0xbfce9b8 (0x81b6 bytes) 2024-09-18T00:18:10.998 INFO:teuthology.orchestra.run.smithi097.stdout:110 write 0x12b564a thru 0x12b7801 (0x21b8 bytes) 2024-09-18T00:18:11.004 INFO:teuthology.orchestra.run.smithi097.stdout:111 punch from 0x631fb84 to 0x632928d, (0x9709 bytes) 2024-09-18T00:18:11.028 INFO:teuthology.orchestra.run.smithi097.stdout:112 trunc from 0xe3506f4 to 0x261e3bd 2024-09-18T00:18:11.043 INFO:teuthology.orchestra.run.smithi097.stdout:113 clone 1 order 19 su 262144 sc 3 2024-09-18T00:18:11.096 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:11.091+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:11.126 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:11.126 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:11.127 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:18:11.127 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:11.141 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:18:11.196 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:11.195+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:11.298 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:11.295+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:11.303 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:11.304 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:11.318 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:18:11.398 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:11.395+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:11.449 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:11.450 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:11.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:11.450 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:11.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.450 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.450 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.440942+0000", 2024-09-18T00:18:11.450 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.0047493, 2024-09-18T00:18:11.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042441465999999997, 2024-09-18T00:18:11.450 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.450 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.450 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.450 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.451 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.451 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.451 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-18T00:18:11.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.451 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.440942+0000", 2024-09-18T00:18:11.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.440942+0000", 2024-09-18T00:18:11.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.440945+0000", 2024-09-18T00:18:11.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5969999999999999e-06 2024-09-18T00:18:11.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.440958+0000", 2024-09-18T00:18:11.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3393e-05 2024-09-18T00:18:11.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.440981+0000", 2024-09-18T00:18:11.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3054e-05 2024-09-18T00:18:11.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.441001+0000", 2024-09-18T00:18:11.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.969e-05 2024-09-18T00:18:11.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.441125+0000", 2024-09-18T00:18:11.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012457200000000001 2024-09-18T00:18:11.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:18:11.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.441143+0000", 2024-09-18T00:18:11.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7876999999999998e-05 2024-09-18T00:18:11.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.477045+0000", 2024-09-18T00:18:11.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035901699000000002 2024-09-18T00:18:11.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.477360+0000", 2024-09-18T00:18:11.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031502400000000001 2024-09-18T00:18:11.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.477816+0000", 2024-09-18T00:18:11.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045637 2024-09-18T00:18:11.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.482239+0000", 2024-09-18T00:18:11.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044223659999999996 2024-09-18T00:18:11.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.483228+0000", 2024-09-18T00:18:11.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00098898600000000008 2024-09-18T00:18:11.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.483324+0000", 2024-09-18T00:18:11.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5878999999999995e-05 2024-09-18T00:18:11.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.483384+0000", 2024-09-18T00:18:11.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9959e-05 2024-09-18T00:18:11.456 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.456 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.456 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.456 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.457 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.549389+0000", 2024-09-18T00:18:11.457 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.896302436999999, 2024-09-18T00:18:11.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014668851, 2024-09-18T00:18:11.457 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.457 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.457 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.457 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.457 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.457 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.457 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34 2024-09-18T00:18:11.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.457 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549389+0000", 2024-09-18T00:18:11.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549389+0000", 2024-09-18T00:18:11.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549390+0000", 2024-09-18T00:18:11.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5400000000000001e-07 2024-09-18T00:18:11.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549395+0000", 2024-09-18T00:18:11.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5060000000000003e-06 2024-09-18T00:18:11.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549405+0000", 2024-09-18T00:18:11.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4399999999999994e-06 2024-09-18T00:18:11.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549412+0000", 2024-09-18T00:18:11.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.007e-06 2024-09-18T00:18:11.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.556213+0000", 2024-09-18T00:18:11.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068017370000000004 2024-09-18T00:18:11.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.556465+0000", 2024-09-18T00:18:11.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025113499999999999 2024-09-18T00:18:11.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.556837+0000", 2024-09-18T00:18:11.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037237400000000002 2024-09-18T00:18:11.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.562324+0000", 2024-09-18T00:18:11.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054871649999999996 2024-09-18T00:18:11.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.563789+0000", 2024-09-18T00:18:11.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0014652210000000001 2024-09-18T00:18:11.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.563940+0000", 2024-09-18T00:18:11.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015076299999999999 2024-09-18T00:18:11.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564058+0000", 2024-09-18T00:18:11.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000117849 2024-09-18T00:18:11.462 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.462 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.462 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.462 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.463 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.549430+0000", 2024-09-18T00:18:11.463 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.896261673000001, 2024-09-18T00:18:11.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015014833999999999, 2024-09-18T00:18:11.463 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.463 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.463 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.463 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.463 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.463 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.463 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35 2024-09-18T00:18:11.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.463 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549430+0000", 2024-09-18T00:18:11.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549430+0000", 2024-09-18T00:18:11.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549431+0000", 2024-09-18T00:18:11.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0599999999999999e-07 2024-09-18T00:18:11.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549517+0000", 2024-09-18T00:18:11.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6159e-05 2024-09-18T00:18:11.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549530+0000", 2024-09-18T00:18:11.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3604000000000001e-05 2024-09-18T00:18:11.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549538+0000", 2024-09-18T00:18:11.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7959999999999996e-06 2024-09-18T00:18:11.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.558970+0000", 2024-09-18T00:18:11.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.009432134 2024-09-18T00:18:11.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.559209+0000", 2024-09-18T00:18:11.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023852500000000001 2024-09-18T00:18:11.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.559572+0000", 2024-09-18T00:18:11.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00036321999999999998 2024-09-18T00:18:11.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564262+0000", 2024-09-18T00:18:11.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046898909999999999 2024-09-18T00:18:11.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564293+0000", 2024-09-18T00:18:11.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.074e-05 2024-09-18T00:18:11.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564373+0000", 2024-09-18T00:18:11.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9785999999999998e-05 2024-09-18T00:18:11.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564445+0000", 2024-09-18T00:18:11.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2072999999999999e-05 2024-09-18T00:18:11.468 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.468 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.468 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.468 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.468 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.549552+0000", 2024-09-18T00:18:11.468 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.896139644000002, 2024-09-18T00:18:11.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019865803000000001, 2024-09-18T00:18:11.469 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.469 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.469 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.469 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.469 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.469 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.469 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 36 2024-09-18T00:18:11.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.469 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549552+0000", 2024-09-18T00:18:11.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549552+0000", 2024-09-18T00:18:11.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549553+0000", 2024-09-18T00:18:11.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4600000000000004e-07 2024-09-18T00:18:11.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549558+0000", 2024-09-18T00:18:11.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6099999999999997e-06 2024-09-18T00:18:11.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549568+0000", 2024-09-18T00:18:11.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3470000000000005e-06 2024-09-18T00:18:11.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549576+0000", 2024-09-18T00:18:11.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0399999999999993e-06 2024-09-18T00:18:11.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.561348+0000", 2024-09-18T00:18:11.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011771992 2024-09-18T00:18:11.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.561491+0000", 2024-09-18T00:18:11.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014369299999999999 2024-09-18T00:18:11.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.561712+0000", 2024-09-18T00:18:11.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000220361 2024-09-18T00:18:11.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564458+0000", 2024-09-18T00:18:11.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0027459519999999999 2024-09-18T00:18:11.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569146+0000", 2024-09-18T00:18:11.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046883070000000001 2024-09-18T00:18:11.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569303+0000", 2024-09-18T00:18:11.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015697199999999999 2024-09-18T00:18:11.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569418+0000", 2024-09-18T00:18:11.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000114783 2024-09-18T00:18:11.474 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.474 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.474 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.475 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.475 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.549587+0000", 2024-09-18T00:18:11.475 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.896104703999999, 2024-09-18T00:18:11.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.020238056000000001, 2024-09-18T00:18:11.475 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.475 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.475 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.475 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.475 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.475 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.475 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 37 2024-09-18T00:18:11.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.475 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549587+0000", 2024-09-18T00:18:11.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549587+0000", 2024-09-18T00:18:11.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549587+0000", 2024-09-18T00:18:11.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3300000000000002e-07 2024-09-18T00:18:11.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549593+0000", 2024-09-18T00:18:11.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2700000000000004e-06 2024-09-18T00:18:11.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549602+0000", 2024-09-18T00:18:11.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0559999999999994e-06 2024-09-18T00:18:11.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549609+0000", 2024-09-18T00:18:11.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1930000000000004e-06 2024-09-18T00:18:11.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.562442+0000", 2024-09-18T00:18:11.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012833229 2024-09-18T00:18:11.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.562723+0000", 2024-09-18T00:18:11.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028104399999999999 2024-09-18T00:18:11.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.563094+0000", 2024-09-18T00:18:11.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037122000000000001 2024-09-18T00:18:11.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564844+0000", 2024-09-18T00:18:11.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00174986 2024-09-18T00:18:11.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569573+0000", 2024-09-18T00:18:11.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047285829999999997 2024-09-18T00:18:11.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569727+0000", 2024-09-18T00:18:11.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000154146 2024-09-18T00:18:11.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569825+0000", 2024-09-18T00:18:11.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7822e-05 2024-09-18T00:18:11.480 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.480 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.481 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.481 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.481 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.677038+0000", 2024-09-18T00:18:11.481 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.768653825000001, 2024-09-18T00:18:11.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011802852000000001, 2024-09-18T00:18:11.481 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.481 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.481 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.481 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.481 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.481 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.481 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 88 2024-09-18T00:18:11.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.482 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677038+0000", 2024-09-18T00:18:11.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677038+0000", 2024-09-18T00:18:11.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677039+0000", 2024-09-18T00:18:11.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.53e-06 2024-09-18T00:18:11.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677050+0000", 2024-09-18T00:18:11.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1253000000000001e-05 2024-09-18T00:18:11.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677070+0000", 2024-09-18T00:18:11.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9707e-05 2024-09-18T00:18:11.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677088+0000", 2024-09-18T00:18:11.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8352999999999999e-05 2024-09-18T00:18:11.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.685999+0000", 2024-09-18T00:18:11.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089100550000000001 2024-09-18T00:18:11.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.686128+0000", 2024-09-18T00:18:11.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012984199999999999 2024-09-18T00:18:11.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.686318+0000", 2024-09-18T00:18:11.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019002900000000001 2024-09-18T00:18:11.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.687712+0000", 2024-09-18T00:18:11.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001393799 2024-09-18T00:18:11.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.688703+0000", 2024-09-18T00:18:11.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000990875 2024-09-18T00:18:11.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.688776+0000", 2024-09-18T00:18:11.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3280000000000003e-05 2024-09-18T00:18:11.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.688841+0000", 2024-09-18T00:18:11.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4128999999999995e-05 2024-09-18T00:18:11.486 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.486 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.486 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.487 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.487 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836576+0000", 2024-09-18T00:18:11.487 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.609115096, 2024-09-18T00:18:11.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01473359, 2024-09-18T00:18:11.487 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.487 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.487 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.487 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.487 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.487 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.487 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110 2024-09-18T00:18:11.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.488 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836576+0000", 2024-09-18T00:18:11.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836576+0000", 2024-09-18T00:18:11.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836577+0000", 2024-09-18T00:18:11.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4e-07 2024-09-18T00:18:11.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836586+0000", 2024-09-18T00:18:11.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3699999999999995e-06 2024-09-18T00:18:11.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836599+0000", 2024-09-18T00:18:11.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.343e-05 2024-09-18T00:18:11.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836608+0000", 2024-09-18T00:18:11.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2199999999999998e-06 2024-09-18T00:18:11.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.846689+0000", 2024-09-18T00:18:11.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010080512 2024-09-18T00:18:11.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.846824+0000", 2024-09-18T00:18:11.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000134902 2024-09-18T00:18:11.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.847020+0000", 2024-09-18T00:18:11.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019580199999999999 2024-09-18T00:18:11.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851018+0000", 2024-09-18T00:18:11.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0039983629999999996 2024-09-18T00:18:11.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851062+0000", 2024-09-18T00:18:11.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3769000000000001e-05 2024-09-18T00:18:11.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851205+0000", 2024-09-18T00:18:11.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014293199999999999 2024-09-18T00:18:11.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851310+0000", 2024-09-18T00:18:11.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010535 2024-09-18T00:18:11.492 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.492 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.492 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.493 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.493 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836620+0000", 2024-09-18T00:18:11.493 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.609071362999998, 2024-09-18T00:18:11.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016200902, 2024-09-18T00:18:11.493 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.493 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.493 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.493 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.493 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.493 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.493 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 111 2024-09-18T00:18:11.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.494 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836620+0000", 2024-09-18T00:18:11.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836620+0000", 2024-09-18T00:18:11.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836621+0000", 2024-09-18T00:18:11.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2699999999999999e-07 2024-09-18T00:18:11.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836626+0000", 2024-09-18T00:18:11.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2360000000000004e-06 2024-09-18T00:18:11.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836636+0000", 2024-09-18T00:18:11.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6169999999999996e-06 2024-09-18T00:18:11.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836679+0000", 2024-09-18T00:18:11.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3415999999999999e-05 2024-09-18T00:18:11.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.847736+0000", 2024-09-18T00:18:11.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011057361999999999 2024-09-18T00:18:11.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.847978+0000", 2024-09-18T00:18:11.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024185400000000001 2024-09-18T00:18:11.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.848415+0000", 2024-09-18T00:18:11.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000436904 2024-09-18T00:18:11.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851430+0000", 2024-09-18T00:18:11.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030148369999999998 2024-09-18T00:18:11.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.852537+0000", 2024-09-18T00:18:11.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011069910000000001 2024-09-18T00:18:11.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.852703+0000", 2024-09-18T00:18:11.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016631600000000001 2024-09-18T00:18:11.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.852821+0000", 2024-09-18T00:18:11.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000117642 2024-09-18T00:18:11.498 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.498 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.498 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.499 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.499 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836714+0000", 2024-09-18T00:18:11.499 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.608977299999999, 2024-09-18T00:18:11.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017759086, 2024-09-18T00:18:11.499 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.499 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.499 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.499 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.499 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.499 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.499 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 112 2024-09-18T00:18:11.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.499 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836714+0000", 2024-09-18T00:18:11.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836714+0000", 2024-09-18T00:18:11.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836715+0000", 2024-09-18T00:18:11.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1030000000000001e-06 2024-09-18T00:18:11.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836810+0000", 2024-09-18T00:18:11.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5192000000000006e-05 2024-09-18T00:18:11.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836839+0000", 2024-09-18T00:18:11.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8753999999999999e-05 2024-09-18T00:18:11.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836849+0000", 2024-09-18T00:18:11.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7860000000000008e-06 2024-09-18T00:18:11.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.849128+0000", 2024-09-18T00:18:11.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012278839 2024-09-18T00:18:11.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.849256+0000", 2024-09-18T00:18:11.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012836499999999999 2024-09-18T00:18:11.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.849450+0000", 2024-09-18T00:18:11.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019406600000000001 2024-09-18T00:18:11.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.852958+0000", 2024-09-18T00:18:11.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035073930000000001 2024-09-18T00:18:11.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854162+0000", 2024-09-18T00:18:11.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0012046100000000001 2024-09-18T00:18:11.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854355+0000", 2024-09-18T00:18:11.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000192449 2024-09-18T00:18:11.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854473+0000", 2024-09-18T00:18:11.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000118529 2024-09-18T00:18:11.504 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.504 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.504 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.504 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.504 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836865+0000", 2024-09-18T00:18:11.504 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.608826785000002, 2024-09-18T00:18:11.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019780444000000001, 2024-09-18T00:18:11.505 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.505 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.505 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.505 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.505 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.505 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.505 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 113 2024-09-18T00:18:11.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.505 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836865+0000", 2024-09-18T00:18:11.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836865+0000", 2024-09-18T00:18:11.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836866+0000", 2024-09-18T00:18:11.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7400000000000002e-07 2024-09-18T00:18:11.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836872+0000", 2024-09-18T00:18:11.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0260000000000004e-06 2024-09-18T00:18:11.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836885+0000", 2024-09-18T00:18:11.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3303e-05 2024-09-18T00:18:11.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836892+0000", 2024-09-18T00:18:11.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5700000000000004e-06 2024-09-18T00:18:11.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851546+0000", 2024-09-18T00:18:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014653497999999999 2024-09-18T00:18:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851760+0000", 2024-09-18T00:18:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021415400000000001 2024-09-18T00:18:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851954+0000", 2024-09-18T00:18:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019398199999999999 2024-09-18T00:18:11.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856341+0000", 2024-09-18T00:18:11.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0043870489999999996 2024-09-18T00:18:11.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856388+0000", 2024-09-18T00:18:11.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6919999999999998e-05 2024-09-18T00:18:11.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856530+0000", 2024-09-18T00:18:11.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142249 2024-09-18T00:18:11.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856645+0000", 2024-09-18T00:18:11.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000114819 2024-09-18T00:18:11.510 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.510 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.510 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.510 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.510 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836911+0000", 2024-09-18T00:18:11.510 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.608780074999999, 2024-09-18T00:18:11.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.021208742999999999, 2024-09-18T00:18:11.511 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.511 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.511 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.511 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.511 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.511 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.511 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 114 2024-09-18T00:18:11.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.511 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836911+0000", 2024-09-18T00:18:11.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836911+0000", 2024-09-18T00:18:11.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836912+0000", 2024-09-18T00:18:11.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5600000000000005e-07 2024-09-18T00:18:11.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836919+0000", 2024-09-18T00:18:11.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4400000000000002e-06 2024-09-18T00:18:11.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836928+0000", 2024-09-18T00:18:11.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.91e-06 2024-09-18T00:18:11.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836935+0000", 2024-09-18T00:18:11.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0829999999999998e-06 2024-09-18T00:18:11.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.853068+0000", 2024-09-18T00:18:11.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016132082999999998 2024-09-18T00:18:11.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.853344+0000", 2024-09-18T00:18:11.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027593799999999999 2024-09-18T00:18:11.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.853571+0000", 2024-09-18T00:18:11.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022752800000000001 2024-09-18T00:18:11.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856732+0000", 2024-09-18T00:18:11.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031610779999999999 2024-09-18T00:18:11.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.857856+0000", 2024-09-18T00:18:11.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001123752 2024-09-18T00:18:11.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.857998+0000", 2024-09-18T00:18:11.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142292 2024-09-18T00:18:11.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.858120+0000", 2024-09-18T00:18:11.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121983 2024-09-18T00:18:11.515 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.516 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.516 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.516 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.516 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836969+0000", 2024-09-18T00:18:11.516 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.608722356000001, 2024-09-18T00:18:11.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.022609931999999999, 2024-09-18T00:18:11.516 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.516 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.516 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.516 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.516 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.516 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.516 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 115 2024-09-18T00:18:11.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.517 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836969+0000", 2024-09-18T00:18:11.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836969+0000", 2024-09-18T00:18:11.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836970+0000", 2024-09-18T00:18:11.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1399999999999996e-07 2024-09-18T00:18:11.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837068+0000", 2024-09-18T00:18:11.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7868999999999994e-05 2024-09-18T00:18:11.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837095+0000", 2024-09-18T00:18:11.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7793e-05 2024-09-18T00:18:11.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837106+0000", 2024-09-18T00:18:11.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.028e-05 2024-09-18T00:18:11.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854556+0000", 2024-09-18T00:18:11.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017449742000000001 2024-09-18T00:18:11.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854690+0000", 2024-09-18T00:18:11.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013440500000000001 2024-09-18T00:18:11.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854900+0000", 2024-09-18T00:18:11.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000210306 2024-09-18T00:18:11.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.858224+0000", 2024-09-18T00:18:11.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0033242010000000002 2024-09-18T00:18:11.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859316+0000", 2024-09-18T00:18:11.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010916109999999999 2024-09-18T00:18:11.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859450+0000", 2024-09-18T00:18:11.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013410900000000001 2024-09-18T00:18:11.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859579+0000", 2024-09-18T00:18:11.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012890199999999999 2024-09-18T00:18:11.521 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.521 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.521 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.521 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.521 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.837122+0000", 2024-09-18T00:18:11.521 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.608569453000001, 2024-09-18T00:18:11.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.022693474000000002, 2024-09-18T00:18:11.522 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.522 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.522 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.522 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.522 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.522 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.522 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116 2024-09-18T00:18:11.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.522 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837122+0000", 2024-09-18T00:18:11.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837122+0000", 2024-09-18T00:18:11.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837123+0000", 2024-09-18T00:18:11.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.33e-07 2024-09-18T00:18:11.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837131+0000", 2024-09-18T00:18:11.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5029999999999997e-06 2024-09-18T00:18:11.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837142+0000", 2024-09-18T00:18:11.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0329999999999999e-05 2024-09-18T00:18:11.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837149+0000", 2024-09-18T00:18:11.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.99e-06 2024-09-18T00:18:11.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856845+0000", 2024-09-18T00:18:11.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019696074000000001 2024-09-18T00:18:11.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.857107+0000", 2024-09-18T00:18:11.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026233799999999998 2024-09-18T00:18:11.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.857298+0000", 2024-09-18T00:18:11.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019087600000000001 2024-09-18T00:18:11.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859604+0000", 2024-09-18T00:18:11.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023056980000000001 2024-09-18T00:18:11.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859667+0000", 2024-09-18T00:18:11.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3696999999999997e-05 2024-09-18T00:18:11.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859749+0000", 2024-09-18T00:18:11.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1526000000000005e-05 2024-09-18T00:18:11.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859816+0000", 2024-09-18T00:18:11.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6508999999999996e-05 2024-09-18T00:18:11.527 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.527 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.527 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.527 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.527 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.837160+0000", 2024-09-18T00:18:11.527 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.608531380999999, 2024-09-18T00:18:11.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023190008000000002, 2024-09-18T00:18:11.527 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.527 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.527 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.527 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.527 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.527 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.528 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 117 2024-09-18T00:18:11.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.528 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837160+0000", 2024-09-18T00:18:11.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837160+0000", 2024-09-18T00:18:11.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837161+0000", 2024-09-18T00:18:11.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2399999999999998e-07 2024-09-18T00:18:11.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837166+0000", 2024-09-18T00:18:11.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.716e-06 2024-09-18T00:18:11.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837179+0000", 2024-09-18T00:18:11.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2063e-05 2024-09-18T00:18:11.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837186+0000", 2024-09-18T00:18:11.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1040000000000001e-06 2024-09-18T00:18:11.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.858312+0000", 2024-09-18T00:18:11.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.021126147000000001 2024-09-18T00:18:11.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.858553+0000", 2024-09-18T00:18:11.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024152099999999999 2024-09-18T00:18:11.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.858880+0000", 2024-09-18T00:18:11.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000326461 2024-09-18T00:18:11.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859939+0000", 2024-09-18T00:18:11.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001059735 2024-09-18T00:18:11.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.860212+0000", 2024-09-18T00:18:11.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027284400000000001 2024-09-18T00:18:11.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.860292+0000", 2024-09-18T00:18:11.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9983999999999999e-05 2024-09-18T00:18:11.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.860350+0000", 2024-09-18T00:18:11.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7809000000000002e-05 2024-09-18T00:18:11.532 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.532 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.532 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.532 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.532 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.317506+0000", 2024-09-18T00:18:11.532 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.128185220999999, 2024-09-18T00:18:11.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011471278999999999, 2024-09-18T00:18:11.533 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.533 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.533 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.533 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.533 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.533 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.533 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 135 2024-09-18T00:18:11.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.533 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317506+0000", 2024-09-18T00:18:11.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317506+0000", 2024-09-18T00:18:11.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317507+0000", 2024-09-18T00:18:11.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7000000000000003e-07 2024-09-18T00:18:11.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317513+0000", 2024-09-18T00:18:11.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5160000000000002e-06 2024-09-18T00:18:11.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317522+0000", 2024-09-18T00:18:11.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2599999999999994e-06 2024-09-18T00:18:11.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317529+0000", 2024-09-18T00:18:11.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1570000000000003e-06 2024-09-18T00:18:11.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320156+0000", 2024-09-18T00:18:11.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00262737 2024-09-18T00:18:11.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320285+0000", 2024-09-18T00:18:11.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012859199999999999 2024-09-18T00:18:11.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320398+0000", 2024-09-18T00:18:11.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112553 2024-09-18T00:18:11.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.324003+0000", 2024-09-18T00:18:11.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036049300000000001 2024-09-18T00:18:11.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.328923+0000", 2024-09-18T00:18:11.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049205899999999999 2024-09-18T00:18:11.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.328952+0000", 2024-09-18T00:18:11.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8858999999999999e-05 2024-09-18T00:18:11.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.328978+0000", 2024-09-18T00:18:11.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5582e-05 2024-09-18T00:18:11.538 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.538 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.538 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.538 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.538 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.317540+0000", 2024-09-18T00:18:11.538 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.128151871, 2024-09-18T00:18:11.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011513031999999999, 2024-09-18T00:18:11.538 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.538 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.538 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.538 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.539 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.539 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.539 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 136 2024-09-18T00:18:11.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.539 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317540+0000", 2024-09-18T00:18:11.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317540+0000", 2024-09-18T00:18:11.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317540+0000", 2024-09-18T00:18:11.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9999999999999997e-07 2024-09-18T00:18:11.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317545+0000", 2024-09-18T00:18:11.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1030000000000001e-06 2024-09-18T00:18:11.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317554+0000", 2024-09-18T00:18:11.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9430000000000006e-06 2024-09-18T00:18:11.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317561+0000", 2024-09-18T00:18:11.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0299999999999996e-06 2024-09-18T00:18:11.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320658+0000", 2024-09-18T00:18:11.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030962950000000002 2024-09-18T00:18:11.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320712+0000", 2024-09-18T00:18:11.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4159000000000001e-05 2024-09-18T00:18:11.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320791+0000", 2024-09-18T00:18:11.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.962e-05 2024-09-18T00:18:11.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.324006+0000", 2024-09-18T00:18:11.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032141320000000002 2024-09-18T00:18:11.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329007+0000", 2024-09-18T00:18:11.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050010899999999997 2024-09-18T00:18:11.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329031+0000", 2024-09-18T00:18:11.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4493000000000001e-05 2024-09-18T00:18:11.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329053+0000", 2024-09-18T00:18:11.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1467e-05 2024-09-18T00:18:11.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.544 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.544 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.544 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.544 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.317592+0000", 2024-09-18T00:18:11.544 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.128099564999999, 2024-09-18T00:18:11.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011529286, 2024-09-18T00:18:11.544 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.544 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.544 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.545 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.545 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 137 2024-09-18T00:18:11.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.545 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317592+0000", 2024-09-18T00:18:11.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317592+0000", 2024-09-18T00:18:11.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317593+0000", 2024-09-18T00:18:11.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7000000000000003e-07 2024-09-18T00:18:11.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317750+0000", 2024-09-18T00:18:11.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015739199999999999 2024-09-18T00:18:11.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317764+0000", 2024-09-18T00:18:11.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3845999999999999e-05 2024-09-18T00:18:11.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317840+0000", 2024-09-18T00:18:11.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6253000000000003e-05 2024-09-18T00:18:11.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.321078+0000", 2024-09-18T00:18:11.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032379560000000002 2024-09-18T00:18:11.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.321125+0000", 2024-09-18T00:18:11.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6585999999999997e-05 2024-09-18T00:18:11.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.321195+0000", 2024-09-18T00:18:11.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0188e-05 2024-09-18T00:18:11.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.324011+0000", 2024-09-18T00:18:11.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0028159410000000002 2024-09-18T00:18:11.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329075+0000", 2024-09-18T00:18:11.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050643379999999998 2024-09-18T00:18:11.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329103+0000", 2024-09-18T00:18:11.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7616000000000001e-05 2024-09-18T00:18:11.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329121+0000", 2024-09-18T00:18:11.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8199999999999999e-05 2024-09-18T00:18:11.549 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.549 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.550 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.550 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.550 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.317909+0000", 2024-09-18T00:18:11.550 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.127782168, 2024-09-18T00:18:11.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011404755000000001, 2024-09-18T00:18:11.550 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.550 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.550 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.550 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.550 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.550 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.550 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 139 2024-09-18T00:18:11.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.551 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317909+0000", 2024-09-18T00:18:11.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317909+0000", 2024-09-18T00:18:11.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317910+0000", 2024-09-18T00:18:11.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2699999999999999e-07 2024-09-18T00:18:11.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317916+0000", 2024-09-18T00:18:11.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5230000000000003e-06 2024-09-18T00:18:11.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317924+0000", 2024-09-18T00:18:11.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9239999999999996e-06 2024-09-18T00:18:11.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317932+0000", 2024-09-18T00:18:11.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0700000000000001e-06 2024-09-18T00:18:11.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322402+0000", 2024-09-18T00:18:11.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044705420000000001 2024-09-18T00:18:11.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322452+0000", 2024-09-18T00:18:11.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0297999999999999e-05 2024-09-18T00:18:11.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322533+0000", 2024-09-18T00:18:11.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1092000000000002e-05 2024-09-18T00:18:11.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.324722+0000", 2024-09-18T00:18:11.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0021886010000000001 2024-09-18T00:18:11.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329227+0000", 2024-09-18T00:18:11.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045046419999999997 2024-09-18T00:18:11.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329264+0000", 2024-09-18T00:18:11.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7577999999999998e-05 2024-09-18T00:18:11.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329314+0000", 2024-09-18T00:18:11.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9758e-05 2024-09-18T00:18:11.555 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.555 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.555 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.555 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.555 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.317962+0000", 2024-09-18T00:18:11.555 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.127729084999999, 2024-09-18T00:18:11.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01213116, 2024-09-18T00:18:11.556 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.556 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.556 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.556 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.556 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.556 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.556 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 140 2024-09-18T00:18:11.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.556 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317962+0000", 2024-09-18T00:18:11.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317962+0000", 2024-09-18T00:18:11.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317963+0000", 2024-09-18T00:18:11.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8700000000000004e-07 2024-09-18T00:18:11.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318086+0000", 2024-09-18T00:18:11.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000122862 2024-09-18T00:18:11.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318099+0000", 2024-09-18T00:18:11.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2459999999999999e-05 2024-09-18T00:18:11.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318107+0000", 2024-09-18T00:18:11.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-18T00:18:11.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322800+0000", 2024-09-18T00:18:11.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046924649999999998 2024-09-18T00:18:11.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322849+0000", 2024-09-18T00:18:11.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9406000000000001e-05 2024-09-18T00:18:11.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322927+0000", 2024-09-18T00:18:11.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7763999999999996e-05 2024-09-18T00:18:11.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.324725+0000", 2024-09-18T00:18:11.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0017981060000000001 2024-09-18T00:18:11.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329976+0000", 2024-09-18T00:18:11.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052507109999999999 2024-09-18T00:18:11.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.330045+0000", 2024-09-18T00:18:11.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8998999999999994e-05 2024-09-18T00:18:11.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.330094+0000", 2024-09-18T00:18:11.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.905e-05 2024-09-18T00:18:11.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.560 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.561 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:11.561 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.318122+0000", 2024-09-18T00:18:11.561 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.127569829999999, 2024-09-18T00:18:11.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012634492000000001, 2024-09-18T00:18:11.561 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:11.561 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:11.561 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:11.561 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:11.561 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:11.561 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:11.561 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 141 2024-09-18T00:18:11.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.561 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:11.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:11.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318122+0000", 2024-09-18T00:18:11.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:11.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318122+0000", 2024-09-18T00:18:11.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:11.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:11.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318122+0000", 2024-09-18T00:18:11.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9400000000000005e-07 2024-09-18T00:18:11.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:11.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318128+0000", 2024-09-18T00:18:11.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3800000000000002e-06 2024-09-18T00:18:11.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:11.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318136+0000", 2024-09-18T00:18:11.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7760000000000003e-06 2024-09-18T00:18:11.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:11.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318143+0000", 2024-09-18T00:18:11.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-18T00:18:11.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:11.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.323651+0000", 2024-09-18T00:18:11.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055075269999999999 2024-09-18T00:18:11.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:11.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.323700+0000", 2024-09-18T00:18:11.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8724999999999999e-05 2024-09-18T00:18:11.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:11.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.323772+0000", 2024-09-18T00:18:11.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2378000000000004e-05 2024-09-18T00:18:11.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:11.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.325433+0000", 2024-09-18T00:18:11.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001661278 2024-09-18T00:18:11.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:11.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.330670+0000", 2024-09-18T00:18:11.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052365689999999999 2024-09-18T00:18:11.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:11.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.330737+0000", 2024-09-18T00:18:11.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6866999999999995e-05 2024-09-18T00:18:11.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:11.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:11.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:11.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.330756+0000", 2024-09-18T00:18:11.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9313999999999998e-05 2024-09-18T00:18:11.566 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.566 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.566 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.566 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:11.566 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:11.566 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:11.567 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:11.563+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:11.568 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:18:11.622 INFO:teuthology.orchestra.run.smithi097.stdout:116 writesame 0x35b9bca thru 0x35be868 data_size 0x1(0x4c9f bytes) 2024-09-18T00:18:11.623 INFO:teuthology.orchestra.run.smithi097.stdout:118 read 0x3f01ae thru 0x3fefb3 (0xee06 bytes) 2024-09-18T00:18:11.623 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:11.623 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:11.645 INFO:teuthology.orchestra.run.smithi097.stdout:123 write 0xb0241ea thru 0xb02cc26 (0x8a3d bytes) 2024-09-18T00:18:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:125 punch from 0x795c0a4 to 0x796baeb, (0xfa47 bytes) 2024-09-18T00:18:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:127 punch from 0x43ba895 to 0x43c8afa, (0xe265 bytes) 2024-09-18T00:18:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:128 punch from 0x87d71b8 to 0x87dcb47, (0x598f bytes) 2024-09-18T00:18:11.654 INFO:teuthology.orchestra.run.smithi097.stdout:129 punch from 0x8c4d22b to 0x8c53c65, (0x6a3a bytes) 2024-09-18T00:18:11.668 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:11.667+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:11.669 INFO:teuthology.orchestra.run.smithi097.stdout:131 writesame 0xed531c2 thru 0xed5fb62 data_size 0x1(0xc9a1 bytes) 2024-09-18T00:18:11.670 INFO:teuthology.orchestra.run.smithi097.stdout:132 read 0x20f77e8 thru 0x210100d (0x9826 bytes) 2024-09-18T00:18:11.670 INFO:teuthology.orchestra.run.smithi097.stdout:135 read 0x4655e84 thru 0x465e775 (0x88f2 bytes) 2024-09-18T00:18:11.670 INFO:teuthology.orchestra.run.smithi097.stdout:136 write 0xd70191 thru 0xd77b7b (0x79eb bytes) 2024-09-18T00:18:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:138 read 0x386965 thru 0x389e4a (0x34e6 bytes) 2024-09-18T00:18:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:139 write 0xcec03d1 thru 0xcec0ba1 (0x7d1 bytes) 2024-09-18T00:18:11.694 INFO:teuthology.orchestra.run.smithi097.stdout:140 writesame 0xeda9974 thru 0xedb402d data_size 0x1(0xa6ba bytes) 2024-09-18T00:18:11.694 INFO:teuthology.orchestra.run.smithi097.stdout:141 write 0xc80b28 thru 0xc8e063 (0xd53c bytes) 2024-09-18T00:18:11.699 INFO:teuthology.orchestra.run.smithi097.stdout:143 read 0x6a5dd93 thru 0x6a62808 (0x4a76 bytes) 2024-09-18T00:18:11.705 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:11.705 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:11.705 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:11.705 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:11.708 INFO:teuthology.orchestra.run.smithi097.stdout:144 trunc from 0xedb402e to 0x9285cd7 2024-09-18T00:18:11.722 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:18:11.735 INFO:teuthology.orchestra.run.smithi097.stdout:145 trunc from 0x9285cd7 to 0x68ae04e 2024-09-18T00:18:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:146 read 0x4de3122 thru 0x4ded071 (0x9f50 bytes) 2024-09-18T00:18:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:147 write 0x48ba989 thru 0x48c5163 (0xa7db bytes) 2024-09-18T00:18:11.756 INFO:teuthology.orchestra.run.smithi097.stdout:148 write 0xc6fd64 thru 0xc7e742 (0xe9df bytes) 2024-09-18T00:18:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:149 punch from 0x48778d8 to 0x487a029, (0x2751 bytes) 2024-09-18T00:18:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:150 punch from 0x180eb4a to 0x181d21d, (0xe6d3 bytes) 2024-09-18T00:18:11.765 INFO:teuthology.orchestra.run.smithi097.stdout:151 punch from 0x3c5b027 to 0x3c6a465, (0xf43e bytes) 2024-09-18T00:18:11.770 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:11.767+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:11.779 INFO:teuthology.orchestra.run.smithi097.stdout:152 write 0x8f819ba thru 0x8f8f1fb (0xd842 bytes) 2024-09-18T00:18:11.788 INFO:teuthology.orchestra.run.smithi097.stdout:153 writesame 0x7b1a9d thru 0x7b3cea data_size 0x1(0x224e bytes) 2024-09-18T00:18:11.804 INFO:teuthology.orchestra.run.smithi097.stdout:155 write 0x3e57b0f thru 0x3e5be3a (0x432c bytes) 2024-09-18T00:18:11.814 INFO:teuthology.orchestra.run.smithi097.stdout:157 read 0x4a1ea5d thru 0x4a23d56 (0x52fa bytes) 2024-09-18T00:18:11.814 INFO:teuthology.orchestra.run.smithi097.stdout:158 punch from 0x5bbec5a to 0x5bcce70, (0xe216 bytes) 2024-09-18T00:18:11.814 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:11.814 INFO:teuthology.orchestra.run.smithi097.stdout:160 write 0x62aff07 thru 0x62bd737 (0xd831 bytes) 2024-09-18T00:18:11.828 INFO:teuthology.orchestra.run.smithi097.stdout:161 read 0x24459b2 thru 0x2451ea7 (0xc4f6 bytes) 2024-09-18T00:18:11.828 INFO:teuthology.orchestra.run.smithi097.stdout:162 punch from 0x722396d to 0x72301c3, (0xc856 bytes) 2024-09-18T00:18:11.828 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:11.828 INFO:teuthology.orchestra.run.smithi097.stdout:166 punch from 0x18f6d01 to 0x18fe86b, (0x7b6a bytes) 2024-09-18T00:18:11.845 INFO:teuthology.orchestra.run.smithi097.stdout:167 write 0xd643715 thru 0xd64d6cf (0x9fbb bytes) 2024-09-18T00:18:11.864 INFO:teuthology.orchestra.run.smithi097.stdout:168 write 0xde565c6 thru 0xde5cfbd (0x69f8 bytes) 2024-09-18T00:18:11.871 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:11.867+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:11.871 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:11.871 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:11.871 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:11.871 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:11.871 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:11.872 INFO:teuthology.orchestra.run.smithi097.stdout:169 read 0x14e317f thru 0x14ec8aa (0x972c bytes) 2024-09-18T00:18:11.872 INFO:teuthology.orchestra.run.smithi097.stdout:170 write 0x9a39401 thru 0x9a44e73 (0xba73 bytes) 2024-09-18T00:18:11.884 INFO:teuthology.orchestra.run.smithi097.stdout:171 read 0x1cef3c8 thru 0x1cf2411 (0x304a bytes) 2024-09-18T00:18:11.884 INFO:teuthology.orchestra.run.smithi097.stdout:172 write 0x478ce75 thru 0x4794942 (0x7ace bytes) 2024-09-18T00:18:11.886 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:18:11.906 INFO:teuthology.orchestra.run.smithi097.stdout:175 write 0xe28daaa thru 0xe29b690 (0xdbe7 bytes) 2024-09-18T00:18:11.914 INFO:teuthology.orchestra.run.smithi097.stdout:176 read 0x43eab23 thru 0x43f2433 (0x7911 bytes) 2024-09-18T00:18:11.915 INFO:teuthology.orchestra.run.smithi097.stdout:177 punch from 0xdd247b8 to 0xdd25565, (0xdad bytes) 2024-09-18T00:18:11.915 INFO:teuthology.orchestra.run.smithi097.stdout:178 read 0x479c3d5 thru 0x47a01dc (0x3e08 bytes) 2024-09-18T00:18:11.925 INFO:teuthology.orchestra.run.smithi097.stdout:179 trunc from 0xe29b691 to 0x7b9ac0e 2024-09-18T00:18:11.947 INFO:teuthology.orchestra.run.smithi097.stdout:180 write 0x41683e2 thru 0x416a393 (0x1fb2 bytes) 2024-09-18T00:18:11.955 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:11.955 INFO:teuthology.orchestra.run.smithi097.stdout:truncating to largest ever: 0xe362201 2024-09-18T00:18:11.966 INFO:teuthology.orchestra.run.smithi097.stdout:183 trunc from 0x7b9ac0e to 0xe362201 2024-09-18T00:18:11.971 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:11.967+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:11.975 INFO:teuthology.orchestra.run.smithi097.stdout:185 punch from 0x4f438b6 to 0x4f4724b, (0x3995 bytes) 2024-09-18T00:18:11.975 INFO:teuthology.orchestra.run.smithi097.stdout:187 writesame 0x8846ded thru 0x884e65a data_size 0x5(0x786e bytes) 2024-09-18T00:18:11.997 INFO:teuthology.orchestra.run.smithi097.stdout:188 punch from 0x859be3d to 0x85a9069, (0xd22c bytes) 2024-09-18T00:18:11.997 INFO:teuthology.orchestra.run.smithi097.stdout:191 writesame 0x7faf38 thru 0x7fdc43 data_size 0x1f(0x2d0c bytes) 2024-09-18T00:18:12.017 INFO:teuthology.orchestra.run.smithi097.stdout:192 trunc from 0xe362201 to 0x925bd0b 2024-09-18T00:18:12.027 INFO:teuthology.orchestra.run.smithi097.stdout:193 read 0x8b59963 thru 0x8b647c7 (0xae65 bytes) 2024-09-18T00:18:12.041 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:12.041 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:12.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:12.042 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:12.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.042 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:34 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'20, mlcod=17'14)", 2024-09-18T00:18:12.042 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:51.558207+0000", 2024-09-18T00:18:12.042 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.480827820999998, 2024-09-18T00:18:12.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0046458009999999998, 2024-09-18T00:18:12.042 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.042 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.042 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.042 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.558207+0000", 2024-09-18T00:18:12.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.558207+0000", 2024-09-18T00:18:12.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.558209+0000", 2024-09-18T00:18:12.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3769999999999999e-06 2024-09-18T00:18:12.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.558230+0000", 2024-09-18T00:18:12.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1336e-05 2024-09-18T00:18:12.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.558249+0000", 2024-09-18T00:18:12.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.84e-05 2024-09-18T00:18:12.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.558259+0000", 2024-09-18T00:18:12.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0720000000000001e-05 2024-09-18T00:18:12.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.558428+0000", 2024-09-18T00:18:12.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000168519 2024-09-18T00:18:12.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.558457+0000", 2024-09-18T00:18:12.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9456000000000001e-05 2024-09-18T00:18:12.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.562721+0000", 2024-09-18T00:18:12.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042641739999999999 2024-09-18T00:18:12.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.562853+0000", 2024-09-18T00:18:12.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013181900000000001 2024-09-18T00:18:12.046 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.046 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.046 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.046 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:36 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'22, mlcod=17'18)", 2024-09-18T00:18:12.047 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:51.563151+0000", 2024-09-18T00:18:12.047 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.475883962000001, 2024-09-18T00:18:12.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067203280000000002, 2024-09-18T00:18:12.047 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.047 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.047 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.047 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563151+0000", 2024-09-18T00:18:12.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563151+0000", 2024-09-18T00:18:12.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563153+0000", 2024-09-18T00:18:12.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.42e-06 2024-09-18T00:18:12.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563163+0000", 2024-09-18T00:18:12.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.077e-05 2024-09-18T00:18:12.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563191+0000", 2024-09-18T00:18:12.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.724e-05 2024-09-18T00:18:12.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563204+0000", 2024-09-18T00:18:12.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3266e-05 2024-09-18T00:18:12.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563319+0000", 2024-09-18T00:18:12.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115343 2024-09-18T00:18:12.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563371+0000", 2024-09-18T00:18:12.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1922000000000002e-05 2024-09-18T00:18:12.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.569760+0000", 2024-09-18T00:18:12.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388721 2024-09-18T00:18:12.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.569872+0000", 2024-09-18T00:18:12.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011164599999999999 2024-09-18T00:18:12.051 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.051 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.051 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.051 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:37 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'23, mlcod=17'18)", 2024-09-18T00:18:12.051 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:51.564518+0000", 2024-09-18T00:18:12.051 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.474516896000001, 2024-09-18T00:18:12.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005870994, 2024-09-18T00:18:12.051 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.051 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.051 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.051 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564518+0000", 2024-09-18T00:18:12.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564518+0000", 2024-09-18T00:18:12.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564520+0000", 2024-09-18T00:18:12.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8300000000000001e-06 2024-09-18T00:18:12.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564532+0000", 2024-09-18T00:18:12.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1754e-05 2024-09-18T00:18:12.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564560+0000", 2024-09-18T00:18:12.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8486000000000001e-05 2024-09-18T00:18:12.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564573+0000", 2024-09-18T00:18:12.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3069e-05 2024-09-18T00:18:12.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564660+0000", 2024-09-18T00:18:12.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6263999999999999e-05 2024-09-18T00:18:12.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564695+0000", 2024-09-18T00:18:12.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5296000000000002e-05 2024-09-18T00:18:12.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.570284+0000", 2024-09-18T00:18:12.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055890030000000004 2024-09-18T00:18:12.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.570389+0000", 2024-09-18T00:18:12.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010529199999999999 2024-09-18T00:18:12.055 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.055 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.055 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.055 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:130 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'88, mlcod=17'85)", 2024-09-18T00:18:12.055 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.319467+0000", 2024-09-18T00:18:12.055 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.719568038999999, 2024-09-18T00:18:12.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047039869999999998, 2024-09-18T00:18:12.056 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.056 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.056 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.056 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319467+0000", 2024-09-18T00:18:12.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319467+0000", 2024-09-18T00:18:12.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319469+0000", 2024-09-18T00:18:12.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0229999999999999e-06 2024-09-18T00:18:12.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319512+0000", 2024-09-18T00:18:12.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3232999999999998e-05 2024-09-18T00:18:12.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319542+0000", 2024-09-18T00:18:12.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0046e-05 2024-09-18T00:18:12.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319564+0000", 2024-09-18T00:18:12.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1489999999999999e-05 2024-09-18T00:18:12.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319691+0000", 2024-09-18T00:18:12.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012706600000000001 2024-09-18T00:18:12.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319726+0000", 2024-09-18T00:18:12.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5206000000000001e-05 2024-09-18T00:18:12.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324096+0000", 2024-09-18T00:18:12.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043692959999999999 2024-09-18T00:18:12.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324171+0000", 2024-09-18T00:18:12.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5626999999999999e-05 2024-09-18T00:18:12.059 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.059 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.059 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.060 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:131 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'89, mlcod=17'85)", 2024-09-18T00:18:12.060 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.319608+0000", 2024-09-18T00:18:12.060 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.719426807000001, 2024-09-18T00:18:12.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0058413620000000001, 2024-09-18T00:18:12.060 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.060 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.060 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.060 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319608+0000", 2024-09-18T00:18:12.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319608+0000", 2024-09-18T00:18:12.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319609+0000", 2024-09-18T00:18:12.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3e-07 2024-09-18T00:18:12.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319676+0000", 2024-09-18T00:18:12.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6889000000000003e-05 2024-09-18T00:18:12.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319694+0000", 2024-09-18T00:18:12.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7703e-05 2024-09-18T00:18:12.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319706+0000", 2024-09-18T00:18:12.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2577e-05 2024-09-18T00:18:12.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320272+0000", 2024-09-18T00:18:12.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056591599999999997 2024-09-18T00:18:12.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320306+0000", 2024-09-18T00:18:12.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4116000000000003e-05 2024-09-18T00:18:12.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325386+0000", 2024-09-18T00:18:12.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050793349999999999 2024-09-18T00:18:12.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325450+0000", 2024-09-18T00:18:12.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4196000000000007e-05 2024-09-18T00:18:12.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.064 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.064 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:132 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'90, mlcod=17'85)", 2024-09-18T00:18:12.064 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.320067+0000", 2024-09-18T00:18:12.065 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.718968274000002, 2024-09-18T00:18:12.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0054358590000000004, 2024-09-18T00:18:12.065 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.065 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.065 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.065 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320067+0000", 2024-09-18T00:18:12.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320067+0000", 2024-09-18T00:18:12.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320068+0000", 2024-09-18T00:18:12.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0100000000000001e-06 2024-09-18T00:18:12.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320079+0000", 2024-09-18T00:18:12.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1146e-05 2024-09-18T00:18:12.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320101+0000", 2024-09-18T00:18:12.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1994e-05 2024-09-18T00:18:12.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320110+0000", 2024-09-18T00:18:12.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:18:12.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321623+0000", 2024-09-18T00:18:12.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001512981 2024-09-18T00:18:12.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321656+0000", 2024-09-18T00:18:12.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2539999999999997e-05 2024-09-18T00:18:12.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325455+0000", 2024-09-18T00:18:12.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0037992880000000001 2024-09-18T00:18:12.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325503+0000", 2024-09-18T00:18:12.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7567e-05 2024-09-18T00:18:12.069 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.069 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.069 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.069 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:133 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'91, mlcod=17'85)", 2024-09-18T00:18:12.069 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.320439+0000", 2024-09-18T00:18:12.069 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.718596130999998, 2024-09-18T00:18:12.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005115858, 2024-09-18T00:18:12.069 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.069 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.069 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.069 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320439+0000", 2024-09-18T00:18:12.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320439+0000", 2024-09-18T00:18:12.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320440+0000", 2024-09-18T00:18:12.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2999999999999999e-07 2024-09-18T00:18:12.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320454+0000", 2024-09-18T00:18:12.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4107000000000001e-05 2024-09-18T00:18:12.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320472+0000", 2024-09-18T00:18:12.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7499999999999998e-05 2024-09-18T00:18:12.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320480+0000", 2024-09-18T00:18:12.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0970000000000006e-06 2024-09-18T00:18:12.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322261+0000", 2024-09-18T00:18:12.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001780946 2024-09-18T00:18:12.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322297+0000", 2024-09-18T00:18:12.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6510000000000001e-05 2024-09-18T00:18:12.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325507+0000", 2024-09-18T00:18:12.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0032097919999999999 2024-09-18T00:18:12.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325555+0000", 2024-09-18T00:18:12.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8075999999999998e-05 2024-09-18T00:18:12.073 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.073 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.073 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.073 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:134 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'92, mlcod=17'85)", 2024-09-18T00:18:12.073 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.320847+0000", 2024-09-18T00:18:12.073 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.718188399999999, 2024-09-18T00:18:12.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0068502509999999999, 2024-09-18T00:18:12.073 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.074 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.074 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.074 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320847+0000", 2024-09-18T00:18:12.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320847+0000", 2024-09-18T00:18:12.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320848+0000", 2024-09-18T00:18:12.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-18T00:18:12.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320971+0000", 2024-09-18T00:18:12.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012338599999999999 2024-09-18T00:18:12.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320990+0000", 2024-09-18T00:18:12.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9009999999999999e-05 2024-09-18T00:18:12.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321000+0000", 2024-09-18T00:18:12.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9399999999999997e-06 2024-09-18T00:18:12.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322866+0000", 2024-09-18T00:18:12.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001866089 2024-09-18T00:18:12.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322902+0000", 2024-09-18T00:18:12.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5312999999999999e-05 2024-09-18T00:18:12.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327608+0000", 2024-09-18T00:18:12.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047063000000000001 2024-09-18T00:18:12.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327697+0000", 2024-09-18T00:18:12.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9049999999999996e-05 2024-09-18T00:18:12.077 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.077 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.077 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.078 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:135 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'93, mlcod=17'85)", 2024-09-18T00:18:12.078 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.321689+0000", 2024-09-18T00:18:12.078 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.71734575, 2024-09-18T00:18:12.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0082046600000000008, 2024-09-18T00:18:12.078 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.078 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.078 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.078 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321689+0000", 2024-09-18T00:18:12.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321689+0000", 2024-09-18T00:18:12.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321691+0000", 2024-09-18T00:18:12.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0229999999999999e-06 2024-09-18T00:18:12.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321706+0000", 2024-09-18T00:18:12.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4513e-05 2024-09-18T00:18:12.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321727+0000", 2024-09-18T00:18:12.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0939999999999999e-05 2024-09-18T00:18:12.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321745+0000", 2024-09-18T00:18:12.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7986999999999999e-05 2024-09-18T00:18:12.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324259+0000", 2024-09-18T00:18:12.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025143069999999999 2024-09-18T00:18:12.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324291+0000", 2024-09-18T00:18:12.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2116999999999998e-05 2024-09-18T00:18:12.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329824+0000", 2024-09-18T00:18:12.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055326300000000002 2024-09-18T00:18:12.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329894+0000", 2024-09-18T00:18:12.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0142999999999996e-05 2024-09-18T00:18:12.082 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.082 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.082 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.082 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:136 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'94, mlcod=17'85)", 2024-09-18T00:18:12.082 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.322041+0000", 2024-09-18T00:18:12.082 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.716994563, 2024-09-18T00:18:12.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0079092060000000002, 2024-09-18T00:18:12.082 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.083 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.083 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.083 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322041+0000", 2024-09-18T00:18:12.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322041+0000", 2024-09-18T00:18:12.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322042+0000", 2024-09-18T00:18:12.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.22e-06 2024-09-18T00:18:12.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322053+0000", 2024-09-18T00:18:12.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1584e-05 2024-09-18T00:18:12.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322073+0000", 2024-09-18T00:18:12.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0023000000000001e-05 2024-09-18T00:18:12.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322092+0000", 2024-09-18T00:18:12.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8063e-05 2024-09-18T00:18:12.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324848+0000", 2024-09-18T00:18:12.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027561420000000001 2024-09-18T00:18:12.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324877+0000", 2024-09-18T00:18:12.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9003e-05 2024-09-18T00:18:12.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329900+0000", 2024-09-18T00:18:12.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050233650000000001 2024-09-18T00:18:12.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329950+0000", 2024-09-18T00:18:12.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9805999999999997e-05 2024-09-18T00:18:12.087 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.087 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.087 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.087 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:137 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'95, mlcod=17'85)", 2024-09-18T00:18:12.087 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.322465+0000", 2024-09-18T00:18:12.087 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.716569859, 2024-09-18T00:18:12.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0075526680000000002, 2024-09-18T00:18:12.087 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.087 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.087 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.087 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322465+0000", 2024-09-18T00:18:12.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322465+0000", 2024-09-18T00:18:12.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322467+0000", 2024-09-18T00:18:12.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5999999999999999e-06 2024-09-18T00:18:12.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322612+0000", 2024-09-18T00:18:12.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014553499999999999 2024-09-18T00:18:12.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322631+0000", 2024-09-18T00:18:12.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8627e-05 2024-09-18T00:18:12.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322645+0000", 2024-09-18T00:18:12.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4109e-05 2024-09-18T00:18:12.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325646+0000", 2024-09-18T00:18:12.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0030007179999999999 2024-09-18T00:18:12.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325674+0000", 2024-09-18T00:18:12.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7733e-05 2024-09-18T00:18:12.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329958+0000", 2024-09-18T00:18:12.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042841329999999999 2024-09-18T00:18:12.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330018+0000", 2024-09-18T00:18:12.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0213000000000001e-05 2024-09-18T00:18:12.091 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.091 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.091 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.092 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:138 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'96, mlcod=17'85)", 2024-09-18T00:18:12.092 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.323237+0000", 2024-09-18T00:18:12.092 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.715798712000002, 2024-09-18T00:18:12.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0068313540000000004, 2024-09-18T00:18:12.092 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.092 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.092 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.092 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323237+0000", 2024-09-18T00:18:12.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323237+0000", 2024-09-18T00:18:12.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323238+0000", 2024-09-18T00:18:12.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.127e-06 2024-09-18T00:18:12.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323249+0000", 2024-09-18T00:18:12.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1157e-05 2024-09-18T00:18:12.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323277+0000", 2024-09-18T00:18:12.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8178999999999999e-05 2024-09-18T00:18:12.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323287+0000", 2024-09-18T00:18:12.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9699999999999994e-06 2024-09-18T00:18:12.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327148+0000", 2024-09-18T00:18:12.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038609109999999999 2024-09-18T00:18:12.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327178+0000", 2024-09-18T00:18:12.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0156000000000001e-05 2024-09-18T00:18:12.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330022+0000", 2024-09-18T00:18:12.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028444579999999998 2024-09-18T00:18:12.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330068+0000", 2024-09-18T00:18:12.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5395999999999997e-05 2024-09-18T00:18:12.096 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.096 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.096 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.096 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:139 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'86)", 2024-09-18T00:18:12.096 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.323792+0000", 2024-09-18T00:18:12.096 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.715243062999999, 2024-09-18T00:18:12.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063243850000000001, 2024-09-18T00:18:12.097 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.097 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.097 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.097 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323792+0000", 2024-09-18T00:18:12.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323792+0000", 2024-09-18T00:18:12.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323793+0000", 2024-09-18T00:18:12.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4399999999999998e-07 2024-09-18T00:18:12.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323806+0000", 2024-09-18T00:18:12.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3346000000000001e-05 2024-09-18T00:18:12.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323824+0000", 2024-09-18T00:18:12.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.789e-05 2024-09-18T00:18:12.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323836+0000", 2024-09-18T00:18:12.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2082999999999999e-05 2024-09-18T00:18:12.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327773+0000", 2024-09-18T00:18:12.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039365099999999998 2024-09-18T00:18:12.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327802+0000", 2024-09-18T00:18:12.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8617000000000001e-05 2024-09-18T00:18:12.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330072+0000", 2024-09-18T00:18:12.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022704219999999998 2024-09-18T00:18:12.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330117+0000", 2024-09-18T00:18:12.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4573000000000002e-05 2024-09-18T00:18:12.101 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.101 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.101 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.101 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:140 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'86)", 2024-09-18T00:18:12.101 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.324263+0000", 2024-09-18T00:18:12.101 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.714772610000001, 2024-09-18T00:18:12.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0066665350000000003, 2024-09-18T00:18:12.101 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.101 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.101 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.101 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324263+0000", 2024-09-18T00:18:12.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324263+0000", 2024-09-18T00:18:12.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324264+0000", 2024-09-18T00:18:12.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-18T00:18:12.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324387+0000", 2024-09-18T00:18:12.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012346899999999999 2024-09-18T00:18:12.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324402+0000", 2024-09-18T00:18:12.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5087e-05 2024-09-18T00:18:12.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324412+0000", 2024-09-18T00:18:12.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5969999999999998e-06 2024-09-18T00:18:12.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.328364+0000", 2024-09-18T00:18:12.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039523559999999998 2024-09-18T00:18:12.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.328397+0000", 2024-09-18T00:18:12.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2733e-05 2024-09-18T00:18:12.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330864+0000", 2024-09-18T00:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0024676670000000002 2024-09-18T00:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330929+0000", 2024-09-18T00:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4665999999999994e-05 2024-09-18T00:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:141 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'86)", 2024-09-18T00:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.325030+0000", 2024-09-18T00:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.714005631999999, 2024-09-18T00:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006603462, 2024-09-18T00:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325030+0000", 2024-09-18T00:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325030+0000", 2024-09-18T00:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325031+0000", 2024-09-18T00:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.53e-06 2024-09-18T00:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325042+0000", 2024-09-18T00:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0506000000000001e-05 2024-09-18T00:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325080+0000", 2024-09-18T00:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8300000000000003e-05 2024-09-18T00:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325098+0000", 2024-09-18T00:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8539999999999999e-05 2024-09-18T00:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330164+0000", 2024-09-18T00:18:12.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050653709999999999 2024-09-18T00:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330194+0000", 2024-09-18T00:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0275999999999999e-05 2024-09-18T00:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.331562+0000", 2024-09-18T00:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0013680330000000001 2024-09-18T00:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.331633+0000", 2024-09-18T00:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0906000000000002e-05 2024-09-18T00:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:142 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'100, mlcod=17'98)", 2024-09-18T00:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.332949+0000", 2024-09-18T00:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.706085937000001, 2024-09-18T00:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059501479999999997, 2024-09-18T00:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.110 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.332949+0000", 2024-09-18T00:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.332949+0000", 2024-09-18T00:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.332951+0000", 2024-09-18T00:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-06 2024-09-18T00:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333024+0000", 2024-09-18T00:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3112000000000001e-05 2024-09-18T00:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333050+0000", 2024-09-18T00:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5673e-05 2024-09-18T00:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333063+0000", 2024-09-18T00:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2644e-05 2024-09-18T00:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333228+0000", 2024-09-18T00:18:12.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016579799999999999 2024-09-18T00:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333267+0000", 2024-09-18T00:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8189999999999999e-05 2024-09-18T00:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338834+0000", 2024-09-18T00:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055676679999999996 2024-09-18T00:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338899+0000", 2024-09-18T00:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5198999999999997e-05 2024-09-18T00:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.115 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:143 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'101, mlcod=17'98)", 2024-09-18T00:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.333556+0000", 2024-09-18T00:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.705478882000001, 2024-09-18T00:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005386053, 2024-09-18T00:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333556+0000", 2024-09-18T00:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333556+0000", 2024-09-18T00:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333558+0000", 2024-09-18T00:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333677+0000", 2024-09-18T00:18:12.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119196 2024-09-18T00:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333698+0000", 2024-09-18T00:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1087e-05 2024-09-18T00:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333708+0000", 2024-09-18T00:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0125999999999999e-05 2024-09-18T00:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334689+0000", 2024-09-18T00:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00098056899999999993 2024-09-18T00:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334719+0000", 2024-09-18T00:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9963000000000001e-05 2024-09-18T00:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338907+0000", 2024-09-18T00:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041881619999999996 2024-09-18T00:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338942+0000", 2024-09-18T00:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5660000000000001e-05 2024-09-18T00:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:144 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'102, mlcod=17'98)", 2024-09-18T00:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.334453+0000", 2024-09-18T00:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.704582455000001, 2024-09-18T00:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045227619999999996, 2024-09-18T00:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.120 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334453+0000", 2024-09-18T00:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334453+0000", 2024-09-18T00:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334454+0000", 2024-09-18T00:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6759999999999999e-06 2024-09-18T00:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334507+0000", 2024-09-18T00:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2726999999999998e-05 2024-09-18T00:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334536+0000", 2024-09-18T00:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8385999999999999e-05 2024-09-18T00:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334549+0000", 2024-09-18T00:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3237e-05 2024-09-18T00:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.335984+0000", 2024-09-18T00:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001435609 2024-09-18T00:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.336013+0000", 2024-09-18T00:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8822999999999998e-05 2024-09-18T00:18:12.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338944+0000", 2024-09-18T00:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029308839999999999 2024-09-18T00:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338976+0000", 2024-09-18T00:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1420000000000001e-05 2024-09-18T00:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T00:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:07.405976+0000", 2024-09-18T00:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.6330595810000004, 2024-09-18T00:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063813710000000003, 2024-09-18T00:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11 2024-09-18T00:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.125 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.405976+0000", 2024-09-18T00:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.405976+0000", 2024-09-18T00:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.405979+0000", 2024-09-18T00:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9009999999999998e-06 2024-09-18T00:18:12.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.405998+0000", 2024-09-18T00:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9653e-05 2024-09-18T00:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.406027+0000", 2024-09-18T00:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8442e-05 2024-09-18T00:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.406041+0000", 2024-09-18T00:18:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4273e-05 2024-09-18T00:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.406141+0000", 2024-09-18T00:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010008199999999999 2024-09-18T00:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.406147+0000", 2024-09-18T00:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9340000000000003e-06 2024-09-18T00:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.408986+0000", 2024-09-18T00:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028392959999999998 2024-09-18T00:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.409279+0000", 2024-09-18T00:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000292596 2024-09-18T00:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.409615+0000", 2024-09-18T00:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033606 2024-09-18T00:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.411318+0000", 2024-09-18T00:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017026890000000001 2024-09-18T00:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.412269+0000", 2024-09-18T00:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00095177200000000003 2024-09-18T00:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.412319+0000", 2024-09-18T00:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9975999999999997e-05 2024-09-18T00:18:12.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.412357+0000", 2024-09-18T00:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7697000000000002e-05 2024-09-18T00:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:231 3.17 3:ee570356:::rbd_data.112a45c8e167.0000000000000037:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1398888~44282] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T00:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.796692+0000", 2024-09-18T00:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.242342925, 2024-09-18T00:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0077415899999999996, 2024-09-18T00:18:12.131 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 231 2024-09-18T00:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796692+0000", 2024-09-18T00:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796692+0000", 2024-09-18T00:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796694+0000", 2024-09-18T00:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.291e-06 2024-09-18T00:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796736+0000", 2024-09-18T00:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2082999999999997e-05 2024-09-18T00:18:12.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796747+0000", 2024-09-18T00:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1144999999999999e-05 2024-09-18T00:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796752+0000", 2024-09-18T00:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5439999999999998e-06 2024-09-18T00:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796844+0000", 2024-09-18T00:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1847000000000003e-05 2024-09-18T00:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796952+0000", 2024-09-18T00:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108223 2024-09-18T00:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.797040+0000", 2024-09-18T00:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7416000000000005e-05 2024-09-18T00:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.802075+0000", 2024-09-18T00:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050354390000000001 2024-09-18T00:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.804151+0000", 2024-09-18T00:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0020752769999999999 2024-09-18T00:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.804314+0000", 2024-09-18T00:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016367899999999999 2024-09-18T00:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.804434+0000", 2024-09-18T00:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119646 2024-09-18T00:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.137 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:12.138 INFO:teuthology.orchestra.run.smithi097.stdout:194 write 0xe7bce65 thru 0xe7cbec2 (0xf05e bytes) 2024-09-18T00:18:12.139 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout:195 punch from 0xa6f7c67 to 0xa6f875c, (0xaf5 bytes) 2024-09-18T00:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout:196 trunc from 0xe7cbec3 to 0x64ead0c 2024-09-18T00:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout:197 write 0xeef0a2 thru 0xef4197 (0x50f6 bytes) 2024-09-18T00:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout:198 punch from 0x3507ae3 to 0x35097ed, (0x1d0a bytes) 2024-09-18T00:18:12.140 INFO:teuthology.orchestra.run.smithi097.stdout:199 read 0xeadc19 thru 0xeaf633 (0x1a1b bytes) 2024-09-18T00:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout:201 writesame 0xd3c6d1 thru 0xd3fc19 data_size 0x1(0x3549 bytes) 2024-09-18T00:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout:202 read 0x1b0e25e thru 0x1b149b6 (0x6759 bytes) 2024-09-18T00:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout:204 trunc from 0x64ead0c to 0x7257630 2024-09-18T00:18:12.141 INFO:teuthology.orchestra.run.smithi097.stdout:205 trunc from 0x7257630 to 0x32ac891 2024-09-18T00:18:12.141 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:12.135+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:12.164 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:12.164 INFO:teuthology.orchestra.run.smithi097.stdout:207 read 0x2ea92c6 thru 0x2eabe91 (0x2bcc bytes) 2024-09-18T00:18:12.164 INFO:teuthology.orchestra.run.smithi097.stdout:208 write 0x185ae76 thru 0x186234d (0x74d8 bytes) 2024-09-18T00:18:12.175 INFO:teuthology.orchestra.run.smithi097.stdout:209 punch from 0xbbaad1 to 0xbbca61, (0x1f90 bytes) 2024-09-18T00:18:12.187 INFO:teuthology.orchestra.run.smithi097.stdout:210 trunc from 0x32ac891 to 0x96a434a 2024-09-18T00:18:12.196 INFO:teuthology.orchestra.run.smithi097.stdout:211 write 0x6750d6b thru 0x6755a79 (0x4d0f bytes) 2024-09-18T00:18:12.205 INFO:teuthology.orchestra.run.smithi097.stdout:212 read 0x879f2b0 thru 0x87a65bd (0x730e bytes) 2024-09-18T00:18:12.205 INFO:teuthology.orchestra.run.smithi097.stdout:215 write 0x7ab547c thru 0x7abbf06 (0x6a8b bytes) 2024-09-18T00:18:12.215 INFO:teuthology.orchestra.run.smithi097.stdout:218 punch from 0x39a705e to 0x39aa1b0, (0x3152 bytes) 2024-09-18T00:18:12.215 INFO:teuthology.orchestra.run.smithi097.stdout:219 punch from 0x5a9d8c8 to 0x5a9ea12, (0x114a bytes) 2024-09-18T00:18:12.215 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:221 writesame 0xeaee56b thru 0xeafae8a data_size 0x10(0xc920 bytes) 2024-09-18T00:18:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:224 punch from 0x2636668 to 0x263fb69, (0x9501 bytes) 2024-09-18T00:18:12.233 INFO:teuthology.orchestra.run.smithi097.stdout:225 punch from 0x260008 to 0x2640b7, (0x40af bytes) 2024-09-18T00:18:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:226 punch from 0xd9981a2 to 0xd99e0d8, (0x5f36 bytes) 2024-09-18T00:18:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:227 read 0x58074ec thru 0x580c2b5 (0x4dca bytes) 2024-09-18T00:18:12.234 INFO:teuthology.orchestra.run.smithi097.stdout:229 writesame 0x2c621f5 thru 0x2c64d1a data_size 0x15(0x2b26 bytes) 2024-09-18T00:18:12.239 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:12.235+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:12.261 INFO:teuthology.orchestra.run.smithi097.stdout:230 trunc from 0xeafae8b to 0x7a50d87 2024-09-18T00:18:12.279 INFO:teuthology.orchestra.run.smithi097.stdout:231 read 0x9fa1e9 thru 0x9fb2e9 (0x1101 bytes) 2024-09-18T00:18:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:232 read 0x7421f62 thru 0x742c863 (0xa902 bytes) 2024-09-18T00:18:12.298 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:12.298 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:12.298 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:12.298 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:12.301 INFO:teuthology.orchestra.run.smithi097.stdout:233 write 0xbda5440 thru 0xbdb23e5 (0xcfa6 bytes) 2024-09-18T00:18:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:234 read 0x2388445 thru 0x238d5ed (0x51a9 bytes) 2024-09-18T00:18:12.309 INFO:teuthology.orchestra.run.smithi097.stdout:235 read 0x13721cd thru 0x1379782 (0x75b6 bytes) 2024-09-18T00:18:12.310 INFO:teuthology.orchestra.run.smithi097.stdout:236 writesame 0x4ec5466 thru 0x4ec5515 data_size 0x8(0xb0 bytes) 2024-09-18T00:18:12.313 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:18:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:237 punch from 0x8b788d0 to 0x8b78902, (0x32 bytes) 2024-09-18T00:18:12.324 INFO:teuthology.orchestra.run.smithi097.stdout:238 trunc from 0xbdb23e6 to 0x7b5d05d 2024-09-18T00:18:12.339 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:12.335+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:12.344 INFO:teuthology.orchestra.run.smithi097.stdout:239 writesame 0x3de686c thru 0x3de850b data_size 0x10(0x1ca0 bytes) 2024-09-18T00:18:12.372 INFO:teuthology.orchestra.run.smithi097.stdout:241 write 0xe665f93 thru 0xe66ebfd (0x8c6b bytes) 2024-09-18T00:18:12.377 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:12.377 INFO:teuthology.orchestra.run.smithi097.stdout:243 writesame 0x94e2386 thru 0x94e42f3 data_size 0x1b(0x1f6e bytes) 2024-09-18T00:18:12.384 INFO:teuthology.orchestra.run.smithi097.stdout:244 punch from 0x9c14238 to 0x9c1e905, (0xa6cd bytes) 2024-09-18T00:18:12.384 INFO:teuthology.orchestra.run.smithi097.stdout:245 writesame 0x2ad442 thru 0x2b790d data_size 0x6a(0xa4cc bytes) 2024-09-18T00:18:12.394 INFO:teuthology.orchestra.run.smithi097.stdout:246 write 0x2733025 thru 0x2740828 (0xd804 bytes) 2024-09-18T00:18:12.411 INFO:teuthology.orchestra.run.smithi097.stdout:247 read 0x8c35cb1 thru 0x8c3a600 (0x4950 bytes) 2024-09-18T00:18:12.411 INFO:teuthology.orchestra.run.smithi097.stdout:248 writesame 0xabd313f thru 0xabd9153 data_size 0x9(0x6015 bytes) 2024-09-18T00:18:12.416 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:18:12.441 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:12.435+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:12.445 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:12.445 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:12.445 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:12.445 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:12.445 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:12.450 INFO:teuthology.orchestra.run.smithi097.stdout:249 trunc from 0xe66ebfe to 0x24cb27b 2024-09-18T00:18:12.465 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:18:12.489 INFO:teuthology.orchestra.run.smithi097.stdout:250 punch from 0x1dad110 to 0x1dad1ac, (0x9c bytes) 2024-09-18T00:18:12.501 INFO:teuthology.orchestra.run.smithi097.stdout:251 write 0x40924ec thru 0x409e79e (0xc2b3 bytes) 2024-09-18T00:18:12.511 INFO:teuthology.orchestra.run.smithi097.stdout:252 write 0x60e792 thru 0x616ff9 (0x8868 bytes) 2024-09-18T00:18:12.522 INFO:teuthology.orchestra.run.smithi097.stdout:255 punch from 0xd117d6 to 0xd1c38b, (0xabb5 bytes) 2024-09-18T00:18:12.522 INFO:teuthology.orchestra.run.smithi097.stdout:256 read 0x139ebcb thru 0x13adff4 (0xf42a bytes) 2024-09-18T00:18:12.539 INFO:teuthology.orchestra.run.smithi097.stdout:257 write 0x9761c62 thru 0x976c2ee (0xa68d bytes) 2024-09-18T00:18:12.541 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:12.539+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:12.545 INFO:teuthology.orchestra.run.smithi097.stdout:258 read 0x12d29bb thru 0x12d8cda (0x6320 bytes) 2024-09-18T00:18:12.553 INFO:teuthology.orchestra.run.smithi097.stdout:259 trunc from 0x976c2ef to 0x520c87f 2024-09-18T00:18:12.570 INFO:teuthology.orchestra.run.smithi097.stdout:263 writesame 0x382eb08 thru 0x383ac08 data_size 0x1(0xc101 bytes) 2024-09-18T00:18:12.616 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:220 3.19 3:9e8c0c63:::rbd_data.112a45c8e167.0000000000000031:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T00:18:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.761391+0000", 2024-09-18T00:18:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.854394544, 2024-09-18T00:18:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062183000000000004, 2024-09-18T00:18:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:12.617 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 220 2024-09-18T00:18:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761391+0000", 2024-09-18T00:18:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761391+0000", 2024-09-18T00:18:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761392+0000", 2024-09-18T00:18:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.838e-06 2024-09-18T00:18:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761407+0000", 2024-09-18T00:18:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4538999999999999e-05 2024-09-18T00:18:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761421+0000", 2024-09-18T00:18:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3896e-05 2024-09-18T00:18:12.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761427+0000", 2024-09-18T00:18:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2519999999999996e-06 2024-09-18T00:18:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761507+0000", 2024-09-18T00:18:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9909999999999999e-05 2024-09-18T00:18:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761697+0000", 2024-09-18T00:18:12.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000189834 2024-09-18T00:18:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-18T00:18:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.762095+0000", 2024-09-18T00:18:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00039823799999999998 2024-09-18T00:18:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767289+0000", 2024-09-18T00:18:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0051936930000000001 2024-09-18T00:18:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767536+0000", 2024-09-18T00:18:12.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024718899999999998 2024-09-18T00:18:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767573+0000", 2024-09-18T00:18:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6792000000000003e-05 2024-09-18T00:18:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767609+0000", 2024-09-18T00:18:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6118999999999998e-05 2024-09-18T00:18:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:220 3.19 e28/23 ondisk, result = 0)", 2024-09-18T00:18:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.763702+0000", 2024-09-18T00:18:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.852083087, 2024-09-18T00:18:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035976879999999999, 2024-09-18T00:18:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763702+0000", 2024-09-18T00:18:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763702+0000", 2024-09-18T00:18:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763703+0000", 2024-09-18T00:18:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5099999999999999e-07 2024-09-18T00:18:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763707+0000", 2024-09-18T00:18:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7450000000000003e-06 2024-09-18T00:18:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763720+0000", 2024-09-18T00:18:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2215000000000001e-05 2024-09-18T00:18:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763725+0000", 2024-09-18T00:18:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.604e-06 2024-09-18T00:18:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767267+0000", 2024-09-18T00:18:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035417729999999998 2024-09-18T00:18:12.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767279+0000", 2024-09-18T00:18:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2267e-05 2024-09-18T00:18:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767300+0000", 2024-09-18T00:18:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0333e-05 2024-09-18T00:18:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:372 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.457128+0000", 2024-09-18T00:18:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.158656621, 2024-09-18T00:18:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003527082, 2024-09-18T00:18:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 372 2024-09-18T00:18:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457128+0000", 2024-09-18T00:18:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457128+0000", 2024-09-18T00:18:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457131+0000", 2024-09-18T00:18:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9739999999999998e-06 2024-09-18T00:18:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457146+0000", 2024-09-18T00:18:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4516e-05 2024-09-18T00:18:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457178+0000", 2024-09-18T00:18:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2475999999999999e-05 2024-09-18T00:18:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457199+0000", 2024-09-18T00:18:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:18:12.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457348+0000", 2024-09-18T00:18:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014881200000000001 2024-09-18T00:18:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457602+0000", 2024-09-18T00:18:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000253732 2024-09-18T00:18:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457977+0000", 2024-09-18T00:18:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037547299999999997 2024-09-18T00:18:12.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.459553+0000", 2024-09-18T00:18:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001576261 2024-09-18T00:18:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460497+0000", 2024-09-18T00:18:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094395500000000001 2024-09-18T00:18:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460588+0000", 2024-09-18T00:18:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0443000000000001e-05 2024-09-18T00:18:12.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460656+0000", 2024-09-18T00:18:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7650000000000005e-05 2024-09-18T00:18:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:373 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.set_stripe_unit_count in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.460868+0000", 2024-09-18T00:18:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.154916713, 2024-09-18T00:18:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031188050000000001, 2024-09-18T00:18:12.632 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 373 2024-09-18T00:18:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460868+0000", 2024-09-18T00:18:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460868+0000", 2024-09-18T00:18:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460870+0000", 2024-09-18T00:18:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.733e-06 2024-09-18T00:18:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460880+0000", 2024-09-18T00:18:12.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5429999999999999e-06 2024-09-18T00:18:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460913+0000", 2024-09-18T00:18:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3333000000000001e-05 2024-09-18T00:18:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460937+0000", 2024-09-18T00:18:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3836999999999999e-05 2024-09-18T00:18:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.461083+0000", 2024-09-18T00:18:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145952 2024-09-18T00:18:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.461327+0000", 2024-09-18T00:18:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024425100000000002 2024-09-18T00:18:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.461895+0000", 2024-09-18T00:18:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056844600000000001 2024-09-18T00:18:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.462901+0000", 2024-09-18T00:18:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001005939 2024-09-18T00:18:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.463800+0000", 2024-09-18T00:18:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00089826999999999995 2024-09-18T00:18:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.463885+0000", 2024-09-18T00:18:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5768999999999998e-05 2024-09-18T00:18:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.463987+0000", 2024-09-18T00:18:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010173199999999999 2024-09-18T00:18:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.641 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:387 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [watch watch cookie 140686016265056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.485197+0000", 2024-09-18T00:18:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.1305883889999999, 2024-09-18T00:18:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029145590000000002, 2024-09-18T00:18:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:12.642 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 387 2024-09-18T00:18:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485197+0000", 2024-09-18T00:18:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485197+0000", 2024-09-18T00:18:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485199+0000", 2024-09-18T00:18:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4849999999999999e-06 2024-09-18T00:18:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485209+0000", 2024-09-18T00:18:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0251999999999999e-05 2024-09-18T00:18:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485238+0000", 2024-09-18T00:18:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8708e-05 2024-09-18T00:18:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485255+0000", 2024-09-18T00:18:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7011e-05 2024-09-18T00:18:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485400+0000", 2024-09-18T00:18:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144504 2024-09-18T00:18:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485654+0000", 2024-09-18T00:18:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025417400000000002 2024-09-18T00:18:12.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.486056+0000", 2024-09-18T00:18:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000402147 2024-09-18T00:18:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.487262+0000", 2024-09-18T00:18:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0012063480000000001 2024-09-18T00:18:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.487908+0000", 2024-09-18T00:18:12.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00064552100000000003 2024-09-18T00:18:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.487993+0000", 2024-09-18T00:18:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5263e-05 2024-09-18T00:18:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488111+0000", 2024-09-18T00:18:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000118146 2024-09-18T00:18:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.647 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:388 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.parent_attach in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.488320+0000", 2024-09-18T00:18:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.1274650390000001, 2024-09-18T00:18:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0030081399999999999, 2024-09-18T00:18:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:12.648 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 388 2024-09-18T00:18:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488320+0000", 2024-09-18T00:18:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488320+0000", 2024-09-18T00:18:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488322+0000", 2024-09-18T00:18:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-06 2024-09-18T00:18:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488331+0000", 2024-09-18T00:18:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5880000000000004e-06 2024-09-18T00:18:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488356+0000", 2024-09-18T00:18:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4403e-05 2024-09-18T00:18:12.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488368+0000", 2024-09-18T00:18:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2690999999999999e-05 2024-09-18T00:18:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488485+0000", 2024-09-18T00:18:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011662099999999999 2024-09-18T00:18:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488623+0000", 2024-09-18T00:18:12.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000137472 2024-09-18T00:18:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.489032+0000", 2024-09-18T00:18:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00040919699999999998 2024-09-18T00:18:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.490129+0000", 2024-09-18T00:18:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001096765 2024-09-18T00:18:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:12.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.491175+0000", 2024-09-18T00:18:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0010461380000000001 2024-09-18T00:18:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.491259+0000", 2024-09-18T00:18:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4625999999999999e-05 2024-09-18T00:18:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.491328+0000", 2024-09-18T00:18:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8895999999999999e-05 2024-09-18T00:18:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:397 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [watch unwatch cookie 140686016265056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.500193+0000", 2024-09-18T00:18:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.115591767, 2024-09-18T00:18:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002572633, 2024-09-18T00:18:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 397 2024-09-18T00:18:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.654 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.500193+0000", 2024-09-18T00:18:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.500193+0000", 2024-09-18T00:18:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.500195+0000", 2024-09-18T00:18:12.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.517e-06 2024-09-18T00:18:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.500203+0000", 2024-09-18T00:18:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0900000000000005e-06 2024-09-18T00:18:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.500227+0000", 2024-09-18T00:18:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3926000000000001e-05 2024-09-18T00:18:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.500241+0000", 2024-09-18T00:18:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.435e-05 2024-09-18T00:18:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.500344+0000", 2024-09-18T00:18:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010270800000000001 2024-09-18T00:18:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.500466+0000", 2024-09-18T00:18:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012216500000000001 2024-09-18T00:18:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.500667+0000", 2024-09-18T00:18:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020112900000000001 2024-09-18T00:18:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.501971+0000", 2024-09-18T00:18:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0013036250000000001 2024-09-18T00:18:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.502568+0000", 2024-09-18T00:18:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000597655 2024-09-18T00:18:12.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.502658+0000", 2024-09-18T00:18:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9763000000000001e-05 2024-09-18T00:18:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.502766+0000", 2024-09-18T00:18:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107705 2024-09-18T00:18:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:531 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.840412+0000", 2024-09-18T00:18:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.77537269499999995, 2024-09-18T00:18:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025285210000000002, 2024-09-18T00:18:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 531 2024-09-18T00:18:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.660 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.840412+0000", 2024-09-18T00:18:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.840412+0000", 2024-09-18T00:18:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.840415+0000", 2024-09-18T00:18:12.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9000000000000002e-06 2024-09-18T00:18:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.840430+0000", 2024-09-18T00:18:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4871e-05 2024-09-18T00:18:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.840461+0000", 2024-09-18T00:18:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0595000000000001e-05 2024-09-18T00:18:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.840475+0000", 2024-09-18T00:18:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4703e-05 2024-09-18T00:18:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.840627+0000", 2024-09-18T00:18:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000151495 2024-09-18T00:18:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.840785+0000", 2024-09-18T00:18:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015812899999999999 2024-09-18T00:18:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.841121+0000", 2024-09-18T00:18:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033605700000000001 2024-09-18T00:18:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.842068+0000", 2024-09-18T00:18:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094686599999999998 2024-09-18T00:18:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.842864+0000", 2024-09-18T00:18:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00079647799999999999 2024-09-18T00:18:12.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.842905+0000", 2024-09-18T00:18:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0099999999999999e-05 2024-09-18T00:18:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.842941+0000", 2024-09-18T00:18:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6327e-05 2024-09-18T00:18:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.665 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:542 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.859144+0000", 2024-09-18T00:18:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.75664062399999998, 2024-09-18T00:18:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025482859999999999, 2024-09-18T00:18:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 542 2024-09-18T00:18:12.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.859144+0000", 2024-09-18T00:18:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.859144+0000", 2024-09-18T00:18:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.859147+0000", 2024-09-18T00:18:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6979999999999998e-06 2024-09-18T00:18:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.859160+0000", 2024-09-18T00:18:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3035e-05 2024-09-18T00:18:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.859188+0000", 2024-09-18T00:18:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7985999999999999e-05 2024-09-18T00:18:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.859202+0000", 2024-09-18T00:18:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4243000000000001e-05 2024-09-18T00:18:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.859322+0000", 2024-09-18T00:18:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119787 2024-09-18T00:18:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.859475+0000", 2024-09-18T00:18:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000153108 2024-09-18T00:18:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.859807+0000", 2024-09-18T00:18:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033201500000000001 2024-09-18T00:18:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.860930+0000", 2024-09-18T00:18:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001122818 2024-09-18T00:18:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.861609+0000", 2024-09-18T00:18:12.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00067914799999999997 2024-09-18T00:18:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.861653+0000", 2024-09-18T00:18:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3300999999999998e-05 2024-09-18T00:18:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.861693+0000", 2024-09-18T00:18:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0147e-05 2024-09-18T00:18:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:554 3.8 e29/23 3:14210294:::rbd_data.112ae20746f5.0000000000000133:head v 29'1, mlcod=0'0)", 2024-09-18T00:18:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.879349+0000", 2024-09-18T00:18:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.736436117, 2024-09-18T00:18:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043514469999999996, 2024-09-18T00:18:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879349+0000", 2024-09-18T00:18:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879349+0000", 2024-09-18T00:18:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879351+0000", 2024-09-18T00:18:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8050000000000001e-06 2024-09-18T00:18:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879392+0000", 2024-09-18T00:18:12.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1054999999999999e-05 2024-09-18T00:18:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879413+0000", 2024-09-18T00:18:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0784999999999998e-05 2024-09-18T00:18:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879425+0000", 2024-09-18T00:18:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2896e-05 2024-09-18T00:18:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879540+0000", 2024-09-18T00:18:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011473500000000001 2024-09-18T00:18:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879579+0000", 2024-09-18T00:18:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8578999999999998e-05 2024-09-18T00:18:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.883623+0000", 2024-09-18T00:18:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040441959999999999 2024-09-18T00:18:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.883700+0000", 2024-09-18T00:18:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7396000000000002e-05 2024-09-18T00:18:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:557 3.e e29/27 3:736a090f:::rbd_data.112ae20746f5.000000000000008e:head v 29'6, mlcod=28'4)", 2024-09-18T00:18:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.890014+0000", 2024-09-18T00:18:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.72577077899999998, 2024-09-18T00:18:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002682841, 2024-09-18T00:18:12.676 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.890014+0000", 2024-09-18T00:18:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.890014+0000", 2024-09-18T00:18:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.890017+0000", 2024-09-18T00:18:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4399999999999999e-06 2024-09-18T00:18:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.890202+0000", 2024-09-18T00:18:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018565899999999999 2024-09-18T00:18:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.890232+0000", 2024-09-18T00:18:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9416000000000001e-05 2024-09-18T00:18:12.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.890246+0000", 2024-09-18T00:18:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4207e-05 2024-09-18T00:18:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.890369+0000", 2024-09-18T00:18:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012293199999999999 2024-09-18T00:18:12.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.890413+0000", 2024-09-18T00:18:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4545000000000001e-05 2024-09-18T00:18:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.892644+0000", 2024-09-18T00:18:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022309869999999998 2024-09-18T00:18:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.892697+0000", 2024-09-18T00:18:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2655000000000003e-05 2024-09-18T00:18:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:612 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.044307+0000", 2024-09-18T00:18:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.57147789000000004, 2024-09-18T00:18:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027702650000000001, 2024-09-18T00:18:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.681 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 612 2024-09-18T00:18:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.044307+0000", 2024-09-18T00:18:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.044307+0000", 2024-09-18T00:18:12.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.044311+0000", 2024-09-18T00:18:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4800000000000001e-06 2024-09-18T00:18:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.044326+0000", 2024-09-18T00:18:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5440000000000001e-05 2024-09-18T00:18:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.044363+0000", 2024-09-18T00:18:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7073000000000002e-05 2024-09-18T00:18:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.044381+0000", 2024-09-18T00:18:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7606999999999999e-05 2024-09-18T00:18:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.044505+0000", 2024-09-18T00:18:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012385200000000001 2024-09-18T00:18:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.044662+0000", 2024-09-18T00:18:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015708599999999999 2024-09-18T00:18:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.045002+0000", 2024-09-18T00:18:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034022000000000002 2024-09-18T00:18:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.046136+0000", 2024-09-18T00:18:12.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001133974 2024-09-18T00:18:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.046898+0000", 2024-09-18T00:18:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00076233200000000001 2024-09-18T00:18:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.046995+0000", 2024-09-18T00:18:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6695999999999997e-05 2024-09-18T00:18:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.047077+0000", 2024-09-18T00:18:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2504999999999997e-05 2024-09-18T00:18:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:668 3.1a e29/23 3:59ee8e8b:::rbd_data.112ae20746f5.0000000000000031:head v 29'3, mlcod=25'1)", 2024-09-18T00:18:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.170930+0000", 2024-09-18T00:18:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.44485497800000001, 2024-09-18T00:18:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052043310000000004, 2024-09-18T00:18:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.687 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.170930+0000", 2024-09-18T00:18:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.170930+0000", 2024-09-18T00:18:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.170932+0000", 2024-09-18T00:18:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4699999999999999e-06 2024-09-18T00:18:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171038+0000", 2024-09-18T00:18:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106205 2024-09-18T00:18:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171061+0000", 2024-09-18T00:18:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3329999999999999e-05 2024-09-18T00:18:12.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171072+0000", 2024-09-18T00:18:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0806e-05 2024-09-18T00:18:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171201+0000", 2024-09-18T00:18:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012925799999999999 2024-09-18T00:18:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171239+0000", 2024-09-18T00:18:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7866e-05 2024-09-18T00:18:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.176016+0000", 2024-09-18T00:18:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047769930000000002 2024-09-18T00:18:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.176134+0000", 2024-09-18T00:18:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000118403 2024-09-18T00:18:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:692 3.0 3:03fab4ad:::rbd_data.112ae20746f5.0000000000000057:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.238016+0000", 2024-09-18T00:18:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.37776926799999999, 2024-09-18T00:18:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0087112969999999998, 2024-09-18T00:18:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:12.692 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 692 2024-09-18T00:18:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238016+0000", 2024-09-18T00:18:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238016+0000", 2024-09-18T00:18:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238017+0000", 2024-09-18T00:18:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.257e-06 2024-09-18T00:18:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238023+0000", 2024-09-18T00:18:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5470000000000004e-06 2024-09-18T00:18:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238046+0000", 2024-09-18T00:18:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3088e-05 2024-09-18T00:18:12.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238054+0000", 2024-09-18T00:18:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3569999999999998e-06 2024-09-18T00:18:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238135+0000", 2024-09-18T00:18:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0814000000000002e-05 2024-09-18T00:18:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238331+0000", 2024-09-18T00:18:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019594 2024-09-18T00:18:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238606+0000", 2024-09-18T00:18:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027473200000000003 2024-09-18T00:18:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.240010+0000", 2024-09-18T00:18:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001404329 2024-09-18T00:18:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246563+0000", 2024-09-18T00:18:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0065535630000000001 2024-09-18T00:18:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246643+0000", 2024-09-18T00:18:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9468999999999994e-05 2024-09-18T00:18:12.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246727+0000", 2024-09-18T00:18:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4201000000000002e-05 2024-09-18T00:18:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:694 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.266659+0000", 2024-09-18T00:18:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.34912621599999999, 2024-09-18T00:18:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002821919, 2024-09-18T00:18:12.698 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 694 2024-09-18T00:18:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.266659+0000", 2024-09-18T00:18:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.266659+0000", 2024-09-18T00:18:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.266662+0000", 2024-09-18T00:18:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1970000000000001e-06 2024-09-18T00:18:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.266677+0000", 2024-09-18T00:18:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5418999999999999e-05 2024-09-18T00:18:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.266712+0000", 2024-09-18T00:18:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4944000000000003e-05 2024-09-18T00:18:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.266731+0000", 2024-09-18T00:18:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8559999999999998e-05 2024-09-18T00:18:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.266860+0000", 2024-09-18T00:18:12.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012881500000000001 2024-09-18T00:18:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.267021+0000", 2024-09-18T00:18:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016144900000000001 2024-09-18T00:18:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.267393+0000", 2024-09-18T00:18:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037151399999999998 2024-09-18T00:18:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:12.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.268580+0000", 2024-09-18T00:18:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001186868 2024-09-18T00:18:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.269186+0000", 2024-09-18T00:18:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060663500000000001 2024-09-18T00:18:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.269352+0000", 2024-09-18T00:18:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016532499999999999 2024-09-18T00:18:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.269481+0000", 2024-09-18T00:18:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012919299999999999 2024-09-18T00:18:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:704 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.293962+0000", 2024-09-18T00:18:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.32182277999999997, 2024-09-18T00:18:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028652980000000001, 2024-09-18T00:18:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.704 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 704 2024-09-18T00:18:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.293962+0000", 2024-09-18T00:18:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.293962+0000", 2024-09-18T00:18:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.293966+0000", 2024-09-18T00:18:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4800000000000001e-06 2024-09-18T00:18:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.293980+0000", 2024-09-18T00:18:12.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4399999999999999e-05 2024-09-18T00:18:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.294017+0000", 2024-09-18T00:18:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6755999999999998e-05 2024-09-18T00:18:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.294035+0000", 2024-09-18T00:18:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7895000000000001e-05 2024-09-18T00:18:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.294156+0000", 2024-09-18T00:18:12.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012162700000000001 2024-09-18T00:18:12.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.294302+0000", 2024-09-18T00:18:12.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145119 2024-09-18T00:18:12.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:12.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.294591+0000", 2024-09-18T00:18:12.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000289924 2024-09-18T00:18:12.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:12.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.295933+0000", 2024-09-18T00:18:12.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00134128 2024-09-18T00:18:12.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:12.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.296528+0000", 2024-09-18T00:18:12.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00059507899999999999 2024-09-18T00:18:12.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.296693+0000", 2024-09-18T00:18:12.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016479600000000001 2024-09-18T00:18:12.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.296828+0000", 2024-09-18T00:18:12.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013494200000000001 2024-09-18T00:18:12.710 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.710 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.710 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.710 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:733 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:12.710 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.365564+0000", 2024-09-18T00:18:12.710 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.250221468, 2024-09-18T00:18:12.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038647540000000002, 2024-09-18T00:18:12.711 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.711 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.711 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:12.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:12.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:12.711 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 733 2024-09-18T00:18:12.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.711 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365564+0000", 2024-09-18T00:18:12.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365564+0000", 2024-09-18T00:18:12.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365566+0000", 2024-09-18T00:18:12.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1909999999999999e-06 2024-09-18T00:18:12.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365589+0000", 2024-09-18T00:18:12.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3096999999999999e-05 2024-09-18T00:18:12.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365616+0000", 2024-09-18T00:18:12.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7577e-05 2024-09-18T00:18:12.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365628+0000", 2024-09-18T00:18:12.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1804000000000001e-05 2024-09-18T00:18:12.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365708+0000", 2024-09-18T00:18:12.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9657e-05 2024-09-18T00:18:12.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365797+0000", 2024-09-18T00:18:12.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8812000000000003e-05 2024-09-18T00:18:12.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:12.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365976+0000", 2024-09-18T00:18:12.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000178853 2024-09-18T00:18:12.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:12.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.367005+0000", 2024-09-18T00:18:12.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001029196 2024-09-18T00:18:12.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:12.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.369312+0000", 2024-09-18T00:18:12.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0023069079999999999 2024-09-18T00:18:12.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.369375+0000", 2024-09-18T00:18:12.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2823000000000005e-05 2024-09-18T00:18:12.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.369428+0000", 2024-09-18T00:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3835999999999998e-05 2024-09-18T00:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:746 3.0 3:03157719:::rbd_data.112ae20746f5.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.389598+0000", 2024-09-18T00:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.22618750100000001, 2024-09-18T00:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006052516, 2024-09-18T00:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 746 2024-09-18T00:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389598+0000", 2024-09-18T00:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389598+0000", 2024-09-18T00:18:12.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389599+0000", 2024-09-18T00:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.844e-06 2024-09-18T00:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389609+0000", 2024-09-18T00:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1039999999999996e-06 2024-09-18T00:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389623+0000", 2024-09-18T00:18:12.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4384e-05 2024-09-18T00:18:12.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389632+0000", 2024-09-18T00:18:12.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5589999999999994e-06 2024-09-18T00:18:12.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389715+0000", 2024-09-18T00:18:12.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2185e-05 2024-09-18T00:18:12.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389923+0000", 2024-09-18T00:18:12.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000208214 2024-09-18T00:18:12.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:12.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.390263+0000", 2024-09-18T00:18:12.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033984 2024-09-18T00:18:12.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:12.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392457+0000", 2024-09-18T00:18:12.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021944180000000001 2024-09-18T00:18:12.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:12.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395565+0000", 2024-09-18T00:18:12.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031083450000000002 2024-09-18T00:18:12.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395616+0000", 2024-09-18T00:18:12.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0949000000000003e-05 2024-09-18T00:18:12.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395650+0000", 2024-09-18T00:18:12.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3674000000000003e-05 2024-09-18T00:18:12.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.722 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.722 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:749 3.1a e29/23 3:5e11336d:::rbd_data.112ae20746f5.000000000000004e:head v 29'4, mlcod=26'2)", 2024-09-18T00:18:12.722 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.403303+0000", 2024-09-18T00:18:12.722 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.212482531, 2024-09-18T00:18:12.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0044570729999999998, 2024-09-18T00:18:12.722 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.722 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.722 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.722 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403303+0000", 2024-09-18T00:18:12.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403303+0000", 2024-09-18T00:18:12.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403303+0000", 2024-09-18T00:18:12.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6300000000000004e-07 2024-09-18T00:18:12.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403446+0000", 2024-09-18T00:18:12.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014246500000000001 2024-09-18T00:18:12.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403461+0000", 2024-09-18T00:18:12.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4904e-05 2024-09-18T00:18:12.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403470+0000", 2024-09-18T00:18:12.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9020000000000005e-06 2024-09-18T00:18:12.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403582+0000", 2024-09-18T00:18:12.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112161 2024-09-18T00:18:12.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403610+0000", 2024-09-18T00:18:12.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7671e-05 2024-09-18T00:18:12.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.407712+0000", 2024-09-18T00:18:12.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041027449999999997 2024-09-18T00:18:12.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.407760+0000", 2024-09-18T00:18:12.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7361999999999998e-05 2024-09-18T00:18:12.726 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.726 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.726 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.726 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:754 3.1a e30/23 3:5e11336d:::rbd_data.112ae20746f5.000000000000004e:head v 30'5, mlcod=29'3)", 2024-09-18T00:18:12.726 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.459551+0000", 2024-09-18T00:18:12.726 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.15623431700000001, 2024-09-18T00:18:12.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029016850000000002, 2024-09-18T00:18:12.726 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:12.727 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:12.727 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:12.727 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:12.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:12.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459551+0000", 2024-09-18T00:18:12.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:12.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459551+0000", 2024-09-18T00:18:12.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:12.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:12.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459552+0000", 2024-09-18T00:18:12.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9300000000000006e-07 2024-09-18T00:18:12.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:12.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459557+0000", 2024-09-18T00:18:12.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3700000000000003e-06 2024-09-18T00:18:12.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:12.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459584+0000", 2024-09-18T00:18:12.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.709e-05 2024-09-18T00:18:12.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:12.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459593+0000", 2024-09-18T00:18:12.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4009999999999997e-06 2024-09-18T00:18:12.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:12.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459650+0000", 2024-09-18T00:18:12.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7436999999999999e-05 2024-09-18T00:18:12.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:12.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459671+0000", 2024-09-18T00:18:12.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0704999999999999e-05 2024-09-18T00:18:12.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:12.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.462395+0000", 2024-09-18T00:18:12.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027243990000000002 2024-09-18T00:18:12.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:12.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:12.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:12.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.462452+0000", 2024-09-18T00:18:12.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7290000000000002e-05 2024-09-18T00:18:12.730 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.730 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.730 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.730 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:12.730 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:12.731 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:12.732 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:18:12.733 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:12.727+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:12.733 INFO:teuthology.orchestra.run.smithi097.stdout:264 punch from 0x33f9250 to 0x3408816, (0xf5c6 bytes) 2024-09-18T00:18:12.733 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:12.751 INFO:teuthology.orchestra.run.smithi097.stdout:267 trunc from 0x520c87f to 0xd45c920 2024-09-18T00:18:12.769 INFO:teuthology.orchestra.run.smithi097.stdout:268 writesame 0xdb1f4b5 thru 0xdb2695f data_size 0x1(0x74ab bytes) 2024-09-18T00:18:12.769 INFO:teuthology.orchestra.run.smithi097.stdout:269 punch from 0x2b858a to 0x2bd089, (0x4aff bytes) 2024-09-18T00:18:12.769 INFO:teuthology.orchestra.run.smithi097.stdout:270 write 0x8a93eb6 thru 0x8a9a276 (0x63c1 bytes) 2024-09-18T00:18:12.776 INFO:teuthology.orchestra.run.smithi097.stdout:271 write 0x2b12529 thru 0x2b16875 (0x434d bytes) 2024-09-18T00:18:12.785 INFO:teuthology.orchestra.run.smithi097.stdout:272 write 0x6b7171 thru 0x6b8039 (0xec9 bytes) 2024-09-18T00:18:12.795 INFO:teuthology.orchestra.run.smithi097.stdout:273 read 0xb264c27 thru 0xb270e94 (0xc26e bytes) 2024-09-18T00:18:12.795 INFO:teuthology.orchestra.run.smithi097.stdout:276 punch from 0xbeb84ca to 0xbec669e, (0xe1d4 bytes) 2024-09-18T00:18:12.795 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:12.795 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:12.795 INFO:teuthology.orchestra.run.smithi097.stdout:280 read 0xb641997 thru 0xb64399f (0x2009 bytes) 2024-09-18T00:18:12.795 INFO:teuthology.orchestra.run.smithi097.stdout:281 writesame 0x66655da thru 0x6670726 data_size 0x1(0xb14d bytes) 2024-09-18T00:18:12.833 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:12.831+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:12.845 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:18:12.860 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:12.860 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:12.860 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:12.860 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:12.878 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:18:12.895 INFO:teuthology.orchestra.run.smithi097.stdout:285 punch from 0xb90a79e to 0xb90e262, (0x3ac4 bytes) 2024-09-18T00:18:12.895 INFO:teuthology.orchestra.run.smithi097.stdout:287 write 0x9a4ecc6 thru 0x9a4fd2b (0x1066 bytes) 2024-09-18T00:18:12.903 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:12.903 INFO:teuthology.orchestra.run.smithi097.stdout:290 read 0x31d8560 thru 0x31d8fe2 (0xa83 bytes) 2024-09-18T00:18:12.903 INFO:teuthology.orchestra.run.smithi097.stdout:291 write 0x4889df9 thru 0x488c9e0 (0x2be8 bytes) 2024-09-18T00:18:12.910 INFO:teuthology.orchestra.run.smithi097.stdout:292 punch from 0x76ccbed to 0x76dbf4b, (0xf35e bytes) 2024-09-18T00:18:12.910 INFO:teuthology.orchestra.run.smithi097.stdout:293 punch from 0x5f9d930 to 0x5fa865c, (0xad2c bytes) 2024-09-18T00:18:12.910 INFO:teuthology.orchestra.run.smithi097.stdout:294 write 0xcd9969c thru 0xcd9b7f2 (0x2157 bytes) 2024-09-18T00:18:12.920 INFO:teuthology.orchestra.run.smithi097.stdout:295 trunc from 0xdb26960 to 0xb8d61c9 2024-09-18T00:18:12.934 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:12.927+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:12.939 INFO:teuthology.orchestra.run.smithi097.stdout:298 read 0xe4190c thru 0xe4c7a2 (0xae97 bytes) 2024-09-18T00:18:12.939 INFO:teuthology.orchestra.run.smithi097.stdout:299 read 0x49c2ee3 thru 0x49c6b7d (0x3c9b bytes) 2024-09-18T00:18:12.939 INFO:teuthology.orchestra.run.smithi097.stdout:300 read 0x8ab1259 thru 0x8ab37fd (0x25a5 bytes) 2024-09-18T00:18:12.940 INFO:teuthology.orchestra.run.smithi097.stdout:304 read 0x9767aba thru 0x976901f (0x1566 bytes) 2024-09-18T00:18:12.940 INFO:teuthology.orchestra.run.smithi097.stdout:305 read 0x44d7db1 thru 0x44dd931 (0x5b81 bytes) 2024-09-18T00:18:12.940 INFO:teuthology.orchestra.run.smithi097.stdout:306 write 0x49b56e0 thru 0x49b6059 (0x97a bytes) 2024-09-18T00:18:12.946 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:12.946 INFO:teuthology.orchestra.run.smithi097.stdout:308 write 0x97e3a6 thru 0x98aa95 (0xc6f0 bytes) 2024-09-18T00:18:12.959 INFO:teuthology.orchestra.run.smithi097.stdout:309 punch from 0xedcea6 to 0xee93f0, (0xc54a bytes) 2024-09-18T00:18:12.959 INFO:teuthology.orchestra.run.smithi097.stdout:312 punch from 0xac37aad to 0xac3e683, (0x6bd6 bytes) 2024-09-18T00:18:12.960 INFO:teuthology.orchestra.run.smithi097.stdout:313 punch from 0x1e35420 to 0x1e44c85, (0xf865 bytes) 2024-09-18T00:18:12.960 INFO:teuthology.orchestra.run.smithi097.stdout:314 write 0x86d282b thru 0x86dbc89 (0x945f bytes) 2024-09-18T00:18:12.970 INFO:teuthology.orchestra.run.smithi097.stdout:315 read 0x19d8ff9 thru 0x19ddca7 (0x4caf bytes) 2024-09-18T00:18:12.970 INFO:teuthology.orchestra.run.smithi097.stdout:316 read 0x1f29898 thru 0x1f2cbe5 (0x334e bytes) 2024-09-18T00:18:12.970 INFO:teuthology.orchestra.run.smithi097.stdout:318 read 0x8fd2d77 thru 0x8fd7a3b (0x4cc5 bytes) 2024-09-18T00:18:12.970 INFO:teuthology.orchestra.run.smithi097.stdout:319 read 0x1ba5067 thru 0x1ba8a43 (0x39dd bytes) 2024-09-18T00:18:12.970 INFO:teuthology.orchestra.run.smithi097.stdout:320 punch from 0xb407214 to 0xb4147b1, (0xd59d bytes) 2024-09-18T00:18:12.970 INFO:teuthology.orchestra.run.smithi097.stdout:321 write 0x3d353e4 thru 0x3d3a0da (0x4cf7 bytes) 2024-09-18T00:18:12.978 INFO:teuthology.orchestra.run.smithi097.stdout:326 punch from 0x4c5918b to 0x4c67999, (0xe80e bytes) 2024-09-18T00:18:12.978 INFO:teuthology.orchestra.run.smithi097.stdout:328 write 0x77336e3 thru 0x77382b0 (0x4bce bytes) 2024-09-18T00:18:12.991 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:12.992 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:12.996 INFO:teuthology.orchestra.run.smithi097.stdout:329 write 0xeacc1da thru 0xeadac15 (0xea3c bytes) 2024-09-18T00:18:13.004 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:13.004 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:13.005 INFO:teuthology.orchestra.run.smithi097.stdout:332 punch from 0xdfc8725 to 0xdfce5a5, (0x5e80 bytes) 2024-09-18T00:18:13.005 INFO:teuthology.orchestra.run.smithi097.stdout:333 write 0x728e7e5 thru 0x729e6e6 (0xff02 bytes) 2024-09-18T00:18:13.009 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:18:13.009 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:18:13.013 INFO:teuthology.orchestra.run.smithi097.stdout:334 write 0xcc58f37 thru 0xcc681a6 (0xf270 bytes) 2024-09-18T00:18:13.018 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:13.018 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:13.019 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:13.019 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:13.027 INFO:teuthology.orchestra.run.smithi097.stdout:336 write 0x436dac5 thru 0x437c7eb (0xed27 bytes) 2024-09-18T00:18:13.027 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:13.027 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:13.033 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:18:13.036 INFO:teuthology.orchestra.run.smithi097.stdout:337 punch from 0xc54235f to 0xc5489ab, (0x664c bytes) 2024-09-18T00:18:13.036 INFO:teuthology.orchestra.run.smithi097.stdout:338 writesame 0x1ff6c35 thru 0x20068ff data_size 0x2b(0xfccb bytes) 2024-09-18T00:18:13.037 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:13.035+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:13.046 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:13.046 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:13.050 INFO:teuthology.orchestra.run.smithi097.stdout:339 trunc from 0xeadac16 to 0xb846bc5 2024-09-18T00:18:13.069 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:13.069 INFO:teuthology.orchestra.run.smithi097.stdout:340 write 0x313917a thru 0x313eb83 (0x5a0a bytes) 2024-09-18T00:18:13.086 INFO:teuthology.orchestra.run.smithi097.stdout:342 trunc from 0xb846bc5 to 0x6ddd4d8 2024-09-18T00:18:13.095 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:18:13.095 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:18:13.117 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:13.126 INFO:teuthology.orchestra.run.smithi097.stdout:343 write 0xdef4818 thru 0xdf00bc6 (0xc3af bytes) 2024-09-18T00:18:13.136 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:13.131+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:13.136 INFO:teuthology.orchestra.run.smithi097.stdout:345 punch from 0xc3065a7 to 0xc311fe7, (0xba40 bytes) 2024-09-18T00:18:13.136 INFO:teuthology.orchestra.run.smithi097.stdout:346 write 0x5e73e2a thru 0x5e79b23 (0x5cfa bytes) 2024-09-18T00:18:13.140 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:13.140 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:13.145 INFO:teuthology.orchestra.run.smithi097.stdout:347 writesame 0x53c5bbe thru 0x53c7577 data_size 0x25(0x19ba bytes) 2024-09-18T00:18:13.152 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:13.152 INFO:teuthology.orchestra.run.smithi097.stdout:350 write 0xb2c2699 thru 0xb2c5230 (0x2b98 bytes) 2024-09-18T00:18:13.160 INFO:teuthology.orchestra.run.smithi097.stdout:351 write 0xcb08af8 thru 0xcb1219f (0x96a8 bytes) 2024-09-18T00:18:13.160 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:13.160 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:13.171 INFO:teuthology.orchestra.run.smithi097.stdout:352 write 0xa6ab6 thru 0xaec4e (0x8199 bytes) 2024-09-18T00:18:13.174 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:18:13.178 INFO:teuthology.orchestra.run.smithi097.stdout:353 write 0x886bf98 thru 0x88760b8 (0xa121 bytes) 2024-09-18T00:18:13.191 INFO:teuthology.orchestra.run.smithi097.stdout:354 trunc from 0xdf00bc7 to 0xc3c8961 2024-09-18T00:18:13.211 INFO:teuthology.orchestra.run.smithi097.stdout:355 write 0x2b097bb thru 0x2b0e2a1 (0x4ae7 bytes) 2024-09-18T00:18:13.216 INFO:teuthology.orchestra.run.smithi097.stdout:357 writesame 0x7e86bbe thru 0x7e8a22b data_size 0x1(0x366e bytes) 2024-09-18T00:18:13.236 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:13.231+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:13.246 INFO:teuthology.orchestra.run.smithi097.stdout:358 write 0xd3a9853 thru 0xd3a9938 (0xe6 bytes) 2024-09-18T00:18:13.251 INFO:teuthology.orchestra.run.smithi097.stdout:359 read 0x1f2a734 thru 0x1f39ac9 (0xf396 bytes) 2024-09-18T00:18:13.255 INFO:teuthology.orchestra.run.smithi097.stdout:360 trunc from 0xd3a9939 to 0xaff0e34 2024-09-18T00:18:13.274 INFO:teuthology.orchestra.run.smithi097.stdout:361 flatten 2024-09-18T00:18:13.298 INFO:teuthology.orchestra.run.smithi097.stdout:362 writesame 0x65927e9 thru 0x65977d6 data_size 0xd(0x4fee bytes) 2024-09-18T00:18:13.305 INFO:teuthology.orchestra.run.smithi097.stdout:364 punch from 0x38a3f1d to 0x38a71bf, (0x32a2 bytes) 2024-09-18T00:18:13.305 INFO:teuthology.orchestra.run.smithi097.stdout:365 punch from 0x1015212 to 0x1015ce0, (0xace bytes) 2024-09-18T00:18:13.305 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:13.305 INFO:teuthology.orchestra.run.smithi097.stdout:367 punch from 0x82aaec4 to 0x82b656c, (0xb6a8 bytes) 2024-09-18T00:18:13.305 INFO:teuthology.orchestra.run.smithi097.stdout:368 read 0x23b0505 thru 0x23bd394 (0xce90 bytes) 2024-09-18T00:18:13.305 INFO:teuthology.orchestra.run.smithi097.stdout:369 write 0x78bf78a thru 0x78c644b (0x6cc2 bytes) 2024-09-18T00:18:13.306 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:13.306 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:13.315 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:13.315 INFO:teuthology.orchestra.run.smithi097.stdout:371 read 0x728a8a1 thru 0x72970ca (0xc82a bytes) 2024-09-18T00:18:13.315 INFO:teuthology.orchestra.run.smithi097.stdout:372 punch from 0xa2cb35d to 0xa2cfee8, (0x4b8b bytes) 2024-09-18T00:18:13.315 INFO:teuthology.orchestra.run.smithi097.stdout:373 writesame 0x51d49e5 thru 0x51dfb4a data_size 0x1d(0xb166 bytes) 2024-09-18T00:18:13.320 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:18:13.337 INFO:teuthology.orchestra.run.smithi097.stdout:374 write 0xb5954d7 thru 0xb5a2944 (0xd46e bytes) 2024-09-18T00:18:13.337 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:13.331+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:13.343 INFO:teuthology.orchestra.run.smithi097.stdout:376 read 0x2a3971 thru 0x2aa855 (0x6ee5 bytes) 2024-09-18T00:18:13.344 INFO:teuthology.orchestra.run.smithi097.stdout:379 read 0xbe3c5e thru 0xbe800f (0x43b2 bytes) 2024-09-18T00:18:13.354 INFO:teuthology.orchestra.run.smithi097.stdout:380 write 0xc560031 thru 0xc568742 (0x8712 bytes) 2024-09-18T00:18:13.362 INFO:teuthology.orchestra.run.smithi097.stdout:381 read 0x898a672 thru 0x898b9d6 (0x1365 bytes) 2024-09-18T00:18:13.362 INFO:teuthology.orchestra.run.smithi097.stdout:382 write 0xfa9162 thru 0xfab0db (0x1f7a bytes) 2024-09-18T00:18:13.368 INFO:teuthology.orchestra.run.smithi097.stdout:383 write 0x69fd4c thru 0x6aa64a (0xa8ff bytes) 2024-09-18T00:18:13.376 INFO:teuthology.orchestra.run.smithi097.stdout:384 write 0x42f82bc thru 0x42fe253 (0x5f98 bytes) 2024-09-18T00:18:13.382 INFO:teuthology.orchestra.run.smithi097.stdout:386 write 0x5f25adb thru 0x5f34166 (0xe68c bytes) 2024-09-18T00:18:13.394 INFO:teuthology.orchestra.run.smithi097.stdout:387 read 0x231d1f4 thru 0x2327075 (0x9e82 bytes) 2024-09-18T00:18:13.394 INFO:teuthology.orchestra.run.smithi097.stdout:388 writesame 0x75ed62a thru 0x75f7244 data_size 0xb(0x9c1b bytes) 2024-09-18T00:18:13.395 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is unset 2024-09-18T00:18:13.413 INFO:teuthology.orchestra.run.smithi097.stdout:390 read 0x8171a30 thru 0x8173c3e (0x220f bytes) 2024-09-18T00:18:13.413 INFO:teuthology.orchestra.run.smithi097.stdout:391 read 0x838434e thru 0x838b008 (0x6cbb bytes) 2024-09-18T00:18:13.413 INFO:teuthology.orchestra.run.smithi097.stdout:392 read 0xc402793 thru 0xc406aa1 (0x430f bytes) 2024-09-18T00:18:13.413 INFO:teuthology.orchestra.run.smithi097.stdout:393 trunc from 0xc568743 to 0xce26406 2024-09-18T00:18:13.422 INFO:teuthology.orchestra.run.smithi097.stdout:395 read 0x6732b73 thru 0x6737e95 (0x5323 bytes) 2024-09-18T00:18:13.431 INFO:teuthology.orchestra.run.smithi097.stdout:396 write 0xd039651 thru 0xd048291 (0xec41 bytes) 2024-09-18T00:18:13.438 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:13.435+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:13.440 INFO:teuthology.orchestra.run.smithi097.stdout:397 write 0x9eb8e39 thru 0x9ec1ac3 (0x8c8b bytes) 2024-09-18T00:18:13.460 INFO:teuthology.orchestra.run.smithi097.stdout:400 writesame 0xdde1ff0 thru 0xddf0136 data_size 0x1(0xe147 bytes) 2024-09-18T00:18:13.464 INFO:teuthology.orchestra.run.smithi097.stdout:401 trunc from 0xddf0137 to 0xb52db89 2024-09-18T00:18:13.475 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:13.475 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:13.486 INFO:teuthology.orchestra.run.smithi097.stdout:402 write 0x965a48d thru 0x966781e (0xd392 bytes) 2024-09-18T00:18:13.491 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:18:13.498 INFO:teuthology.orchestra.run.smithi097.stdout:403 write 0xf3c455 thru 0xf43350 (0x6efc bytes) 2024-09-18T00:18:13.508 INFO:teuthology.orchestra.run.smithi097.stdout:404 read 0xb3bef2b thru 0xb3ca1af (0xb285 bytes) 2024-09-18T00:18:13.508 INFO:teuthology.orchestra.run.smithi097.stdout:405 write 0x1694fa6 thru 0x16a30c0 (0xe11b bytes) 2024-09-18T00:18:13.522 INFO:teuthology.orchestra.run.smithi097.stdout:406 write 0x1514c5 thru 0x158897 (0x73d3 bytes) 2024-09-18T00:18:13.525 INFO:teuthology.orchestra.run.smithi097.stdout:408 punch from 0x68596dd to 0x686022e, (0x6b51 bytes) 2024-09-18T00:18:13.525 INFO:teuthology.orchestra.run.smithi097.stdout:409 read 0x9a34ed1 thru 0x9a43133 (0xe263 bytes) 2024-09-18T00:18:13.525 INFO:teuthology.orchestra.run.smithi097.stdout:410 punch from 0x5d0b16b to 0x5d15cd0, (0xab65 bytes) 2024-09-18T00:18:13.537 INFO:teuthology.orchestra.run.smithi097.stdout:412 write 0xd5ebc5c thru 0xd5f7390 (0xb735 bytes) 2024-09-18T00:18:13.539 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:13.535+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:13.546 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:13.546 INFO:teuthology.orchestra.run.smithi097.stdout:415 writesame 0x22f2b31 thru 0x22fed4e data_size 0x1(0xc21e bytes) 2024-09-18T00:18:13.615 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:13.615 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:13.629 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:18:13.674 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:13.671+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:13.708 INFO:teuthology.orchestra.run.smithi097.stdout:418 writesame 0xc9feff1 thru 0xca01d53 data_size 0x3(0x2d63 bytes) 2024-09-18T00:18:13.721 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:13.738 INFO:teuthology.orchestra.run.smithi097.stdout:420 trunc from 0xd5f7391 to 0x60fba75 2024-09-18T00:18:13.741 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:13.739+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:13.757 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:13.757 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:13.771 INFO:teuthology.orchestra.run.smithi097.stdout:421 read 0x42c23e7 thru 0x42c9348 (0x6f62 bytes) 2024-09-18T00:18:13.772 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:18:13.778 INFO:teuthology.orchestra.run.smithi097.stdout:423 trunc from 0x60fba75 to 0x3bb796a 2024-09-18T00:18:13.804 INFO:teuthology.orchestra.run.smithi097.stdout:425 punch from 0x5f6a18 to 0x5ff6f6, (0x8cde bytes) 2024-09-18T00:18:13.828 INFO:teuthology.orchestra.run.smithi097.stdout:426 write 0x716e3c9 thru 0x7173de3 (0x5a1b bytes) 2024-09-18T00:18:13.834 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:13.841 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:13.839+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:13.857 INFO:teuthology.orchestra.run.smithi097.stdout:429 write 0xa5a4cb5 thru 0xa5b08f1 (0xbc3d bytes) 2024-09-18T00:18:13.863 INFO:teuthology.orchestra.run.smithi097.stdout:430 punch from 0x305de1 to 0x30f691, (0x98b0 bytes) 2024-09-18T00:18:13.864 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:13.864 INFO:teuthology.orchestra.run.smithi097.stdout:432 punch from 0x1f53105 to 0x1f61a9a, (0xe995 bytes) 2024-09-18T00:18:13.864 INFO:teuthology.orchestra.run.smithi097.stdout:433 writesame 0x574264c thru 0x574e133 data_size 0x4(0xbae8 bytes) 2024-09-18T00:18:13.921 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:13.921 INFO:teuthology.orchestra.run.smithi097.stdout:435 write 0x2d0d721 thru 0x2d15727 (0x8007 bytes) 2024-09-18T00:18:13.930 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:13.931 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:13.931 INFO:teuthology.orchestra.run.smithi097.stdout:438 read 0x955ede9 thru 0x955f515 (0x72d bytes) 2024-09-18T00:18:13.931 INFO:teuthology.orchestra.run.smithi097.stdout:439 read 0x33d0527 thru 0x33d6a0f (0x64e9 bytes) 2024-09-18T00:18:13.937 INFO:teuthology.orchestra.run.smithi097.stdout:440 trunc from 0xa5b08f2 to 0xc933966 2024-09-18T00:18:13.938 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:13.938 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:13.942 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:13.939+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:13.948 INFO:teuthology.orchestra.run.smithi097.stdout:441 write 0x96a09bc thru 0x96afe95 (0xf4da bytes) 2024-09-18T00:18:13.952 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:18:13.962 INFO:teuthology.orchestra.run.smithi097.stdout:442 read 0xb290373 thru 0xb29b82d (0xb4bb bytes) 2024-09-18T00:18:13.962 INFO:teuthology.orchestra.run.smithi097.stdout:443 read 0xbbabde7 thru 0xbbaf5c4 (0x37de bytes) 2024-09-18T00:18:13.962 INFO:teuthology.orchestra.run.smithi097.stdout:444 read 0xb106ef6 thru 0xb10dc05 (0x6d10 bytes) 2024-09-18T00:18:13.962 INFO:teuthology.orchestra.run.smithi097.stdout:445 punch from 0xab03acf to 0xab0a8b5, (0x6de6 bytes) 2024-09-18T00:18:13.973 INFO:teuthology.orchestra.run.smithi097.stdout:446 trunc from 0xc933966 to 0x8a14086 2024-09-18T00:18:13.994 INFO:teuthology.orchestra.run.smithi097.stdout:447 read 0x33cdfa3 thru 0x33d52f8 (0x7356 bytes) 2024-09-18T00:18:13.994 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:13.994 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:14.000 INFO:teuthology.orchestra.run.smithi097.stdout:450 trunc from 0x8a14086 to 0xaadd3c9 2024-09-18T00:18:14.012 INFO:teuthology.orchestra.run.smithi097.stdout:451 read 0x3c3d602 thru 0x3c3f9aa (0x23a9 bytes) 2024-09-18T00:18:14.012 INFO:teuthology.orchestra.run.smithi097.stdout:452 write 0xa9fd8c8 thru 0xaa089d6 (0xb10f bytes) 2024-09-18T00:18:14.043 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:14.039+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:14.044 INFO:teuthology.orchestra.run.smithi097.stdout:455 write 0xd4d3182 thru 0xd4d44c3 (0x1342 bytes) 2024-09-18T00:18:14.050 INFO:teuthology.orchestra.run.smithi097.stdout:456 punch from 0x7468eff to 0x7477746, (0xe847 bytes) 2024-09-18T00:18:14.051 INFO:teuthology.orchestra.run.smithi097.stdout:457 trunc from 0xd4d44c4 to 0xcd93b3d 2024-09-18T00:18:14.072 INFO:teuthology.orchestra.run.smithi097.stdout:458 write 0x8a66acf thru 0x8a69960 (0x2e92 bytes) 2024-09-18T00:18:14.079 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:14.108 INFO:teuthology.orchestra.run.smithi097.stdout:461 trunc from 0xcd93b3d to 0x18bbde4 2024-09-18T00:18:14.126 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:14.126 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:14.140 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:14.141 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:18:14.157 INFO:teuthology.orchestra.run.smithi097.stdout:464 writesame 0x356eece thru 0x35742b5 data_size 0x18(0x53e8 bytes) 2024-09-18T00:18:14.157 INFO:teuthology.orchestra.run.smithi097.stdout:465 punch from 0xa34da3 to 0xa40c78, (0xbed5 bytes) 2024-09-18T00:18:14.160 INFO:teuthology.orchestra.run.smithi097.stdout:466 trunc from 0x35742b6 to 0x47b6669 2024-09-18T00:18:14.173 INFO:teuthology.orchestra.run.smithi097.stdout:467 punch from 0x374ff64 to 0x3752671, (0x270d bytes) 2024-09-18T00:18:14.185 INFO:teuthology.orchestra.run.smithi097.stdout:468 write 0x49e4e26 thru 0x49e67fb (0x19d6 bytes) 2024-09-18T00:18:14.186 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:14.183+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:14.209 INFO:teuthology.orchestra.run.smithi097.stdout:469 trunc from 0x49e67fc to 0xadd87b2 2024-09-18T00:18:14.220 INFO:teuthology.orchestra.run.smithi097.stdout:truncating to largest ever: 0xebf0c1d 2024-09-18T00:18:14.231 INFO:teuthology.orchestra.run.smithi097.stdout:470 trunc from 0xadd87b2 to 0xebf0c1d 2024-09-18T00:18:14.242 INFO:teuthology.orchestra.run.smithi097.stdout:471 read 0x9e95d43 thru 0x9e9d0d7 (0x7395 bytes) 2024-09-18T00:18:14.242 INFO:teuthology.orchestra.run.smithi097.stdout:472 writesame 0x2e591fe thru 0x2e5e1dc data_size 0x17(0x4fdf bytes) 2024-09-18T00:18:14.244 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:14.239+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:14.249 INFO:teuthology.orchestra.run.smithi097.stdout:474 punch from 0x99b166a to 0x99ba37c, (0x8d12 bytes) 2024-09-18T00:18:14.249 INFO:teuthology.orchestra.run.smithi097.stdout:475 write 0x1d72865 thru 0x1d817b1 (0xef4d bytes) 2024-09-18T00:18:14.261 INFO:teuthology.orchestra.run.smithi097.stdout:476 write 0x7dd85df thru 0x7de2c14 (0xa636 bytes) 2024-09-18T00:18:14.270 INFO:teuthology.orchestra.run.smithi097.stdout:477 read 0xc5132ae thru 0xc51487c (0x15cf bytes) 2024-09-18T00:18:14.270 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:14.271 INFO:teuthology.orchestra.run.smithi097.stdout:480 write 0x4fd4bad thru 0x4fdd56b (0x89bf bytes) 2024-09-18T00:18:14.279 INFO:teuthology.orchestra.run.smithi097.stdout:481 read 0xda656c thru 0xda958e (0x3023 bytes) 2024-09-18T00:18:14.280 INFO:teuthology.orchestra.run.smithi097.stdout:482 writesame 0x384cde1 thru 0x38562f9 data_size 0x3(0x9519 bytes) 2024-09-18T00:18:14.293 INFO:teuthology.orchestra.run.smithi097.stdout:483 punch from 0x96be71c to 0x96bf472, (0xd56 bytes) 2024-09-18T00:18:14.293 INFO:teuthology.orchestra.run.smithi097.stdout:484 write 0x4ca9a66 thru 0x4cb7530 (0xdacb bytes) 2024-09-18T00:18:14.305 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:14.313 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:14.313 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:14.327 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:18:14.331 INFO:teuthology.orchestra.run.smithi097.stdout:486 trunc from 0xebf0c1d to 0x5356a8e 2024-09-18T00:18:14.350 INFO:teuthology.orchestra.run.smithi097.stdout:488 punch from 0x51fb4f6 to 0x51fd492, (0x1f9c bytes) 2024-09-18T00:18:14.370 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:14.367+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:14.371 INFO:teuthology.orchestra.run.smithi097.stdout:489 write 0x826e4ae thru 0x827bd27 (0xd87a bytes) 2024-09-18T00:18:14.378 INFO:teuthology.orchestra.run.smithi097.stdout:492 punch from 0x3a3ea3 to 0x3af7b3, (0xb910 bytes) 2024-09-18T00:18:14.378 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:14.378 INFO:teuthology.orchestra.run.smithi097.stdout:495 punch from 0x6e32db4 to 0x6e35aed, (0x2d39 bytes) 2024-09-18T00:18:14.382 INFO:teuthology.orchestra.run.smithi097.stdout:496 trunc from 0x827bd28 to 0x9c6ee82 2024-09-18T00:18:14.395 INFO:teuthology.orchestra.run.smithi097.stdout:498 read 0x6255eeb thru 0x6260746 (0xa85c bytes) 2024-09-18T00:18:14.395 INFO:teuthology.orchestra.run.smithi097.stdout:500 write 0x19872e4 thru 0x198a0f7 (0x2e14 bytes) 2024-09-18T00:18:14.402 INFO:teuthology.orchestra.run.smithi097.stdout:501 write 0x303adf5 thru 0x303bbfd (0xe09 bytes) 2024-09-18T00:18:14.407 INFO:teuthology.orchestra.run.smithi097.stdout:502 write 0x65ff805 thru 0x6603673 (0x3e6f bytes) 2024-09-18T00:18:14.419 INFO:teuthology.orchestra.run.smithi097.stdout:506 write 0x8dffca3 thru 0x8e091ab (0x9509 bytes) 2024-09-18T00:18:14.430 INFO:teuthology.orchestra.run.smithi097.stdout:507 read 0x35144e0 thru 0x351b7c6 (0x72e7 bytes) 2024-09-18T00:18:14.430 INFO:teuthology.orchestra.run.smithi097.stdout:508 write 0x7264be3 thru 0x726d991 (0x8daf bytes) 2024-09-18T00:18:14.445 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:14.443+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:14.465 INFO:teuthology.orchestra.run.smithi097.stdout:510 write 0xe48e8c6 thru 0xe49a2d3 (0xba0e bytes) 2024-09-18T00:18:14.472 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:14.472 INFO:teuthology.orchestra.run.smithi097.stdout:512 write 0xb88970d thru 0xb89352d (0x9e21 bytes) 2024-09-18T00:18:14.484 INFO:teuthology.orchestra.run.smithi097.stdout:514 write 0x133bdad thru 0x1341df6 (0x604a bytes) 2024-09-18T00:18:14.494 INFO:teuthology.orchestra.run.smithi097.stdout:515 write 0x569a929 thru 0x56a09b2 (0x608a bytes) 2024-09-18T00:18:14.496 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:14.497 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:14.502 INFO:teuthology.orchestra.run.smithi097.stdout:516 writesame 0xd531207 thru 0xd53d498 data_size 0x55(0xc292 bytes) 2024-09-18T00:18:14.512 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:18:14.529 INFO:teuthology.orchestra.run.smithi097.stdout:517 punch from 0x732f90c to 0x733c33f, (0xca33 bytes) 2024-09-18T00:18:14.529 INFO:teuthology.orchestra.run.smithi097.stdout:518 write 0x2fd489f thru 0x2fd4995 (0xf7 bytes) 2024-09-18T00:18:14.535 INFO:teuthology.orchestra.run.smithi097.stdout:519 writesame 0x42441a9 thru 0x424a13c data_size 0x6(0x5f94 bytes) 2024-09-18T00:18:14.552 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:18:14.553 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:18:14.553 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-09-18T00:18:14.553 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 7 2024-09-18T00:18:14.558 INFO:teuthology.orchestra.run.smithi097.stdout:521 writesame 0x9b6a05a thru 0x9b74098 data_size 0x1(0xa03f bytes) 2024-09-18T00:18:14.558 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:14.555+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:14.643 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:14.643 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:14.646 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:14.643+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:14.658 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:18:14.694 INFO:teuthology.orchestra.run.smithi097.stdout:522 read 0x1ea8ab3 thru 0x1eb1811 (0x8d5f bytes) 2024-09-18T00:18:14.694 INFO:teuthology.orchestra.run.smithi097.stdout:523 write 0xa755262 thru 0xa762f0f (0xdcae bytes) 2024-09-18T00:18:14.701 INFO:teuthology.orchestra.run.smithi097.stdout:524 read 0x1f1f99 thru 0x1f6012 (0x407a bytes) 2024-09-18T00:18:14.702 INFO:teuthology.orchestra.run.smithi097.stdout:525 writesame 0x26c4331 thru 0x26c7eb4 data_size 0x1a(0x3b84 bytes) 2024-09-18T00:18:14.705 INFO:teuthology.orchestra.run.smithi097.stdout:527 write 0x622d11d thru 0x623b62f (0xe513 bytes) 2024-09-18T00:18:14.716 INFO:teuthology.orchestra.run.smithi097.stdout:528 trunc from 0xe49a2d4 to 0xb2b276d 2024-09-18T00:18:14.729 INFO:teuthology.orchestra.run.smithi097.stdout:530 write 0x50bbd75 thru 0x50c7641 (0xb8cd bytes) 2024-09-18T00:18:14.747 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:14.743+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:14.749 INFO:teuthology.orchestra.run.smithi097.stdout:533 trunc from 0xb2b276d to 0x1ac3bed 2024-09-18T00:18:14.800 INFO:teuthology.orchestra.run.smithi097.stdout:535 write 0xe91326f thru 0xe916615 (0x33a7 bytes) 2024-09-18T00:18:14.805 INFO:teuthology.orchestra.run.smithi097.stdout:536 writesame 0x4033ae6 thru 0x403b07d data_size 0x8(0x7598 bytes) 2024-09-18T00:18:14.813 INFO:teuthology.orchestra.run.smithi097.stdout:537 write 0x75f6721 thru 0x75fe79e (0x807e bytes) 2024-09-18T00:18:14.823 INFO:teuthology.orchestra.run.smithi097.stdout:539 write 0xa78cf0 thru 0xa80f27 (0x8238 bytes) 2024-09-18T00:18:14.827 INFO:teuthology.orchestra.run.smithi097.stdout:540 punch from 0xb4ab5f1 to 0xb4ad3e8, (0x1df7 bytes) 2024-09-18T00:18:14.830 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:14.830 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:14.838 INFO:teuthology.orchestra.run.smithi097.stdout:541 trunc from 0xe916616 to 0x75175c9 2024-09-18T00:18:14.844 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:18:14.855 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:14.855 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:14.855 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:14.855 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:14.875 INFO:teuthology.orchestra.run.smithi097.stdout:548 write 0xea7af69 thru 0xea87964 (0xc9fc bytes) 2024-09-18T00:18:14.881 INFO:teuthology.orchestra.run.smithi097.stdout:553 write 0x8a69dfa thru 0x8a6a708 (0x90f bytes) 2024-09-18T00:18:14.885 INFO:teuthology.orchestra.run.smithi097.stdout:554 write 0xd491966 thru 0xd492504 (0xb9f bytes) 2024-09-18T00:18:14.886 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:14.883+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:14.890 INFO:teuthology.orchestra.run.smithi097.stdout:555 read 0x1d3e60 thru 0x1d60be (0x225f bytes) 2024-09-18T00:18:14.890 INFO:teuthology.orchestra.run.smithi097.stdout:556 clone 2 order 18 su 32768 sc 6 2024-09-18T00:18:14.894 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-18T00:18:14.891+0000 7f63a6107640 -1 mon.a@0(leader).osd e31 definitely_dead 0 2024-09-18T00:18:14.949 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:14.943+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:15.009 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:15.009 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:15.023 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:18:15.067 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:15.063+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:15.151 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:15.147+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:15.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:18:15.191 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:15.191 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:15.205 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:18:15.252 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:15.247+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:15.316 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:15.323 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:15.323 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:15.333 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:18:15.333 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:18:15.337 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:18:15.352 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:15.347+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:15.353 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:15.353 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:15.369 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:15.369 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:15.388 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:15.388 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:15.405 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:18:15.406 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:18:15.423 INFO:teuthology.orchestra.run.smithi097.stderr:marked in osd.7. 2024-09-18T00:18:15.429 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:15.429 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:15.430 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:18:15.446 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-09-18T00:18:15.452 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:15.452 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:15.452 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:15.447+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:15.468 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:15.468 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:15.468 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:15.468 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:15.468 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:15.482 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:18:15.570 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:15.567+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:15.654 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:15.651+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:15.655 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:15.655 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:15.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:15.656 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:15.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.656 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.656 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.440942+0000", 2024-09-18T00:18:15.656 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.210123784, 2024-09-18T00:18:15.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042441465999999997, 2024-09-18T00:18:15.656 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.656 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.656 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.656 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.656 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.657 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.657 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-18T00:18:15.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.657 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.440942+0000", 2024-09-18T00:18:15.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.440942+0000", 2024-09-18T00:18:15.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.440945+0000", 2024-09-18T00:18:15.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5969999999999999e-06 2024-09-18T00:18:15.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.440958+0000", 2024-09-18T00:18:15.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3393e-05 2024-09-18T00:18:15.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.440981+0000", 2024-09-18T00:18:15.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3054e-05 2024-09-18T00:18:15.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.441001+0000", 2024-09-18T00:18:15.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.969e-05 2024-09-18T00:18:15.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.441125+0000", 2024-09-18T00:18:15.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012457200000000001 2024-09-18T00:18:15.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:18:15.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.441143+0000", 2024-09-18T00:18:15.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7876999999999998e-05 2024-09-18T00:18:15.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.477045+0000", 2024-09-18T00:18:15.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035901699000000002 2024-09-18T00:18:15.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.477360+0000", 2024-09-18T00:18:15.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031502400000000001 2024-09-18T00:18:15.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.477816+0000", 2024-09-18T00:18:15.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045637 2024-09-18T00:18:15.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.482239+0000", 2024-09-18T00:18:15.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044223659999999996 2024-09-18T00:18:15.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.483228+0000", 2024-09-18T00:18:15.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00098898600000000008 2024-09-18T00:18:15.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.483324+0000", 2024-09-18T00:18:15.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5878999999999995e-05 2024-09-18T00:18:15.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.483384+0000", 2024-09-18T00:18:15.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9959e-05 2024-09-18T00:18:15.663 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.663 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.663 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.663 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.663 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.549389+0000", 2024-09-18T00:18:15.663 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.101676920999999, 2024-09-18T00:18:15.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014668851, 2024-09-18T00:18:15.663 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.663 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.663 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.664 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34 2024-09-18T00:18:15.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.664 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549389+0000", 2024-09-18T00:18:15.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549389+0000", 2024-09-18T00:18:15.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549390+0000", 2024-09-18T00:18:15.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5400000000000001e-07 2024-09-18T00:18:15.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549395+0000", 2024-09-18T00:18:15.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5060000000000003e-06 2024-09-18T00:18:15.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549405+0000", 2024-09-18T00:18:15.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4399999999999994e-06 2024-09-18T00:18:15.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549412+0000", 2024-09-18T00:18:15.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.007e-06 2024-09-18T00:18:15.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.556213+0000", 2024-09-18T00:18:15.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068017370000000004 2024-09-18T00:18:15.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.556465+0000", 2024-09-18T00:18:15.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025113499999999999 2024-09-18T00:18:15.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.556837+0000", 2024-09-18T00:18:15.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037237400000000002 2024-09-18T00:18:15.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.562324+0000", 2024-09-18T00:18:15.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054871649999999996 2024-09-18T00:18:15.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.563789+0000", 2024-09-18T00:18:15.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0014652210000000001 2024-09-18T00:18:15.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.563940+0000", 2024-09-18T00:18:15.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015076299999999999 2024-09-18T00:18:15.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564058+0000", 2024-09-18T00:18:15.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000117849 2024-09-18T00:18:15.669 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.669 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.669 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.669 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.669 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.549430+0000", 2024-09-18T00:18:15.669 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.101636157000002, 2024-09-18T00:18:15.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015014833999999999, 2024-09-18T00:18:15.670 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.670 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.670 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.670 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.670 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.670 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.670 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35 2024-09-18T00:18:15.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.670 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549430+0000", 2024-09-18T00:18:15.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549430+0000", 2024-09-18T00:18:15.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549431+0000", 2024-09-18T00:18:15.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0599999999999999e-07 2024-09-18T00:18:15.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549517+0000", 2024-09-18T00:18:15.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6159e-05 2024-09-18T00:18:15.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549530+0000", 2024-09-18T00:18:15.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3604000000000001e-05 2024-09-18T00:18:15.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549538+0000", 2024-09-18T00:18:15.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7959999999999996e-06 2024-09-18T00:18:15.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.558970+0000", 2024-09-18T00:18:15.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.009432134 2024-09-18T00:18:15.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.559209+0000", 2024-09-18T00:18:15.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023852500000000001 2024-09-18T00:18:15.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.559572+0000", 2024-09-18T00:18:15.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00036321999999999998 2024-09-18T00:18:15.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564262+0000", 2024-09-18T00:18:15.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046898909999999999 2024-09-18T00:18:15.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564293+0000", 2024-09-18T00:18:15.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.074e-05 2024-09-18T00:18:15.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564373+0000", 2024-09-18T00:18:15.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9785999999999998e-05 2024-09-18T00:18:15.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564445+0000", 2024-09-18T00:18:15.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2072999999999999e-05 2024-09-18T00:18:15.675 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.675 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.675 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.675 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.675 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.549552+0000", 2024-09-18T00:18:15.675 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.101514128000002, 2024-09-18T00:18:15.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019865803000000001, 2024-09-18T00:18:15.675 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.675 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.676 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.676 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.676 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.676 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.676 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 36 2024-09-18T00:18:15.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.676 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549552+0000", 2024-09-18T00:18:15.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549552+0000", 2024-09-18T00:18:15.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549553+0000", 2024-09-18T00:18:15.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4600000000000004e-07 2024-09-18T00:18:15.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549558+0000", 2024-09-18T00:18:15.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6099999999999997e-06 2024-09-18T00:18:15.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549568+0000", 2024-09-18T00:18:15.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3470000000000005e-06 2024-09-18T00:18:15.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549576+0000", 2024-09-18T00:18:15.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0399999999999993e-06 2024-09-18T00:18:15.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.561348+0000", 2024-09-18T00:18:15.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011771992 2024-09-18T00:18:15.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.561491+0000", 2024-09-18T00:18:15.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014369299999999999 2024-09-18T00:18:15.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.561712+0000", 2024-09-18T00:18:15.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000220361 2024-09-18T00:18:15.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564458+0000", 2024-09-18T00:18:15.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0027459519999999999 2024-09-18T00:18:15.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569146+0000", 2024-09-18T00:18:15.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046883070000000001 2024-09-18T00:18:15.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569303+0000", 2024-09-18T00:18:15.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015697199999999999 2024-09-18T00:18:15.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569418+0000", 2024-09-18T00:18:15.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000114783 2024-09-18T00:18:15.681 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.681 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.681 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.681 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.681 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.549587+0000", 2024-09-18T00:18:15.681 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.101479187999999, 2024-09-18T00:18:15.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.020238056000000001, 2024-09-18T00:18:15.681 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.681 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.681 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.681 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.681 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.681 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 37 2024-09-18T00:18:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549587+0000", 2024-09-18T00:18:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549587+0000", 2024-09-18T00:18:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549587+0000", 2024-09-18T00:18:15.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3300000000000002e-07 2024-09-18T00:18:15.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549593+0000", 2024-09-18T00:18:15.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2700000000000004e-06 2024-09-18T00:18:15.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549602+0000", 2024-09-18T00:18:15.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0559999999999994e-06 2024-09-18T00:18:15.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.549609+0000", 2024-09-18T00:18:15.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1930000000000004e-06 2024-09-18T00:18:15.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.562442+0000", 2024-09-18T00:18:15.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012833229 2024-09-18T00:18:15.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.562723+0000", 2024-09-18T00:18:15.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028104399999999999 2024-09-18T00:18:15.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.563094+0000", 2024-09-18T00:18:15.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037122000000000001 2024-09-18T00:18:15.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.564844+0000", 2024-09-18T00:18:15.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00174986 2024-09-18T00:18:15.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569573+0000", 2024-09-18T00:18:15.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047285829999999997 2024-09-18T00:18:15.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569727+0000", 2024-09-18T00:18:15.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000154146 2024-09-18T00:18:15.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.569825+0000", 2024-09-18T00:18:15.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7822e-05 2024-09-18T00:18:15.686 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.686 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.686 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.687 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.687 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.677038+0000", 2024-09-18T00:18:15.687 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.974028309000001, 2024-09-18T00:18:15.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011802852000000001, 2024-09-18T00:18:15.687 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.687 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.687 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.687 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.687 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.687 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.687 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 88 2024-09-18T00:18:15.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.687 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677038+0000", 2024-09-18T00:18:15.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677038+0000", 2024-09-18T00:18:15.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677039+0000", 2024-09-18T00:18:15.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.53e-06 2024-09-18T00:18:15.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677050+0000", 2024-09-18T00:18:15.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1253000000000001e-05 2024-09-18T00:18:15.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677070+0000", 2024-09-18T00:18:15.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9707e-05 2024-09-18T00:18:15.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.677088+0000", 2024-09-18T00:18:15.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8352999999999999e-05 2024-09-18T00:18:15.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.685999+0000", 2024-09-18T00:18:15.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089100550000000001 2024-09-18T00:18:15.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.686128+0000", 2024-09-18T00:18:15.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012984199999999999 2024-09-18T00:18:15.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.686318+0000", 2024-09-18T00:18:15.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019002900000000001 2024-09-18T00:18:15.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.687712+0000", 2024-09-18T00:18:15.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001393799 2024-09-18T00:18:15.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.688703+0000", 2024-09-18T00:18:15.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000990875 2024-09-18T00:18:15.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.688776+0000", 2024-09-18T00:18:15.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3280000000000003e-05 2024-09-18T00:18:15.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.688841+0000", 2024-09-18T00:18:15.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4128999999999995e-05 2024-09-18T00:18:15.692 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.692 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.693 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.693 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.693 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836576+0000", 2024-09-18T00:18:15.693 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.81448958, 2024-09-18T00:18:15.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01473359, 2024-09-18T00:18:15.693 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.693 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.693 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.693 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.693 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.693 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.693 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110 2024-09-18T00:18:15.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.694 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836576+0000", 2024-09-18T00:18:15.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836576+0000", 2024-09-18T00:18:15.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836577+0000", 2024-09-18T00:18:15.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4e-07 2024-09-18T00:18:15.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836586+0000", 2024-09-18T00:18:15.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3699999999999995e-06 2024-09-18T00:18:15.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836599+0000", 2024-09-18T00:18:15.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.343e-05 2024-09-18T00:18:15.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836608+0000", 2024-09-18T00:18:15.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2199999999999998e-06 2024-09-18T00:18:15.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.846689+0000", 2024-09-18T00:18:15.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010080512 2024-09-18T00:18:15.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.846824+0000", 2024-09-18T00:18:15.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000134902 2024-09-18T00:18:15.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.847020+0000", 2024-09-18T00:18:15.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019580199999999999 2024-09-18T00:18:15.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851018+0000", 2024-09-18T00:18:15.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0039983629999999996 2024-09-18T00:18:15.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851062+0000", 2024-09-18T00:18:15.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3769000000000001e-05 2024-09-18T00:18:15.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851205+0000", 2024-09-18T00:18:15.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014293199999999999 2024-09-18T00:18:15.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851310+0000", 2024-09-18T00:18:15.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010535 2024-09-18T00:18:15.698 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.698 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.698 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.699 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.699 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836620+0000", 2024-09-18T00:18:15.699 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.814445847000002, 2024-09-18T00:18:15.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016200902, 2024-09-18T00:18:15.699 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.699 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.699 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.699 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.699 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.699 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.699 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 111 2024-09-18T00:18:15.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.699 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836620+0000", 2024-09-18T00:18:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836620+0000", 2024-09-18T00:18:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836621+0000", 2024-09-18T00:18:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2699999999999999e-07 2024-09-18T00:18:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836626+0000", 2024-09-18T00:18:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2360000000000004e-06 2024-09-18T00:18:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836636+0000", 2024-09-18T00:18:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6169999999999996e-06 2024-09-18T00:18:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836679+0000", 2024-09-18T00:18:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3415999999999999e-05 2024-09-18T00:18:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.847736+0000", 2024-09-18T00:18:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011057361999999999 2024-09-18T00:18:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.847978+0000", 2024-09-18T00:18:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024185400000000001 2024-09-18T00:18:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.848415+0000", 2024-09-18T00:18:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000436904 2024-09-18T00:18:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851430+0000", 2024-09-18T00:18:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030148369999999998 2024-09-18T00:18:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.852537+0000", 2024-09-18T00:18:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011069910000000001 2024-09-18T00:18:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.852703+0000", 2024-09-18T00:18:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016631600000000001 2024-09-18T00:18:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.852821+0000", 2024-09-18T00:18:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000117642 2024-09-18T00:18:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836714+0000", 2024-09-18T00:18:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.814351783999999, 2024-09-18T00:18:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017759086, 2024-09-18T00:18:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 112 2024-09-18T00:18:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836714+0000", 2024-09-18T00:18:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836714+0000", 2024-09-18T00:18:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836715+0000", 2024-09-18T00:18:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1030000000000001e-06 2024-09-18T00:18:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836810+0000", 2024-09-18T00:18:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5192000000000006e-05 2024-09-18T00:18:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836839+0000", 2024-09-18T00:18:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8753999999999999e-05 2024-09-18T00:18:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836849+0000", 2024-09-18T00:18:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7860000000000008e-06 2024-09-18T00:18:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.849128+0000", 2024-09-18T00:18:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012278839 2024-09-18T00:18:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.849256+0000", 2024-09-18T00:18:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012836499999999999 2024-09-18T00:18:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.849450+0000", 2024-09-18T00:18:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019406600000000001 2024-09-18T00:18:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.852958+0000", 2024-09-18T00:18:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035073930000000001 2024-09-18T00:18:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854162+0000", 2024-09-18T00:18:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0012046100000000001 2024-09-18T00:18:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854355+0000", 2024-09-18T00:18:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000192449 2024-09-18T00:18:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854473+0000", 2024-09-18T00:18:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000118529 2024-09-18T00:18:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836865+0000", 2024-09-18T00:18:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.814201269000002, 2024-09-18T00:18:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019780444000000001, 2024-09-18T00:18:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 113 2024-09-18T00:18:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836865+0000", 2024-09-18T00:18:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836865+0000", 2024-09-18T00:18:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836866+0000", 2024-09-18T00:18:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7400000000000002e-07 2024-09-18T00:18:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836872+0000", 2024-09-18T00:18:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0260000000000004e-06 2024-09-18T00:18:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836885+0000", 2024-09-18T00:18:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3303e-05 2024-09-18T00:18:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836892+0000", 2024-09-18T00:18:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5700000000000004e-06 2024-09-18T00:18:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851546+0000", 2024-09-18T00:18:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014653497999999999 2024-09-18T00:18:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851760+0000", 2024-09-18T00:18:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021415400000000001 2024-09-18T00:18:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.851954+0000", 2024-09-18T00:18:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019398199999999999 2024-09-18T00:18:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856341+0000", 2024-09-18T00:18:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0043870489999999996 2024-09-18T00:18:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856388+0000", 2024-09-18T00:18:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6919999999999998e-05 2024-09-18T00:18:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856530+0000", 2024-09-18T00:18:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142249 2024-09-18T00:18:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856645+0000", 2024-09-18T00:18:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000114819 2024-09-18T00:18:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836911+0000", 2024-09-18T00:18:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.814154558999999, 2024-09-18T00:18:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.021208742999999999, 2024-09-18T00:18:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 114 2024-09-18T00:18:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836911+0000", 2024-09-18T00:18:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836911+0000", 2024-09-18T00:18:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836912+0000", 2024-09-18T00:18:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5600000000000005e-07 2024-09-18T00:18:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836919+0000", 2024-09-18T00:18:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4400000000000002e-06 2024-09-18T00:18:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836928+0000", 2024-09-18T00:18:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.91e-06 2024-09-18T00:18:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836935+0000", 2024-09-18T00:18:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0829999999999998e-06 2024-09-18T00:18:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.853068+0000", 2024-09-18T00:18:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016132082999999998 2024-09-18T00:18:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.853344+0000", 2024-09-18T00:18:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027593799999999999 2024-09-18T00:18:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.853571+0000", 2024-09-18T00:18:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022752800000000001 2024-09-18T00:18:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856732+0000", 2024-09-18T00:18:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031610779999999999 2024-09-18T00:18:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.857856+0000", 2024-09-18T00:18:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001123752 2024-09-18T00:18:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.857998+0000", 2024-09-18T00:18:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142292 2024-09-18T00:18:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.858120+0000", 2024-09-18T00:18:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121983 2024-09-18T00:18:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.836969+0000", 2024-09-18T00:18:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.814096840000001, 2024-09-18T00:18:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.022609931999999999, 2024-09-18T00:18:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 115 2024-09-18T00:18:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836969+0000", 2024-09-18T00:18:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836969+0000", 2024-09-18T00:18:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.836970+0000", 2024-09-18T00:18:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1399999999999996e-07 2024-09-18T00:18:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837068+0000", 2024-09-18T00:18:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7868999999999994e-05 2024-09-18T00:18:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837095+0000", 2024-09-18T00:18:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7793e-05 2024-09-18T00:18:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837106+0000", 2024-09-18T00:18:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.028e-05 2024-09-18T00:18:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854556+0000", 2024-09-18T00:18:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017449742000000001 2024-09-18T00:18:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854690+0000", 2024-09-18T00:18:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013440500000000001 2024-09-18T00:18:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.854900+0000", 2024-09-18T00:18:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000210306 2024-09-18T00:18:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.858224+0000", 2024-09-18T00:18:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0033242010000000002 2024-09-18T00:18:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859316+0000", 2024-09-18T00:18:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010916109999999999 2024-09-18T00:18:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859450+0000", 2024-09-18T00:18:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013410900000000001 2024-09-18T00:18:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859579+0000", 2024-09-18T00:18:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012890199999999999 2024-09-18T00:18:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.837122+0000", 2024-09-18T00:18:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.813943937000001, 2024-09-18T00:18:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.022693474000000002, 2024-09-18T00:18:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116 2024-09-18T00:18:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837122+0000", 2024-09-18T00:18:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837122+0000", 2024-09-18T00:18:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837123+0000", 2024-09-18T00:18:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.33e-07 2024-09-18T00:18:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837131+0000", 2024-09-18T00:18:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5029999999999997e-06 2024-09-18T00:18:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837142+0000", 2024-09-18T00:18:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0329999999999999e-05 2024-09-18T00:18:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837149+0000", 2024-09-18T00:18:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.99e-06 2024-09-18T00:18:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.856845+0000", 2024-09-18T00:18:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019696074000000001 2024-09-18T00:18:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.857107+0000", 2024-09-18T00:18:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026233799999999998 2024-09-18T00:18:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.857298+0000", 2024-09-18T00:18:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019087600000000001 2024-09-18T00:18:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859604+0000", 2024-09-18T00:18:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023056980000000001 2024-09-18T00:18:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859667+0000", 2024-09-18T00:18:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3696999999999997e-05 2024-09-18T00:18:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859749+0000", 2024-09-18T00:18:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1526000000000005e-05 2024-09-18T00:18:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859816+0000", 2024-09-18T00:18:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6508999999999996e-05 2024-09-18T00:18:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:51.837160+0000", 2024-09-18T00:18:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.813905864999999, 2024-09-18T00:18:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023190008000000002, 2024-09-18T00:18:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 117 2024-09-18T00:18:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837160+0000", 2024-09-18T00:18:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837160+0000", 2024-09-18T00:18:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837161+0000", 2024-09-18T00:18:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2399999999999998e-07 2024-09-18T00:18:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837166+0000", 2024-09-18T00:18:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.716e-06 2024-09-18T00:18:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837179+0000", 2024-09-18T00:18:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2063e-05 2024-09-18T00:18:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.837186+0000", 2024-09-18T00:18:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1040000000000001e-06 2024-09-18T00:18:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.858312+0000", 2024-09-18T00:18:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.021126147000000001 2024-09-18T00:18:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.858553+0000", 2024-09-18T00:18:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024152099999999999 2024-09-18T00:18:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.858880+0000", 2024-09-18T00:18:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000326461 2024-09-18T00:18:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.859939+0000", 2024-09-18T00:18:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001059735 2024-09-18T00:18:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.860212+0000", 2024-09-18T00:18:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027284400000000001 2024-09-18T00:18:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.860292+0000", 2024-09-18T00:18:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9983999999999999e-05 2024-09-18T00:18:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:51.860350+0000", 2024-09-18T00:18:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7809000000000002e-05 2024-09-18T00:18:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.317506+0000", 2024-09-18T00:18:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.333559704999999, 2024-09-18T00:18:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011471278999999999, 2024-09-18T00:18:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 135 2024-09-18T00:18:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317506+0000", 2024-09-18T00:18:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317506+0000", 2024-09-18T00:18:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317507+0000", 2024-09-18T00:18:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7000000000000003e-07 2024-09-18T00:18:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317513+0000", 2024-09-18T00:18:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5160000000000002e-06 2024-09-18T00:18:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317522+0000", 2024-09-18T00:18:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2599999999999994e-06 2024-09-18T00:18:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317529+0000", 2024-09-18T00:18:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1570000000000003e-06 2024-09-18T00:18:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320156+0000", 2024-09-18T00:18:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00262737 2024-09-18T00:18:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320285+0000", 2024-09-18T00:18:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012859199999999999 2024-09-18T00:18:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320398+0000", 2024-09-18T00:18:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112553 2024-09-18T00:18:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.324003+0000", 2024-09-18T00:18:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036049300000000001 2024-09-18T00:18:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.328923+0000", 2024-09-18T00:18:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049205899999999999 2024-09-18T00:18:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.328952+0000", 2024-09-18T00:18:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8858999999999999e-05 2024-09-18T00:18:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.328978+0000", 2024-09-18T00:18:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5582e-05 2024-09-18T00:18:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.317540+0000", 2024-09-18T00:18:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.333526355, 2024-09-18T00:18:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011513031999999999, 2024-09-18T00:18:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 136 2024-09-18T00:18:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317540+0000", 2024-09-18T00:18:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317540+0000", 2024-09-18T00:18:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317540+0000", 2024-09-18T00:18:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9999999999999997e-07 2024-09-18T00:18:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317545+0000", 2024-09-18T00:18:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1030000000000001e-06 2024-09-18T00:18:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317554+0000", 2024-09-18T00:18:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9430000000000006e-06 2024-09-18T00:18:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317561+0000", 2024-09-18T00:18:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0299999999999996e-06 2024-09-18T00:18:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320658+0000", 2024-09-18T00:18:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030962950000000002 2024-09-18T00:18:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320712+0000", 2024-09-18T00:18:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4159000000000001e-05 2024-09-18T00:18:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.320791+0000", 2024-09-18T00:18:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.962e-05 2024-09-18T00:18:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.324006+0000", 2024-09-18T00:18:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032141320000000002 2024-09-18T00:18:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329007+0000", 2024-09-18T00:18:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050010899999999997 2024-09-18T00:18:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329031+0000", 2024-09-18T00:18:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4493000000000001e-05 2024-09-18T00:18:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329053+0000", 2024-09-18T00:18:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1467e-05 2024-09-18T00:18:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.317592+0000", 2024-09-18T00:18:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.333474048999999, 2024-09-18T00:18:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011529286, 2024-09-18T00:18:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 137 2024-09-18T00:18:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317592+0000", 2024-09-18T00:18:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317592+0000", 2024-09-18T00:18:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317593+0000", 2024-09-18T00:18:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7000000000000003e-07 2024-09-18T00:18:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317750+0000", 2024-09-18T00:18:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015739199999999999 2024-09-18T00:18:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317764+0000", 2024-09-18T00:18:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3845999999999999e-05 2024-09-18T00:18:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317840+0000", 2024-09-18T00:18:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6253000000000003e-05 2024-09-18T00:18:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.321078+0000", 2024-09-18T00:18:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032379560000000002 2024-09-18T00:18:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.321125+0000", 2024-09-18T00:18:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6585999999999997e-05 2024-09-18T00:18:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.321195+0000", 2024-09-18T00:18:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0188e-05 2024-09-18T00:18:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.324011+0000", 2024-09-18T00:18:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0028159410000000002 2024-09-18T00:18:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329075+0000", 2024-09-18T00:18:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050643379999999998 2024-09-18T00:18:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329103+0000", 2024-09-18T00:18:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7616000000000001e-05 2024-09-18T00:18:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329121+0000", 2024-09-18T00:18:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8199999999999999e-05 2024-09-18T00:18:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.317909+0000", 2024-09-18T00:18:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.333156652, 2024-09-18T00:18:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011404755000000001, 2024-09-18T00:18:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 139 2024-09-18T00:18:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317909+0000", 2024-09-18T00:18:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317909+0000", 2024-09-18T00:18:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317910+0000", 2024-09-18T00:18:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2699999999999999e-07 2024-09-18T00:18:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317916+0000", 2024-09-18T00:18:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5230000000000003e-06 2024-09-18T00:18:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317924+0000", 2024-09-18T00:18:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9239999999999996e-06 2024-09-18T00:18:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317932+0000", 2024-09-18T00:18:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0700000000000001e-06 2024-09-18T00:18:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322402+0000", 2024-09-18T00:18:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044705420000000001 2024-09-18T00:18:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322452+0000", 2024-09-18T00:18:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0297999999999999e-05 2024-09-18T00:18:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322533+0000", 2024-09-18T00:18:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1092000000000002e-05 2024-09-18T00:18:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.324722+0000", 2024-09-18T00:18:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0021886010000000001 2024-09-18T00:18:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329227+0000", 2024-09-18T00:18:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045046419999999997 2024-09-18T00:18:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329264+0000", 2024-09-18T00:18:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7577999999999998e-05 2024-09-18T00:18:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329314+0000", 2024-09-18T00:18:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9758e-05 2024-09-18T00:18:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.317962+0000", 2024-09-18T00:18:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.333103568999999, 2024-09-18T00:18:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01213116, 2024-09-18T00:18:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 140 2024-09-18T00:18:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317962+0000", 2024-09-18T00:18:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317962+0000", 2024-09-18T00:18:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.317963+0000", 2024-09-18T00:18:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8700000000000004e-07 2024-09-18T00:18:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318086+0000", 2024-09-18T00:18:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000122862 2024-09-18T00:18:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318099+0000", 2024-09-18T00:18:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2459999999999999e-05 2024-09-18T00:18:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318107+0000", 2024-09-18T00:18:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-18T00:18:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322800+0000", 2024-09-18T00:18:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046924649999999998 2024-09-18T00:18:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322849+0000", 2024-09-18T00:18:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9406000000000001e-05 2024-09-18T00:18:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.322927+0000", 2024-09-18T00:18:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7763999999999996e-05 2024-09-18T00:18:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.324725+0000", 2024-09-18T00:18:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0017981060000000001 2024-09-18T00:18:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.329976+0000", 2024-09-18T00:18:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052507109999999999 2024-09-18T00:18:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.330045+0000", 2024-09-18T00:18:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8998999999999994e-05 2024-09-18T00:18:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.330094+0000", 2024-09-18T00:18:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.905e-05 2024-09-18T00:18:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4157.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T00:18:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:52.318122+0000", 2024-09-18T00:18:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.332944313999999, 2024-09-18T00:18:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012634492000000001, 2024-09-18T00:18:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4157", 2024-09-18T00:18:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/726437758", 2024-09-18T00:18:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 141 2024-09-18T00:18:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318122+0000", 2024-09-18T00:18:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318122+0000", 2024-09-18T00:18:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318122+0000", 2024-09-18T00:18:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9400000000000005e-07 2024-09-18T00:18:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318128+0000", 2024-09-18T00:18:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3800000000000002e-06 2024-09-18T00:18:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318136+0000", 2024-09-18T00:18:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7760000000000003e-06 2024-09-18T00:18:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.318143+0000", 2024-09-18T00:18:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-18T00:18:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.323651+0000", 2024-09-18T00:18:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055075269999999999 2024-09-18T00:18:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.323700+0000", 2024-09-18T00:18:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8724999999999999e-05 2024-09-18T00:18:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.323772+0000", 2024-09-18T00:18:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2378000000000004e-05 2024-09-18T00:18:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.325433+0000", 2024-09-18T00:18:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001661278 2024-09-18T00:18:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.330670+0000", 2024-09-18T00:18:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052365689999999999 2024-09-18T00:18:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.330737+0000", 2024-09-18T00:18:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6866999999999995e-05 2024-09-18T00:18:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:52.330756+0000", 2024-09-18T00:18:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9313999999999998e-05 2024-09-18T00:18:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:15.775 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:15.776 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:15.775+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:15.776 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:18:15.877 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:15.875+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:15.898 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:15.899 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:15.899 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:15.899 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:15.912 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:18:16.000 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:15.999+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:16.074 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:16.074 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:16.074 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:16.074 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:16.074 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:16.078 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:16.071+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:16.088 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:18:16.179 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:16.175+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:16.251 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:16.251 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:16.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:16.251 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:16.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.252 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:36 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'22, mlcod=17'18)", 2024-09-18T00:18:16.252 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:51.563151+0000", 2024-09-18T00:18:16.252 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.686059601, 2024-09-18T00:18:16.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067203280000000002, 2024-09-18T00:18:16.252 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.252 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.252 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.252 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563151+0000", 2024-09-18T00:18:16.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563151+0000", 2024-09-18T00:18:16.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563153+0000", 2024-09-18T00:18:16.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.42e-06 2024-09-18T00:18:16.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563163+0000", 2024-09-18T00:18:16.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.077e-05 2024-09-18T00:18:16.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563191+0000", 2024-09-18T00:18:16.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.724e-05 2024-09-18T00:18:16.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563204+0000", 2024-09-18T00:18:16.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3266e-05 2024-09-18T00:18:16.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563319+0000", 2024-09-18T00:18:16.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115343 2024-09-18T00:18:16.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563371+0000", 2024-09-18T00:18:16.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1922000000000002e-05 2024-09-18T00:18:16.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.569760+0000", 2024-09-18T00:18:16.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388721 2024-09-18T00:18:16.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.569872+0000", 2024-09-18T00:18:16.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011164599999999999 2024-09-18T00:18:16.256 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.256 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.256 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.257 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:37 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'23, mlcod=17'18)", 2024-09-18T00:18:16.257 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:51.564518+0000", 2024-09-18T00:18:16.257 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.684692535, 2024-09-18T00:18:16.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005870994, 2024-09-18T00:18:16.257 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.257 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.257 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.257 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564518+0000", 2024-09-18T00:18:16.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564518+0000", 2024-09-18T00:18:16.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564520+0000", 2024-09-18T00:18:16.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8300000000000001e-06 2024-09-18T00:18:16.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564532+0000", 2024-09-18T00:18:16.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1754e-05 2024-09-18T00:18:16.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564560+0000", 2024-09-18T00:18:16.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8486000000000001e-05 2024-09-18T00:18:16.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564573+0000", 2024-09-18T00:18:16.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3069e-05 2024-09-18T00:18:16.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564660+0000", 2024-09-18T00:18:16.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6263999999999999e-05 2024-09-18T00:18:16.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564695+0000", 2024-09-18T00:18:16.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5296000000000002e-05 2024-09-18T00:18:16.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.570284+0000", 2024-09-18T00:18:16.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055890030000000004 2024-09-18T00:18:16.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.570389+0000", 2024-09-18T00:18:16.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010529199999999999 2024-09-18T00:18:16.261 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.261 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.261 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.261 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:131 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'89, mlcod=17'85)", 2024-09-18T00:18:16.261 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.319608+0000", 2024-09-18T00:18:16.262 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.929602446000001, 2024-09-18T00:18:16.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0058413620000000001, 2024-09-18T00:18:16.262 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.262 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.262 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.262 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319608+0000", 2024-09-18T00:18:16.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319608+0000", 2024-09-18T00:18:16.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319609+0000", 2024-09-18T00:18:16.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3e-07 2024-09-18T00:18:16.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319676+0000", 2024-09-18T00:18:16.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6889000000000003e-05 2024-09-18T00:18:16.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319694+0000", 2024-09-18T00:18:16.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7703e-05 2024-09-18T00:18:16.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319706+0000", 2024-09-18T00:18:16.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2577e-05 2024-09-18T00:18:16.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320272+0000", 2024-09-18T00:18:16.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056591599999999997 2024-09-18T00:18:16.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320306+0000", 2024-09-18T00:18:16.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4116000000000003e-05 2024-09-18T00:18:16.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325386+0000", 2024-09-18T00:18:16.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050793349999999999 2024-09-18T00:18:16.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325450+0000", 2024-09-18T00:18:16.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4196000000000007e-05 2024-09-18T00:18:16.266 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.266 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.266 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.266 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:132 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'90, mlcod=17'85)", 2024-09-18T00:18:16.266 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.320067+0000", 2024-09-18T00:18:16.266 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.929143913000001, 2024-09-18T00:18:16.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0054358590000000004, 2024-09-18T00:18:16.266 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.266 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.266 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.266 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320067+0000", 2024-09-18T00:18:16.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320067+0000", 2024-09-18T00:18:16.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320068+0000", 2024-09-18T00:18:16.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0100000000000001e-06 2024-09-18T00:18:16.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320079+0000", 2024-09-18T00:18:16.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1146e-05 2024-09-18T00:18:16.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320101+0000", 2024-09-18T00:18:16.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1994e-05 2024-09-18T00:18:16.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320110+0000", 2024-09-18T00:18:16.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:18:16.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321623+0000", 2024-09-18T00:18:16.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001512981 2024-09-18T00:18:16.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321656+0000", 2024-09-18T00:18:16.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2539999999999997e-05 2024-09-18T00:18:16.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325455+0000", 2024-09-18T00:18:16.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0037992880000000001 2024-09-18T00:18:16.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325503+0000", 2024-09-18T00:18:16.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7567e-05 2024-09-18T00:18:16.270 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.271 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.271 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.271 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:133 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'91, mlcod=17'85)", 2024-09-18T00:18:16.271 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.320439+0000", 2024-09-18T00:18:16.271 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.928771770000001, 2024-09-18T00:18:16.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005115858, 2024-09-18T00:18:16.271 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.271 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.271 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.271 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320439+0000", 2024-09-18T00:18:16.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320439+0000", 2024-09-18T00:18:16.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320440+0000", 2024-09-18T00:18:16.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2999999999999999e-07 2024-09-18T00:18:16.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320454+0000", 2024-09-18T00:18:16.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4107000000000001e-05 2024-09-18T00:18:16.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320472+0000", 2024-09-18T00:18:16.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7499999999999998e-05 2024-09-18T00:18:16.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320480+0000", 2024-09-18T00:18:16.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0970000000000006e-06 2024-09-18T00:18:16.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322261+0000", 2024-09-18T00:18:16.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001780946 2024-09-18T00:18:16.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322297+0000", 2024-09-18T00:18:16.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6510000000000001e-05 2024-09-18T00:18:16.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325507+0000", 2024-09-18T00:18:16.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0032097919999999999 2024-09-18T00:18:16.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325555+0000", 2024-09-18T00:18:16.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8075999999999998e-05 2024-09-18T00:18:16.275 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.275 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.275 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.276 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:134 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'92, mlcod=17'85)", 2024-09-18T00:18:16.276 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.320847+0000", 2024-09-18T00:18:16.276 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.928364039000002, 2024-09-18T00:18:16.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0068502509999999999, 2024-09-18T00:18:16.276 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.276 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.276 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.276 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320847+0000", 2024-09-18T00:18:16.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320847+0000", 2024-09-18T00:18:16.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320848+0000", 2024-09-18T00:18:16.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-18T00:18:16.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320971+0000", 2024-09-18T00:18:16.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012338599999999999 2024-09-18T00:18:16.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320990+0000", 2024-09-18T00:18:16.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9009999999999999e-05 2024-09-18T00:18:16.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321000+0000", 2024-09-18T00:18:16.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9399999999999997e-06 2024-09-18T00:18:16.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322866+0000", 2024-09-18T00:18:16.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001866089 2024-09-18T00:18:16.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322902+0000", 2024-09-18T00:18:16.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5312999999999999e-05 2024-09-18T00:18:16.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327608+0000", 2024-09-18T00:18:16.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047063000000000001 2024-09-18T00:18:16.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327697+0000", 2024-09-18T00:18:16.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9049999999999996e-05 2024-09-18T00:18:16.280 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.280 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.280 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.280 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:135 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'93, mlcod=17'85)", 2024-09-18T00:18:16.280 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.321689+0000", 2024-09-18T00:18:16.280 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.927521388999999, 2024-09-18T00:18:16.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0082046600000000008, 2024-09-18T00:18:16.280 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.280 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.281 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.281 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321689+0000", 2024-09-18T00:18:16.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321689+0000", 2024-09-18T00:18:16.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321691+0000", 2024-09-18T00:18:16.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0229999999999999e-06 2024-09-18T00:18:16.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321706+0000", 2024-09-18T00:18:16.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4513e-05 2024-09-18T00:18:16.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321727+0000", 2024-09-18T00:18:16.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0939999999999999e-05 2024-09-18T00:18:16.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321745+0000", 2024-09-18T00:18:16.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7986999999999999e-05 2024-09-18T00:18:16.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324259+0000", 2024-09-18T00:18:16.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025143069999999999 2024-09-18T00:18:16.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324291+0000", 2024-09-18T00:18:16.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2116999999999998e-05 2024-09-18T00:18:16.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329824+0000", 2024-09-18T00:18:16.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055326300000000002 2024-09-18T00:18:16.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329894+0000", 2024-09-18T00:18:16.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0142999999999996e-05 2024-09-18T00:18:16.284 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.284 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.284 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.285 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:136 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'94, mlcod=17'85)", 2024-09-18T00:18:16.285 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.322041+0000", 2024-09-18T00:18:16.285 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.927170201999999, 2024-09-18T00:18:16.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0079092060000000002, 2024-09-18T00:18:16.285 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.285 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.285 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.285 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322041+0000", 2024-09-18T00:18:16.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322041+0000", 2024-09-18T00:18:16.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322042+0000", 2024-09-18T00:18:16.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.22e-06 2024-09-18T00:18:16.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322053+0000", 2024-09-18T00:18:16.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1584e-05 2024-09-18T00:18:16.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322073+0000", 2024-09-18T00:18:16.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0023000000000001e-05 2024-09-18T00:18:16.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322092+0000", 2024-09-18T00:18:16.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8063e-05 2024-09-18T00:18:16.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324848+0000", 2024-09-18T00:18:16.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027561420000000001 2024-09-18T00:18:16.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324877+0000", 2024-09-18T00:18:16.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9003e-05 2024-09-18T00:18:16.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329900+0000", 2024-09-18T00:18:16.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050233650000000001 2024-09-18T00:18:16.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329950+0000", 2024-09-18T00:18:16.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9805999999999997e-05 2024-09-18T00:18:16.289 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.289 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.289 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.289 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:137 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'95, mlcod=17'85)", 2024-09-18T00:18:16.289 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.322465+0000", 2024-09-18T00:18:16.289 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.926745497999999, 2024-09-18T00:18:16.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0075526680000000002, 2024-09-18T00:18:16.290 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.290 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.290 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.290 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322465+0000", 2024-09-18T00:18:16.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322465+0000", 2024-09-18T00:18:16.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322467+0000", 2024-09-18T00:18:16.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5999999999999999e-06 2024-09-18T00:18:16.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322612+0000", 2024-09-18T00:18:16.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014553499999999999 2024-09-18T00:18:16.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322631+0000", 2024-09-18T00:18:16.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8627e-05 2024-09-18T00:18:16.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322645+0000", 2024-09-18T00:18:16.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4109e-05 2024-09-18T00:18:16.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325646+0000", 2024-09-18T00:18:16.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0030007179999999999 2024-09-18T00:18:16.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325674+0000", 2024-09-18T00:18:16.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7733e-05 2024-09-18T00:18:16.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329958+0000", 2024-09-18T00:18:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042841329999999999 2024-09-18T00:18:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330018+0000", 2024-09-18T00:18:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0213000000000001e-05 2024-09-18T00:18:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:138 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'96, mlcod=17'85)", 2024-09-18T00:18:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.323237+0000", 2024-09-18T00:18:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.925974351000001, 2024-09-18T00:18:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0068313540000000004, 2024-09-18T00:18:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323237+0000", 2024-09-18T00:18:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323237+0000", 2024-09-18T00:18:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323238+0000", 2024-09-18T00:18:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.127e-06 2024-09-18T00:18:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323249+0000", 2024-09-18T00:18:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1157e-05 2024-09-18T00:18:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323277+0000", 2024-09-18T00:18:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8178999999999999e-05 2024-09-18T00:18:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323287+0000", 2024-09-18T00:18:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9699999999999994e-06 2024-09-18T00:18:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327148+0000", 2024-09-18T00:18:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038609109999999999 2024-09-18T00:18:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327178+0000", 2024-09-18T00:18:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0156000000000001e-05 2024-09-18T00:18:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330022+0000", 2024-09-18T00:18:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028444579999999998 2024-09-18T00:18:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330068+0000", 2024-09-18T00:18:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5395999999999997e-05 2024-09-18T00:18:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:139 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'86)", 2024-09-18T00:18:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.323792+0000", 2024-09-18T00:18:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.925418702000002, 2024-09-18T00:18:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063243850000000001, 2024-09-18T00:18:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323792+0000", 2024-09-18T00:18:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323792+0000", 2024-09-18T00:18:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323793+0000", 2024-09-18T00:18:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4399999999999998e-07 2024-09-18T00:18:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323806+0000", 2024-09-18T00:18:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3346000000000001e-05 2024-09-18T00:18:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323824+0000", 2024-09-18T00:18:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.789e-05 2024-09-18T00:18:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323836+0000", 2024-09-18T00:18:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2082999999999999e-05 2024-09-18T00:18:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327773+0000", 2024-09-18T00:18:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039365099999999998 2024-09-18T00:18:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327802+0000", 2024-09-18T00:18:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8617000000000001e-05 2024-09-18T00:18:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330072+0000", 2024-09-18T00:18:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022704219999999998 2024-09-18T00:18:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330117+0000", 2024-09-18T00:18:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4573000000000002e-05 2024-09-18T00:18:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:140 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'86)", 2024-09-18T00:18:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.324263+0000", 2024-09-18T00:18:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.924948249, 2024-09-18T00:18:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0066665350000000003, 2024-09-18T00:18:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324263+0000", 2024-09-18T00:18:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324263+0000", 2024-09-18T00:18:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324264+0000", 2024-09-18T00:18:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-18T00:18:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324387+0000", 2024-09-18T00:18:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012346899999999999 2024-09-18T00:18:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324402+0000", 2024-09-18T00:18:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5087e-05 2024-09-18T00:18:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324412+0000", 2024-09-18T00:18:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5969999999999998e-06 2024-09-18T00:18:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.328364+0000", 2024-09-18T00:18:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039523559999999998 2024-09-18T00:18:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.328397+0000", 2024-09-18T00:18:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2733e-05 2024-09-18T00:18:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330864+0000", 2024-09-18T00:18:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0024676670000000002 2024-09-18T00:18:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330929+0000", 2024-09-18T00:18:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4665999999999994e-05 2024-09-18T00:18:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:141 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'86)", 2024-09-18T00:18:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.325030+0000", 2024-09-18T00:18:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.924181270999998, 2024-09-18T00:18:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006603462, 2024-09-18T00:18:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325030+0000", 2024-09-18T00:18:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325030+0000", 2024-09-18T00:18:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325031+0000", 2024-09-18T00:18:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.53e-06 2024-09-18T00:18:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325042+0000", 2024-09-18T00:18:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0506000000000001e-05 2024-09-18T00:18:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325080+0000", 2024-09-18T00:18:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8300000000000003e-05 2024-09-18T00:18:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325098+0000", 2024-09-18T00:18:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8539999999999999e-05 2024-09-18T00:18:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330164+0000", 2024-09-18T00:18:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050653709999999999 2024-09-18T00:18:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330194+0000", 2024-09-18T00:18:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0275999999999999e-05 2024-09-18T00:18:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.331562+0000", 2024-09-18T00:18:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0013680330000000001 2024-09-18T00:18:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.331633+0000", 2024-09-18T00:18:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0906000000000002e-05 2024-09-18T00:18:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:142 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'100, mlcod=17'98)", 2024-09-18T00:18:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.332949+0000", 2024-09-18T00:18:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.916261576, 2024-09-18T00:18:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059501479999999997, 2024-09-18T00:18:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.332949+0000", 2024-09-18T00:18:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.332949+0000", 2024-09-18T00:18:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.332951+0000", 2024-09-18T00:18:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-06 2024-09-18T00:18:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333024+0000", 2024-09-18T00:18:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3112000000000001e-05 2024-09-18T00:18:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333050+0000", 2024-09-18T00:18:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5673e-05 2024-09-18T00:18:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333063+0000", 2024-09-18T00:18:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2644e-05 2024-09-18T00:18:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333228+0000", 2024-09-18T00:18:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016579799999999999 2024-09-18T00:18:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333267+0000", 2024-09-18T00:18:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8189999999999999e-05 2024-09-18T00:18:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338834+0000", 2024-09-18T00:18:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055676679999999996 2024-09-18T00:18:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338899+0000", 2024-09-18T00:18:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5198999999999997e-05 2024-09-18T00:18:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:143 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'101, mlcod=17'98)", 2024-09-18T00:18:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.333556+0000", 2024-09-18T00:18:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.915654521, 2024-09-18T00:18:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005386053, 2024-09-18T00:18:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333556+0000", 2024-09-18T00:18:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333556+0000", 2024-09-18T00:18:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333558+0000", 2024-09-18T00:18:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:18:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333677+0000", 2024-09-18T00:18:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119196 2024-09-18T00:18:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333698+0000", 2024-09-18T00:18:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1087e-05 2024-09-18T00:18:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333708+0000", 2024-09-18T00:18:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0125999999999999e-05 2024-09-18T00:18:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334689+0000", 2024-09-18T00:18:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00098056899999999993 2024-09-18T00:18:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334719+0000", 2024-09-18T00:18:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9963000000000001e-05 2024-09-18T00:18:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338907+0000", 2024-09-18T00:18:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041881619999999996 2024-09-18T00:18:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338942+0000", 2024-09-18T00:18:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5660000000000001e-05 2024-09-18T00:18:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T00:18:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:07.405976+0000", 2024-09-18T00:18:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.8432352200000004, 2024-09-18T00:18:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063813710000000003, 2024-09-18T00:18:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11 2024-09-18T00:18:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.405976+0000", 2024-09-18T00:18:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.405976+0000", 2024-09-18T00:18:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.405979+0000", 2024-09-18T00:18:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9009999999999998e-06 2024-09-18T00:18:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.405998+0000", 2024-09-18T00:18:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9653e-05 2024-09-18T00:18:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.406027+0000", 2024-09-18T00:18:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8442e-05 2024-09-18T00:18:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.406041+0000", 2024-09-18T00:18:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4273e-05 2024-09-18T00:18:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.406141+0000", 2024-09-18T00:18:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010008199999999999 2024-09-18T00:18:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:18:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.406147+0000", 2024-09-18T00:18:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9340000000000003e-06 2024-09-18T00:18:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.408986+0000", 2024-09-18T00:18:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028392959999999998 2024-09-18T00:18:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.409279+0000", 2024-09-18T00:18:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000292596 2024-09-18T00:18:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.409615+0000", 2024-09-18T00:18:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033606 2024-09-18T00:18:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.411318+0000", 2024-09-18T00:18:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017026890000000001 2024-09-18T00:18:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.412269+0000", 2024-09-18T00:18:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00095177200000000003 2024-09-18T00:18:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.412319+0000", 2024-09-18T00:18:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9975999999999997e-05 2024-09-18T00:18:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.412357+0000", 2024-09-18T00:18:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7697000000000002e-05 2024-09-18T00:18:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:231 3.17 3:ee570356:::rbd_data.112a45c8e167.0000000000000037:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1398888~44282] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T00:18:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.796692+0000", 2024-09-18T00:18:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.452518564, 2024-09-18T00:18:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0077415899999999996, 2024-09-18T00:18:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 231 2024-09-18T00:18:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796692+0000", 2024-09-18T00:18:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796692+0000", 2024-09-18T00:18:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796694+0000", 2024-09-18T00:18:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.291e-06 2024-09-18T00:18:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796736+0000", 2024-09-18T00:18:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2082999999999997e-05 2024-09-18T00:18:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796747+0000", 2024-09-18T00:18:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1144999999999999e-05 2024-09-18T00:18:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796752+0000", 2024-09-18T00:18:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5439999999999998e-06 2024-09-18T00:18:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796844+0000", 2024-09-18T00:18:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1847000000000003e-05 2024-09-18T00:18:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796952+0000", 2024-09-18T00:18:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108223 2024-09-18T00:18:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.797040+0000", 2024-09-18T00:18:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7416000000000005e-05 2024-09-18T00:18:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.802075+0000", 2024-09-18T00:18:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050354390000000001 2024-09-18T00:18:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.804151+0000", 2024-09-18T00:18:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0020752769999999999 2024-09-18T00:18:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.804314+0000", 2024-09-18T00:18:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016367899999999999 2024-09-18T00:18:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.804434+0000", 2024-09-18T00:18:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119646 2024-09-18T00:18:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:763 3.7 e30/27 3:e4fee1a8:::rbd_data.112ae20746f5.00000000000001cb:head v 30'6, mlcod=29'4)", 2024-09-18T00:18:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.458804+0000", 2024-09-18T00:18:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.790407197, 2024-09-18T00:18:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0060225469999999996, 2024-09-18T00:18:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458804+0000", 2024-09-18T00:18:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458804+0000", 2024-09-18T00:18:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458804+0000", 2024-09-18T00:18:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7599999999999998e-07 2024-09-18T00:18:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458809+0000", 2024-09-18T00:18:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000001e-06 2024-09-18T00:18:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458821+0000", 2024-09-18T00:18:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2176e-05 2024-09-18T00:18:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458830+0000", 2024-09-18T00:18:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1859999999999992e-06 2024-09-18T00:18:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458929+0000", 2024-09-18T00:18:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9661999999999995e-05 2024-09-18T00:18:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458950+0000", 2024-09-18T00:18:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0424e-05 2024-09-18T00:18:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.464761+0000", 2024-09-18T00:18:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0058110389999999996 2024-09-18T00:18:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.464826+0000", 2024-09-18T00:18:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5563999999999998e-05 2024-09-18T00:18:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:758 3.f e30/27 3:f05256a6:::rbd_data.112ae20746f5.000000000000009c:head v 30'2, mlcod=0'0)", 2024-09-18T00:18:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.459619+0000", 2024-09-18T00:18:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.7895919170000001, 2024-09-18T00:18:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052728799999999998, 2024-09-18T00:18:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459619+0000", 2024-09-18T00:18:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459619+0000", 2024-09-18T00:18:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459619+0000", 2024-09-18T00:18:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.65e-07 2024-09-18T00:18:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459624+0000", 2024-09-18T00:18:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4240000000000003e-06 2024-09-18T00:18:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459635+0000", 2024-09-18T00:18:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1273e-05 2024-09-18T00:18:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459642+0000", 2024-09-18T00:18:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.187e-06 2024-09-18T00:18:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459713+0000", 2024-09-18T00:18:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0434999999999999e-05 2024-09-18T00:18:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459740+0000", 2024-09-18T00:18:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7177000000000001e-05 2024-09-18T00:18:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.464831+0000", 2024-09-18T00:18:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050911899999999998 2024-09-18T00:18:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.464892+0000", 2024-09-18T00:18:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0928999999999999e-05 2024-09-18T00:18:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:759 3.3 e30/23 3:c17b399b:::rbd_data.112ae20746f5.00000000000000ce:head v 30'8, mlcod=29'6)", 2024-09-18T00:18:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.459662+0000", 2024-09-18T00:18:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.7895491940000001, 2024-09-18T00:18:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052242, 2024-09-18T00:18:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459662+0000", 2024-09-18T00:18:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459662+0000", 2024-09-18T00:18:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459662+0000", 2024-09-18T00:18:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1899999999999998e-07 2024-09-18T00:18:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459666+0000", 2024-09-18T00:18:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9770000000000002e-06 2024-09-18T00:18:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459673+0000", 2024-09-18T00:18:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5509999999999996e-06 2024-09-18T00:18:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459677+0000", 2024-09-18T00:18:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8509999999999998e-06 2024-09-18T00:18:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459755+0000", 2024-09-18T00:18:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7887999999999997e-05 2024-09-18T00:18:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459775+0000", 2024-09-18T00:18:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9324e-05 2024-09-18T00:18:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.464829+0000", 2024-09-18T00:18:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050545399999999997 2024-09-18T00:18:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.464886+0000", 2024-09-18T00:18:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6749999999999997e-05 2024-09-18T00:18:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.349 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:16.350 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:18:16.351 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:16.351+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:16.418 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is set 2024-09-18T00:18:16.451 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:16.447+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:16.524 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:16.524 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:16.525 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:16.525 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:16.539 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:18:16.552 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:16.547+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:16.653 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:16.647+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:16.655 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:16.655 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:16.655 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:16.655 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:16.655 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:16.672 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:18:16.753 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:16.747+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:16.799 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:220 3.19 3:9e8c0c63:::rbd_data.112a45c8e167.0000000000000031:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T00:18:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.761391+0000", 2024-09-18T00:18:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.0354451679999999, 2024-09-18T00:18:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062183000000000004, 2024-09-18T00:18:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.800 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 220 2024-09-18T00:18:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761391+0000", 2024-09-18T00:18:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761391+0000", 2024-09-18T00:18:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761392+0000", 2024-09-18T00:18:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.838e-06 2024-09-18T00:18:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761407+0000", 2024-09-18T00:18:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4538999999999999e-05 2024-09-18T00:18:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761421+0000", 2024-09-18T00:18:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3896e-05 2024-09-18T00:18:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761427+0000", 2024-09-18T00:18:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2519999999999996e-06 2024-09-18T00:18:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761507+0000", 2024-09-18T00:18:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9909999999999999e-05 2024-09-18T00:18:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761697+0000", 2024-09-18T00:18:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000189834 2024-09-18T00:18:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-18T00:18:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.762095+0000", 2024-09-18T00:18:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00039823799999999998 2024-09-18T00:18:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767289+0000", 2024-09-18T00:18:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0051936930000000001 2024-09-18T00:18:16.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767536+0000", 2024-09-18T00:18:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024718899999999998 2024-09-18T00:18:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767573+0000", 2024-09-18T00:18:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6792000000000003e-05 2024-09-18T00:18:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767609+0000", 2024-09-18T00:18:16.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6118999999999998e-05 2024-09-18T00:18:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:220 3.19 e28/23 ondisk, result = 0)", 2024-09-18T00:18:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.763702+0000", 2024-09-18T00:18:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.0331337109999996, 2024-09-18T00:18:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035976879999999999, 2024-09-18T00:18:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:16.806 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763702+0000", 2024-09-18T00:18:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763702+0000", 2024-09-18T00:18:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763703+0000", 2024-09-18T00:18:16.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5099999999999999e-07 2024-09-18T00:18:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763707+0000", 2024-09-18T00:18:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7450000000000003e-06 2024-09-18T00:18:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763720+0000", 2024-09-18T00:18:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2215000000000001e-05 2024-09-18T00:18:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763725+0000", 2024-09-18T00:18:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.604e-06 2024-09-18T00:18:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767267+0000", 2024-09-18T00:18:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035417729999999998 2024-09-18T00:18:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767279+0000", 2024-09-18T00:18:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2267e-05 2024-09-18T00:18:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767300+0000", 2024-09-18T00:18:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0333e-05 2024-09-18T00:18:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:372 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.457128+0000", 2024-09-18T00:18:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.3397072449999996, 2024-09-18T00:18:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003527082, 2024-09-18T00:18:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.810 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 372 2024-09-18T00:18:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457128+0000", 2024-09-18T00:18:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457128+0000", 2024-09-18T00:18:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457131+0000", 2024-09-18T00:18:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9739999999999998e-06 2024-09-18T00:18:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457146+0000", 2024-09-18T00:18:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4516e-05 2024-09-18T00:18:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457178+0000", 2024-09-18T00:18:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2475999999999999e-05 2024-09-18T00:18:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457199+0000", 2024-09-18T00:18:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:18:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457348+0000", 2024-09-18T00:18:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014881200000000001 2024-09-18T00:18:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457602+0000", 2024-09-18T00:18:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000253732 2024-09-18T00:18:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457977+0000", 2024-09-18T00:18:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037547299999999997 2024-09-18T00:18:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.459553+0000", 2024-09-18T00:18:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001576261 2024-09-18T00:18:16.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460497+0000", 2024-09-18T00:18:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094395500000000001 2024-09-18T00:18:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460588+0000", 2024-09-18T00:18:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0443000000000001e-05 2024-09-18T00:18:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460656+0000", 2024-09-18T00:18:16.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7650000000000005e-05 2024-09-18T00:18:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:373 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.set_stripe_unit_count in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.460868+0000", 2024-09-18T00:18:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.3359673369999996, 2024-09-18T00:18:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031188050000000001, 2024-09-18T00:18:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:16.816 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 373 2024-09-18T00:18:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460868+0000", 2024-09-18T00:18:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460868+0000", 2024-09-18T00:18:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460870+0000", 2024-09-18T00:18:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.733e-06 2024-09-18T00:18:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460880+0000", 2024-09-18T00:18:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5429999999999999e-06 2024-09-18T00:18:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460913+0000", 2024-09-18T00:18:16.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3333000000000001e-05 2024-09-18T00:18:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460937+0000", 2024-09-18T00:18:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3836999999999999e-05 2024-09-18T00:18:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.461083+0000", 2024-09-18T00:18:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145952 2024-09-18T00:18:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.461327+0000", 2024-09-18T00:18:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024425100000000002 2024-09-18T00:18:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.461895+0000", 2024-09-18T00:18:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056844600000000001 2024-09-18T00:18:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.462901+0000", 2024-09-18T00:18:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001005939 2024-09-18T00:18:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.463800+0000", 2024-09-18T00:18:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00089826999999999995 2024-09-18T00:18:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.463885+0000", 2024-09-18T00:18:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5768999999999998e-05 2024-09-18T00:18:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.463987+0000", 2024-09-18T00:18:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010173199999999999 2024-09-18T00:18:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.821 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:387 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [watch watch cookie 140686016265056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.485197+0000", 2024-09-18T00:18:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.3116390129999997, 2024-09-18T00:18:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029145590000000002, 2024-09-18T00:18:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:16.822 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 387 2024-09-18T00:18:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485197+0000", 2024-09-18T00:18:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485197+0000", 2024-09-18T00:18:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485199+0000", 2024-09-18T00:18:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4849999999999999e-06 2024-09-18T00:18:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485209+0000", 2024-09-18T00:18:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0251999999999999e-05 2024-09-18T00:18:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485238+0000", 2024-09-18T00:18:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8708e-05 2024-09-18T00:18:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485255+0000", 2024-09-18T00:18:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7011e-05 2024-09-18T00:18:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485400+0000", 2024-09-18T00:18:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144504 2024-09-18T00:18:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485654+0000", 2024-09-18T00:18:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025417400000000002 2024-09-18T00:18:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.486056+0000", 2024-09-18T00:18:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000402147 2024-09-18T00:18:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.487262+0000", 2024-09-18T00:18:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0012063480000000001 2024-09-18T00:18:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.487908+0000", 2024-09-18T00:18:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00064552100000000003 2024-09-18T00:18:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.487993+0000", 2024-09-18T00:18:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5263e-05 2024-09-18T00:18:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488111+0000", 2024-09-18T00:18:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000118146 2024-09-18T00:18:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:388 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.parent_attach in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.488320+0000", 2024-09-18T00:18:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.3085156629999997, 2024-09-18T00:18:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0030081399999999999, 2024-09-18T00:18:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 388 2024-09-18T00:18:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488320+0000", 2024-09-18T00:18:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488320+0000", 2024-09-18T00:18:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488322+0000", 2024-09-18T00:18:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-06 2024-09-18T00:18:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488331+0000", 2024-09-18T00:18:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5880000000000004e-06 2024-09-18T00:18:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488356+0000", 2024-09-18T00:18:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4403e-05 2024-09-18T00:18:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488368+0000", 2024-09-18T00:18:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2690999999999999e-05 2024-09-18T00:18:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488485+0000", 2024-09-18T00:18:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011662099999999999 2024-09-18T00:18:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488623+0000", 2024-09-18T00:18:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000137472 2024-09-18T00:18:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.489032+0000", 2024-09-18T00:18:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00040919699999999998 2024-09-18T00:18:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.490129+0000", 2024-09-18T00:18:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001096765 2024-09-18T00:18:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.491175+0000", 2024-09-18T00:18:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0010461380000000001 2024-09-18T00:18:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.491259+0000", 2024-09-18T00:18:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4625999999999999e-05 2024-09-18T00:18:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.491328+0000", 2024-09-18T00:18:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8895999999999999e-05 2024-09-18T00:18:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:554 3.8 e29/23 3:14210294:::rbd_data.112ae20746f5.0000000000000133:head v 29'1, mlcod=0'0)", 2024-09-18T00:18:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.879349+0000", 2024-09-18T00:18:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.9174867410000003, 2024-09-18T00:18:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043514469999999996, 2024-09-18T00:18:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879349+0000", 2024-09-18T00:18:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879349+0000", 2024-09-18T00:18:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879351+0000", 2024-09-18T00:18:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8050000000000001e-06 2024-09-18T00:18:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879392+0000", 2024-09-18T00:18:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1054999999999999e-05 2024-09-18T00:18:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879413+0000", 2024-09-18T00:18:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0784999999999998e-05 2024-09-18T00:18:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879425+0000", 2024-09-18T00:18:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2896e-05 2024-09-18T00:18:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879540+0000", 2024-09-18T00:18:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011473500000000001 2024-09-18T00:18:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879579+0000", 2024-09-18T00:18:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8578999999999998e-05 2024-09-18T00:18:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.883623+0000", 2024-09-18T00:18:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040441959999999999 2024-09-18T00:18:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.883700+0000", 2024-09-18T00:18:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7396000000000002e-05 2024-09-18T00:18:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:557 3.e e29/27 3:736a090f:::rbd_data.112ae20746f5.000000000000008e:head v 29'6, mlcod=28'4)", 2024-09-18T00:18:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.890014+0000", 2024-09-18T00:18:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.9068214030000004, 2024-09-18T00:18:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002682841, 2024-09-18T00:18:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.890014+0000", 2024-09-18T00:18:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.890014+0000", 2024-09-18T00:18:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.890017+0000", 2024-09-18T00:18:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4399999999999999e-06 2024-09-18T00:18:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.890202+0000", 2024-09-18T00:18:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018565899999999999 2024-09-18T00:18:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.890232+0000", 2024-09-18T00:18:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9416000000000001e-05 2024-09-18T00:18:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.890246+0000", 2024-09-18T00:18:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4207e-05 2024-09-18T00:18:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.890369+0000", 2024-09-18T00:18:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012293199999999999 2024-09-18T00:18:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.890413+0000", 2024-09-18T00:18:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4545000000000001e-05 2024-09-18T00:18:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.892644+0000", 2024-09-18T00:18:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022309869999999998 2024-09-18T00:18:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.892697+0000", 2024-09-18T00:18:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2655000000000003e-05 2024-09-18T00:18:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:612 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.044307+0000", 2024-09-18T00:18:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.7525285139999998, 2024-09-18T00:18:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027702650000000001, 2024-09-18T00:18:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 612 2024-09-18T00:18:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.044307+0000", 2024-09-18T00:18:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.044307+0000", 2024-09-18T00:18:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.044311+0000", 2024-09-18T00:18:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4800000000000001e-06 2024-09-18T00:18:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.044326+0000", 2024-09-18T00:18:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5440000000000001e-05 2024-09-18T00:18:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.044363+0000", 2024-09-18T00:18:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7073000000000002e-05 2024-09-18T00:18:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.044381+0000", 2024-09-18T00:18:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7606999999999999e-05 2024-09-18T00:18:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.044505+0000", 2024-09-18T00:18:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012385200000000001 2024-09-18T00:18:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.044662+0000", 2024-09-18T00:18:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015708599999999999 2024-09-18T00:18:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.045002+0000", 2024-09-18T00:18:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034022000000000002 2024-09-18T00:18:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.046136+0000", 2024-09-18T00:18:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001133974 2024-09-18T00:18:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.046898+0000", 2024-09-18T00:18:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00076233200000000001 2024-09-18T00:18:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.046995+0000", 2024-09-18T00:18:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6695999999999997e-05 2024-09-18T00:18:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.047077+0000", 2024-09-18T00:18:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2504999999999997e-05 2024-09-18T00:18:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:668 3.1a e29/23 3:59ee8e8b:::rbd_data.112ae20746f5.0000000000000031:head v 29'3, mlcod=25'1)", 2024-09-18T00:18:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.170930+0000", 2024-09-18T00:18:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.6259056019999996, 2024-09-18T00:18:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052043310000000004, 2024-09-18T00:18:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.170930+0000", 2024-09-18T00:18:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.170930+0000", 2024-09-18T00:18:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.170932+0000", 2024-09-18T00:18:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4699999999999999e-06 2024-09-18T00:18:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171038+0000", 2024-09-18T00:18:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106205 2024-09-18T00:18:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171061+0000", 2024-09-18T00:18:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3329999999999999e-05 2024-09-18T00:18:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171072+0000", 2024-09-18T00:18:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0806e-05 2024-09-18T00:18:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171201+0000", 2024-09-18T00:18:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012925799999999999 2024-09-18T00:18:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171239+0000", 2024-09-18T00:18:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7866e-05 2024-09-18T00:18:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.176016+0000", 2024-09-18T00:18:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047769930000000002 2024-09-18T00:18:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.176134+0000", 2024-09-18T00:18:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000118403 2024-09-18T00:18:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:692 3.0 3:03fab4ad:::rbd_data.112ae20746f5.0000000000000057:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.238016+0000", 2024-09-18T00:18:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.5588198919999998, 2024-09-18T00:18:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0087112969999999998, 2024-09-18T00:18:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 692 2024-09-18T00:18:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238016+0000", 2024-09-18T00:18:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238016+0000", 2024-09-18T00:18:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238017+0000", 2024-09-18T00:18:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.257e-06 2024-09-18T00:18:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238023+0000", 2024-09-18T00:18:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5470000000000004e-06 2024-09-18T00:18:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238046+0000", 2024-09-18T00:18:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3088e-05 2024-09-18T00:18:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238054+0000", 2024-09-18T00:18:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3569999999999998e-06 2024-09-18T00:18:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238135+0000", 2024-09-18T00:18:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0814000000000002e-05 2024-09-18T00:18:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238331+0000", 2024-09-18T00:18:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019594 2024-09-18T00:18:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238606+0000", 2024-09-18T00:18:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027473200000000003 2024-09-18T00:18:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.240010+0000", 2024-09-18T00:18:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001404329 2024-09-18T00:18:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246563+0000", 2024-09-18T00:18:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0065535630000000001 2024-09-18T00:18:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246643+0000", 2024-09-18T00:18:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9468999999999994e-05 2024-09-18T00:18:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246727+0000", 2024-09-18T00:18:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4201000000000002e-05 2024-09-18T00:18:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:694 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.266659+0000", 2024-09-18T00:18:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.5301768400000002, 2024-09-18T00:18:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002821919, 2024-09-18T00:18:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 694 2024-09-18T00:18:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.266659+0000", 2024-09-18T00:18:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.266659+0000", 2024-09-18T00:18:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.266662+0000", 2024-09-18T00:18:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1970000000000001e-06 2024-09-18T00:18:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.266677+0000", 2024-09-18T00:18:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5418999999999999e-05 2024-09-18T00:18:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.266712+0000", 2024-09-18T00:18:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4944000000000003e-05 2024-09-18T00:18:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.266731+0000", 2024-09-18T00:18:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8559999999999998e-05 2024-09-18T00:18:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.266860+0000", 2024-09-18T00:18:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012881500000000001 2024-09-18T00:18:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.267021+0000", 2024-09-18T00:18:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016144900000000001 2024-09-18T00:18:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.267393+0000", 2024-09-18T00:18:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037151399999999998 2024-09-18T00:18:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.268580+0000", 2024-09-18T00:18:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001186868 2024-09-18T00:18:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.269186+0000", 2024-09-18T00:18:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00060663500000000001 2024-09-18T00:18:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.269352+0000", 2024-09-18T00:18:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016532499999999999 2024-09-18T00:18:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.269481+0000", 2024-09-18T00:18:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012919299999999999 2024-09-18T00:18:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:704 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.293962+0000", 2024-09-18T00:18:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.5028734039999998, 2024-09-18T00:18:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028652980000000001, 2024-09-18T00:18:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 704 2024-09-18T00:18:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.293962+0000", 2024-09-18T00:18:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.293962+0000", 2024-09-18T00:18:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.293966+0000", 2024-09-18T00:18:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4800000000000001e-06 2024-09-18T00:18:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.293980+0000", 2024-09-18T00:18:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4399999999999999e-05 2024-09-18T00:18:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.294017+0000", 2024-09-18T00:18:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6755999999999998e-05 2024-09-18T00:18:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.294035+0000", 2024-09-18T00:18:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7895000000000001e-05 2024-09-18T00:18:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.294156+0000", 2024-09-18T00:18:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012162700000000001 2024-09-18T00:18:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.294302+0000", 2024-09-18T00:18:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145119 2024-09-18T00:18:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.294591+0000", 2024-09-18T00:18:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000289924 2024-09-18T00:18:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.295933+0000", 2024-09-18T00:18:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00134128 2024-09-18T00:18:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.296528+0000", 2024-09-18T00:18:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00059507899999999999 2024-09-18T00:18:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.296693+0000", 2024-09-18T00:18:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016479600000000001 2024-09-18T00:18:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.296828+0000", 2024-09-18T00:18:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013494200000000001 2024-09-18T00:18:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:733 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.365564+0000", 2024-09-18T00:18:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4312720920000004, 2024-09-18T00:18:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038647540000000002, 2024-09-18T00:18:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 733 2024-09-18T00:18:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365564+0000", 2024-09-18T00:18:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365564+0000", 2024-09-18T00:18:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365566+0000", 2024-09-18T00:18:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1909999999999999e-06 2024-09-18T00:18:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365589+0000", 2024-09-18T00:18:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3096999999999999e-05 2024-09-18T00:18:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365616+0000", 2024-09-18T00:18:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7577e-05 2024-09-18T00:18:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365628+0000", 2024-09-18T00:18:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1804000000000001e-05 2024-09-18T00:18:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365708+0000", 2024-09-18T00:18:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9657e-05 2024-09-18T00:18:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365797+0000", 2024-09-18T00:18:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8812000000000003e-05 2024-09-18T00:18:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365976+0000", 2024-09-18T00:18:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000178853 2024-09-18T00:18:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.367005+0000", 2024-09-18T00:18:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001029196 2024-09-18T00:18:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.369312+0000", 2024-09-18T00:18:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0023069079999999999 2024-09-18T00:18:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.369375+0000", 2024-09-18T00:18:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2823000000000005e-05 2024-09-18T00:18:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.369428+0000", 2024-09-18T00:18:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3835999999999998e-05 2024-09-18T00:18:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:746 3.0 3:03157719:::rbd_data.112ae20746f5.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.389598+0000", 2024-09-18T00:18:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4072381250000001, 2024-09-18T00:18:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006052516, 2024-09-18T00:18:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 746 2024-09-18T00:18:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389598+0000", 2024-09-18T00:18:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389598+0000", 2024-09-18T00:18:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389599+0000", 2024-09-18T00:18:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.844e-06 2024-09-18T00:18:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389609+0000", 2024-09-18T00:18:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1039999999999996e-06 2024-09-18T00:18:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389623+0000", 2024-09-18T00:18:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4384e-05 2024-09-18T00:18:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389632+0000", 2024-09-18T00:18:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5589999999999994e-06 2024-09-18T00:18:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389715+0000", 2024-09-18T00:18:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2185e-05 2024-09-18T00:18:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389923+0000", 2024-09-18T00:18:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000208214 2024-09-18T00:18:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.390263+0000", 2024-09-18T00:18:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033984 2024-09-18T00:18:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392457+0000", 2024-09-18T00:18:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021944180000000001 2024-09-18T00:18:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395565+0000", 2024-09-18T00:18:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031083450000000002 2024-09-18T00:18:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395616+0000", 2024-09-18T00:18:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0949000000000003e-05 2024-09-18T00:18:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395650+0000", 2024-09-18T00:18:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3674000000000003e-05 2024-09-18T00:18:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:749 3.1a e29/23 3:5e11336d:::rbd_data.112ae20746f5.000000000000004e:head v 29'4, mlcod=26'2)", 2024-09-18T00:18:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.403303+0000", 2024-09-18T00:18:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.3935331550000001, 2024-09-18T00:18:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0044570729999999998, 2024-09-18T00:18:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403303+0000", 2024-09-18T00:18:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403303+0000", 2024-09-18T00:18:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403303+0000", 2024-09-18T00:18:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6300000000000004e-07 2024-09-18T00:18:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403446+0000", 2024-09-18T00:18:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014246500000000001 2024-09-18T00:18:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403461+0000", 2024-09-18T00:18:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4904e-05 2024-09-18T00:18:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403470+0000", 2024-09-18T00:18:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9020000000000005e-06 2024-09-18T00:18:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403582+0000", 2024-09-18T00:18:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112161 2024-09-18T00:18:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403610+0000", 2024-09-18T00:18:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7671e-05 2024-09-18T00:18:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.407712+0000", 2024-09-18T00:18:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041027449999999997 2024-09-18T00:18:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.407760+0000", 2024-09-18T00:18:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7361999999999998e-05 2024-09-18T00:18:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:754 3.1a e30/23 3:5e11336d:::rbd_data.112ae20746f5.000000000000004e:head v 30'5, mlcod=29'3)", 2024-09-18T00:18:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.459551+0000", 2024-09-18T00:18:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.3372849410000001, 2024-09-18T00:18:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029016850000000002, 2024-09-18T00:18:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459551+0000", 2024-09-18T00:18:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459551+0000", 2024-09-18T00:18:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459552+0000", 2024-09-18T00:18:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9300000000000006e-07 2024-09-18T00:18:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459557+0000", 2024-09-18T00:18:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3700000000000003e-06 2024-09-18T00:18:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459584+0000", 2024-09-18T00:18:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.709e-05 2024-09-18T00:18:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459593+0000", 2024-09-18T00:18:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4009999999999997e-06 2024-09-18T00:18:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459650+0000", 2024-09-18T00:18:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7436999999999999e-05 2024-09-18T00:18:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459671+0000", 2024-09-18T00:18:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0704999999999999e-05 2024-09-18T00:18:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.462395+0000", 2024-09-18T00:18:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027243990000000002 2024-09-18T00:18:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.462452+0000", 2024-09-18T00:18:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7290000000000002e-05 2024-09-18T00:18:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:839 3.0 3:0095abde:::rbd_data.112ae20746f5.0000000000000116:head [set-alloc-hint object_size 524288 write_size 524288,write 81590~25537] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.774325+0000", 2024-09-18T00:18:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.0225107910000002, 2024-09-18T00:18:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033986020000000001, 2024-09-18T00:18:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 839 2024-09-18T00:18:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774325+0000", 2024-09-18T00:18:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774325+0000", 2024-09-18T00:18:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774326+0000", 2024-09-18T00:18:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2419999999999999e-06 2024-09-18T00:18:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774342+0000", 2024-09-18T00:18:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5586999999999998e-05 2024-09-18T00:18:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774357+0000", 2024-09-18T00:18:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4986e-05 2024-09-18T00:18:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774367+0000", 2024-09-18T00:18:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9909999999999997e-06 2024-09-18T00:18:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774446+0000", 2024-09-18T00:18:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9419000000000007e-05 2024-09-18T00:18:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774611+0000", 2024-09-18T00:18:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016517600000000001 2024-09-18T00:18:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774804+0000", 2024-09-18T00:18:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019286300000000001 2024-09-18T00:18:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.777346+0000", 2024-09-18T00:18:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025422399999999999 2024-09-18T00:18:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.777606+0000", 2024-09-18T00:18:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025912400000000001 2024-09-18T00:18:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.777670+0000", 2024-09-18T00:18:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4785999999999999e-05 2024-09-18T00:18:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.777724+0000", 2024-09-18T00:18:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3188e-05 2024-09-18T00:18:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:870 3.0 3:05335421:::rbd_data.112ae20746f5.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~43670] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.955002+0000", 2024-09-18T00:18:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.8418339669999999, 2024-09-18T00:18:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055249590000000003, 2024-09-18T00:18:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 870 2024-09-18T00:18:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955002+0000", 2024-09-18T00:18:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955002+0000", 2024-09-18T00:18:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955004+0000", 2024-09-18T00:18:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0760000000000001e-06 2024-09-18T00:18:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955038+0000", 2024-09-18T00:18:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4008999999999999e-05 2024-09-18T00:18:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955053+0000", 2024-09-18T00:18:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5410999999999999e-05 2024-09-18T00:18:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955060+0000", 2024-09-18T00:18:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7340000000000002e-06 2024-09-18T00:18:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955139+0000", 2024-09-18T00:18:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8733999999999995e-05 2024-09-18T00:18:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955356+0000", 2024-09-18T00:18:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021757600000000001 2024-09-18T00:18:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955658+0000", 2024-09-18T00:18:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00030183500000000003 2024-09-18T00:18:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960123+0000", 2024-09-18T00:18:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0044650269999999999 2024-09-18T00:18:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960423+0000", 2024-09-18T00:18:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000300346 2024-09-18T00:18:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960476+0000", 2024-09-18T00:18:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.295e-05 2024-09-18T00:18:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960527+0000", 2024-09-18T00:18:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0260999999999999e-05 2024-09-18T00:18:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:1563 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T00:18:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:15.425150+0000", 2024-09-18T00:18:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.3716853950000001, 2024-09-18T00:18:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0034638830000000001, 2024-09-18T00:18:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1563 2024-09-18T00:18:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425150+0000", 2024-09-18T00:18:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425150+0000", 2024-09-18T00:18:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425152+0000", 2024-09-18T00:18:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.018e-06 2024-09-18T00:18:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425159+0000", 2024-09-18T00:18:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0319999999999998e-06 2024-09-18T00:18:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425177+0000", 2024-09-18T00:18:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7915999999999999e-05 2024-09-18T00:18:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425189+0000", 2024-09-18T00:18:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1280999999999999e-05 2024-09-18T00:18:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425275+0000", 2024-09-18T00:18:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6277000000000001e-05 2024-09-18T00:18:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425408+0000", 2024-09-18T00:18:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013278199999999999 2024-09-18T00:18:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425923+0000", 2024-09-18T00:18:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051550100000000002 2024-09-18T00:18:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.427067+0000", 2024-09-18T00:18:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001144396 2024-09-18T00:18:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.428466+0000", 2024-09-18T00:18:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0013989060000000001 2024-09-18T00:18:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.428547+0000", 2024-09-18T00:18:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0549000000000004e-05 2024-09-18T00:18:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.428614+0000", 2024-09-18T00:18:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7224999999999994e-05 2024-09-18T00:18:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:16.907 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:16.908 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:18:16.908 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:16.903+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:17.009 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:17.007+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:17.031 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:17.031 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:17.044 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:18:17.109 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:17.103+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:17.210 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:17.207+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:17.211 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:17.211 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:17.212 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:17.212 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:17.212 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:17.225 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:18:17.311 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:17.307+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:17.390 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:17.390 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:17.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:17.391 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:17.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.391 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4263.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-18T00:18:17.391 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:55.481299+0000", 2024-09-18T00:18:17.391 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 21.906917241999999, 2024-09-18T00:18:17.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036304219999999999, 2024-09-18T00:18:17.391 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.391 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.391 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.391 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481299+0000", 2024-09-18T00:18:17.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481299+0000", 2024-09-18T00:18:17.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481300+0000", 2024-09-18T00:18:17.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3e-07 2024-09-18T00:18:17.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481307+0000", 2024-09-18T00:18:17.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4959999999999999e-06 2024-09-18T00:18:17.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481330+0000", 2024-09-18T00:18:17.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3363999999999999e-05 2024-09-18T00:18:17.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481347+0000", 2024-09-18T00:18:17.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6263e-05 2024-09-18T00:18:17.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481428+0000", 2024-09-18T00:18:17.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1612000000000004e-05 2024-09-18T00:18:17.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481495+0000", 2024-09-18T00:18:17.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6439999999999994e-05 2024-09-18T00:18:17.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.484828+0000", 2024-09-18T00:18:17.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033337140000000002 2024-09-18T00:18:17.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.484929+0000", 2024-09-18T00:18:17.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100803 2024-09-18T00:18:17.395 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.395 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.395 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.395 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:285 3.9 e28/23 3:94c9a3ca:::rbd_data.112a45c8e167.000000000000001c:head v 28'1, mlcod=0'0)", 2024-09-18T00:18:17.395 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.892623+0000", 2024-09-18T00:18:17.395 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.4955925450000001, 2024-09-18T00:18:17.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040991170000000002, 2024-09-18T00:18:17.395 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.396 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.396 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.396 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892623+0000", 2024-09-18T00:18:17.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892623+0000", 2024-09-18T00:18:17.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892655+0000", 2024-09-18T00:18:17.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1377000000000001e-05 2024-09-18T00:18:17.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892819+0000", 2024-09-18T00:18:17.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000164275 2024-09-18T00:18:17.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892851+0000", 2024-09-18T00:18:17.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1696000000000003e-05 2024-09-18T00:18:17.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892873+0000", 2024-09-18T00:18:17.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.26e-05 2024-09-18T00:18:17.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892958+0000", 2024-09-18T00:18:17.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4245000000000004e-05 2024-09-18T00:18:17.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892975+0000", 2024-09-18T00:18:17.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7240000000000001e-05 2024-09-18T00:18:17.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.896666+0000", 2024-09-18T00:18:17.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003691083 2024-09-18T00:18:17.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.896723+0000", 2024-09-18T00:18:17.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6601000000000002e-05 2024-09-18T00:18:17.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.399 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.400 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:288 3.9 e28/23 3:94c9a3ca:::rbd_data.112a45c8e167.000000000000001c:head v 28'2, mlcod=0'0)", 2024-09-18T00:18:17.400 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.924244+0000", 2024-09-18T00:18:17.400 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.4639717189999999, 2024-09-18T00:18:17.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0014516939999999999, 2024-09-18T00:18:17.400 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.400 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.400 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.400 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.924244+0000", 2024-09-18T00:18:17.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.924244+0000", 2024-09-18T00:18:17.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.924247+0000", 2024-09-18T00:18:17.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8270000000000002e-06 2024-09-18T00:18:17.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.924265+0000", 2024-09-18T00:18:17.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7943000000000001e-05 2024-09-18T00:18:17.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.924300+0000", 2024-09-18T00:18:17.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5023e-05 2024-09-18T00:18:17.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.924322+0000", 2024-09-18T00:18:17.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.194e-05 2024-09-18T00:18:17.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.924447+0000", 2024-09-18T00:18:17.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012452000000000001 2024-09-18T00:18:17.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.924474+0000", 2024-09-18T00:18:17.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7004000000000001e-05 2024-09-18T00:18:17.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.925639+0000", 2024-09-18T00:18:17.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011659019999999999 2024-09-18T00:18:17.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.925696+0000", 2024-09-18T00:18:17.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6535e-05 2024-09-18T00:18:17.404 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.404 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.404 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.404 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:318 3.9 e28/23 3:945ebdd9:::rbd_data.112a45c8e167.0000000000000012:head v 28'3, mlcod=28'1)", 2024-09-18T00:18:17.404 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.996252+0000", 2024-09-18T00:18:17.404 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.391963702, 2024-09-18T00:18:17.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0024507719999999999, 2024-09-18T00:18:17.404 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.404 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.404 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.404 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.996252+0000", 2024-09-18T00:18:17.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.996252+0000", 2024-09-18T00:18:17.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.996254+0000", 2024-09-18T00:18:17.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6440000000000001e-06 2024-09-18T00:18:17.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.996280+0000", 2024-09-18T00:18:17.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6480000000000001e-05 2024-09-18T00:18:17.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.996299+0000", 2024-09-18T00:18:17.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.855e-05 2024-09-18T00:18:17.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.996308+0000", 2024-09-18T00:18:17.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7630000000000006e-06 2024-09-18T00:18:17.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.996404+0000", 2024-09-18T00:18:17.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6398999999999996e-05 2024-09-18T00:18:17.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.996446+0000", 2024-09-18T00:18:17.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1609000000000001e-05 2024-09-18T00:18:17.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.998595+0000", 2024-09-18T00:18:17.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002149134 2024-09-18T00:18:17.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.998703+0000", 2024-09-18T00:18:17.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108193 2024-09-18T00:18:17.408 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.408 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.408 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.408 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:322 3.9 e28/23 3:945ebdd9:::rbd_data.112a45c8e167.0000000000000012:head v 28'4, mlcod=28'2)", 2024-09-18T00:18:17.408 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.032802+0000", 2024-09-18T00:18:17.409 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.3554144880000001, 2024-09-18T00:18:17.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0014823499999999999, 2024-09-18T00:18:17.409 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.409 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.409 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.409 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.032802+0000", 2024-09-18T00:18:17.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.032802+0000", 2024-09-18T00:18:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.032804+0000", 2024-09-18T00:18:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5770000000000001e-06 2024-09-18T00:18:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.032822+0000", 2024-09-18T00:18:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8258e-05 2024-09-18T00:18:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.032853+0000", 2024-09-18T00:18:17.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0459000000000001e-05 2024-09-18T00:18:17.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.032873+0000", 2024-09-18T00:18:17.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9731000000000001e-05 2024-09-18T00:18:17.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.033012+0000", 2024-09-18T00:18:17.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013900499999999999 2024-09-18T00:18:17.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.033055+0000", 2024-09-18T00:18:17.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3628999999999999e-05 2024-09-18T00:18:17.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.034149+0000", 2024-09-18T00:18:17.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0010941589999999999 2024-09-18T00:18:17.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.034284+0000", 2024-09-18T00:18:17.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013453200000000001 2024-09-18T00:18:17.412 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.413 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.413 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.413 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:372 3.4 e29/23 3:20d2ba28:::rbd_header.112ae20746f5:head v 29'3, mlcod=26'1)", 2024-09-18T00:18:17.413 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.458327+0000", 2024-09-18T00:18:17.413 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.9298892500000004, 2024-09-18T00:18:17.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018742019999999999, 2024-09-18T00:18:17.413 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.413 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.413 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.413 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.458327+0000", 2024-09-18T00:18:17.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.458327+0000", 2024-09-18T00:18:17.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.458328+0000", 2024-09-18T00:18:17.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4600000000000003e-07 2024-09-18T00:18:17.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.458342+0000", 2024-09-18T00:18:17.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3977e-05 2024-09-18T00:18:17.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.458382+0000", 2024-09-18T00:18:17.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0562999999999998e-05 2024-09-18T00:18:17.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.458405+0000", 2024-09-18T00:18:17.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2359999999999999e-05 2024-09-18T00:18:17.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.458576+0000", 2024-09-18T00:18:17.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000170898 2024-09-18T00:18:17.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.458635+0000", 2024-09-18T00:18:17.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9150000000000001e-05 2024-09-18T00:18:17.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460110+0000", 2024-09-18T00:18:17.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0014752190000000001 2024-09-18T00:18:17.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460201+0000", 2024-09-18T00:18:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1088999999999995e-05 2024-09-18T00:18:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:388 3.4 e29/23 3:20d2ba28:::rbd_header.112ae20746f5:head v 29'6, mlcod=29'4)", 2024-09-18T00:18:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.489199+0000", 2024-09-18T00:18:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.8990170690000001, 2024-09-18T00:18:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001707444, 2024-09-18T00:18:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.489199+0000", 2024-09-18T00:18:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.489199+0000", 2024-09-18T00:18:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.489201+0000", 2024-09-18T00:18:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.694e-06 2024-09-18T00:18:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.489242+0000", 2024-09-18T00:18:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1009e-05 2024-09-18T00:18:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.489267+0000", 2024-09-18T00:18:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5636999999999999e-05 2024-09-18T00:18:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.489292+0000", 2024-09-18T00:18:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4576e-05 2024-09-18T00:18:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.489422+0000", 2024-09-18T00:18:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013037199999999999 2024-09-18T00:18:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.489480+0000", 2024-09-18T00:18:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7666e-05 2024-09-18T00:18:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.490816+0000", 2024-09-18T00:18:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001336573 2024-09-18T00:18:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.490906+0000", 2024-09-18T00:18:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9917e-05 2024-09-18T00:18:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:397 3.4 e29/23 3:20d2ba28:::rbd_header.112ae20746f5:head v 29'7, mlcod=29'5)", 2024-09-18T00:18:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.500834+0000", 2024-09-18T00:18:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.8873817949999996, 2024-09-18T00:18:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001456859, 2024-09-18T00:18:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.500834+0000", 2024-09-18T00:18:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.500834+0000", 2024-09-18T00:18:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.500836+0000", 2024-09-18T00:18:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5799999999999999e-06 2024-09-18T00:18:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.500848+0000", 2024-09-18T00:18:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-05 2024-09-18T00:18:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.500873+0000", 2024-09-18T00:18:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5786e-05 2024-09-18T00:18:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.500892+0000", 2024-09-18T00:18:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8817e-05 2024-09-18T00:18:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.500997+0000", 2024-09-18T00:18:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010463600000000001 2024-09-18T00:18:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.501046+0000", 2024-09-18T00:18:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8726000000000001e-05 2024-09-18T00:18:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.502218+0000", 2024-09-18T00:18:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011721710000000001 2024-09-18T00:18:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.502291+0000", 2024-09-18T00:18:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3243000000000003e-05 2024-09-18T00:18:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:459 3.9 e29/23 3:9496b897:::rbd_data.112ae20746f5.0000000000000161:head v 29'5, mlcod=28'3)", 2024-09-18T00:18:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.652427+0000", 2024-09-18T00:18:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.7357887630000004, 2024-09-18T00:18:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021551880000000002, 2024-09-18T00:18:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.652427+0000", 2024-09-18T00:18:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.652427+0000", 2024-09-18T00:18:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.652428+0000", 2024-09-18T00:18:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6399999999999998e-07 2024-09-18T00:18:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.652624+0000", 2024-09-18T00:18:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000196906 2024-09-18T00:18:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.652640+0000", 2024-09-18T00:18:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5406000000000001e-05 2024-09-18T00:18:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.652648+0000", 2024-09-18T00:18:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4419999999999998e-06 2024-09-18T00:18:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.652731+0000", 2024-09-18T00:18:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2610999999999999e-05 2024-09-18T00:18:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.652768+0000", 2024-09-18T00:18:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7347e-05 2024-09-18T00:18:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.654519+0000", 2024-09-18T00:18:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017512680000000001 2024-09-18T00:18:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.654582+0000", 2024-09-18T00:18:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2943999999999999e-05 2024-09-18T00:18:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:509 3.13 e29/27 3:c9e2d71a:::rbd_data.112ae20746f5.0000000000000090:head v 29'1, mlcod=0'0)", 2024-09-18T00:18:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.752123+0000", 2024-09-18T00:18:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.6360933739999997, 2024-09-18T00:18:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001768539, 2024-09-18T00:18:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.752123+0000", 2024-09-18T00:18:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.752123+0000", 2024-09-18T00:18:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.752124+0000", 2024-09-18T00:18:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.299e-06 2024-09-18T00:18:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.752145+0000", 2024-09-18T00:18:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0757000000000001e-05 2024-09-18T00:18:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.752160+0000", 2024-09-18T00:18:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4932e-05 2024-09-18T00:18:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.752169+0000", 2024-09-18T00:18:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1430000000000004e-06 2024-09-18T00:18:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.752234+0000", 2024-09-18T00:18:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5580000000000006e-05 2024-09-18T00:18:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.752253+0000", 2024-09-18T00:18:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.872e-05 2024-09-18T00:18:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.753827+0000", 2024-09-18T00:18:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001573994 2024-09-18T00:18:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.753891+0000", 2024-09-18T00:18:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4114000000000003e-05 2024-09-18T00:18:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:531 3.4 e29/23 3:20d2ba28:::rbd_header.112ae20746f5:head v 29'17, mlcod=29'15)", 2024-09-18T00:18:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.841334+0000", 2024-09-18T00:18:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.5468821290000001, 2024-09-18T00:18:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001393919, 2024-09-18T00:18:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.841334+0000", 2024-09-18T00:18:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.841334+0000", 2024-09-18T00:18:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.841336+0000", 2024-09-18T00:18:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2330000000000001e-06 2024-09-18T00:18:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.841351+0000", 2024-09-18T00:18:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4643e-05 2024-09-18T00:18:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.841385+0000", 2024-09-18T00:18:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4263e-05 2024-09-18T00:18:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.841401+0000", 2024-09-18T00:18:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5937e-05 2024-09-18T00:18:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.841547+0000", 2024-09-18T00:18:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145909 2024-09-18T00:18:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.841628+0000", 2024-09-18T00:18:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1505e-05 2024-09-18T00:18:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.842657+0000", 2024-09-18T00:18:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001028759 2024-09-18T00:18:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.842728+0000", 2024-09-18T00:18:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0669999999999999e-05 2024-09-18T00:18:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:542 3.4 e29/23 3:20d2ba28:::rbd_header.112ae20746f5:head v 29'18, mlcod=29'16)", 2024-09-18T00:18:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.860010+0000", 2024-09-18T00:18:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.5282056339999999, 2024-09-18T00:18:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0014388420000000001, 2024-09-18T00:18:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.860010+0000", 2024-09-18T00:18:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.860010+0000", 2024-09-18T00:18:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.860013+0000", 2024-09-18T00:18:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0070000000000002e-06 2024-09-18T00:18:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.860031+0000", 2024-09-18T00:18:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7893e-05 2024-09-18T00:18:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.860074+0000", 2024-09-18T00:18:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2985999999999998e-05 2024-09-18T00:18:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.860104+0000", 2024-09-18T00:18:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9373000000000002e-05 2024-09-18T00:18:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.860282+0000", 2024-09-18T00:18:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017835600000000001 2024-09-18T00:18:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.860345+0000", 2024-09-18T00:18:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3418999999999997e-05 2024-09-18T00:18:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.861372+0000", 2024-09-18T00:18:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0010261960000000001 2024-09-18T00:18:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.861449+0000", 2024-09-18T00:18:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7612000000000002e-05 2024-09-18T00:18:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:568 3.13 e29/27 3:c87503f4:::rbd_data.112ae20746f5.00000000000001c5:head v 29'2, mlcod=0'0)", 2024-09-18T00:18:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.912514+0000", 2024-09-18T00:18:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.4757024120000004, 2024-09-18T00:18:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026220229999999998, 2024-09-18T00:18:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.912514+0000", 2024-09-18T00:18:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.912514+0000", 2024-09-18T00:18:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.912516+0000", 2024-09-18T00:18:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.08e-06 2024-09-18T00:18:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.912670+0000", 2024-09-18T00:18:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000154129 2024-09-18T00:18:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.912698+0000", 2024-09-18T00:18:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8549000000000001e-05 2024-09-18T00:18:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.912711+0000", 2024-09-18T00:18:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3077e-05 2024-09-18T00:18:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.912828+0000", 2024-09-18T00:18:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000116966 2024-09-18T00:18:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.912870+0000", 2024-09-18T00:18:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1896e-05 2024-09-18T00:18:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.915044+0000", 2024-09-18T00:18:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002173866 2024-09-18T00:18:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.915136+0000", 2024-09-18T00:18:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1459999999999995e-05 2024-09-18T00:18:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:612 3.4 e29/23 3:20d2ba28:::rbd_header.112ae20746f5:head v 29'23, mlcod=29'21)", 2024-09-18T00:18:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.045266+0000", 2024-09-18T00:18:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.3429503269999996, 2024-09-18T00:18:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0013477999999999999, 2024-09-18T00:18:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.045266+0000", 2024-09-18T00:18:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.045266+0000", 2024-09-18T00:18:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.045268+0000", 2024-09-18T00:18:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1059999999999998e-06 2024-09-18T00:18:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.045281+0000", 2024-09-18T00:18:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3517e-05 2024-09-18T00:18:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.045312+0000", 2024-09-18T00:18:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0666000000000001e-05 2024-09-18T00:18:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.045327+0000", 2024-09-18T00:18:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5296999999999999e-05 2024-09-18T00:18:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.045469+0000", 2024-09-18T00:18:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141612 2024-09-18T00:18:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.045517+0000", 2024-09-18T00:18:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8405999999999997e-05 2024-09-18T00:18:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.046533+0000", 2024-09-18T00:18:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0010153619999999999 2024-09-18T00:18:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.046613+0000", 2024-09-18T00:18:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0834000000000006e-05 2024-09-18T00:18:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:692 3.0 e29/23 3:03fab4ad:::rbd_data.112ae20746f5.0000000000000057:head v 29'1, mlcod=0'0)", 2024-09-18T00:18:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.149379057, 2024-09-18T00:18:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074841819999999998, 2024-09-18T00:18:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238838+0000", 2024-09-18T00:18:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.187e-06 2024-09-18T00:18:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238874+0000", 2024-09-18T00:18:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5380999999999999e-05 2024-09-18T00:18:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238893+0000", 2024-09-18T00:18:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999999e-05 2024-09-18T00:18:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238906+0000", 2024-09-18T00:18:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3011e-05 2024-09-18T00:18:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238982+0000", 2024-09-18T00:18:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6018000000000003e-05 2024-09-18T00:18:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.239006+0000", 2024-09-18T00:18:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3683e-05 2024-09-18T00:18:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246227+0000", 2024-09-18T00:18:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072210989999999999 2024-09-18T00:18:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246321+0000", 2024-09-18T00:18:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4263000000000002e-05 2024-09-18T00:18:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:704 3.4 e29/23 3:20d2ba28:::rbd_header.112ae20746f5:head v 29'30, mlcod=29'28)", 2024-09-18T00:18:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.294789+0000", 2024-09-18T00:18:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.0934274100000003, 2024-09-18T00:18:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0015122650000000001, 2024-09-18T00:18:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.294789+0000", 2024-09-18T00:18:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.294789+0000", 2024-09-18T00:18:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.294791+0000", 2024-09-18T00:18:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3099999999999999e-06 2024-09-18T00:18:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.294809+0000", 2024-09-18T00:18:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8093000000000001e-05 2024-09-18T00:18:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.294848+0000", 2024-09-18T00:18:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9390000000000001e-05 2024-09-18T00:18:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.294870+0000", 2024-09-18T00:18:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1667000000000001e-05 2024-09-18T00:18:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.295024+0000", 2024-09-18T00:18:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000153845 2024-09-18T00:18:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.295079+0000", 2024-09-18T00:18:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5402999999999998e-05 2024-09-18T00:18:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.296224+0000", 2024-09-18T00:18:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011451460000000001 2024-09-18T00:18:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.296301+0000", 2024-09-18T00:18:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6410999999999997e-05 2024-09-18T00:18:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:746 3.0 e29/23 3:03157719:::rbd_data.112ae20746f5.0000000000000004:head v 29'2, mlcod=0'0)", 2024-09-18T00:18:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.392074+0000", 2024-09-18T00:18:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.9961415840000001, 2024-09-18T00:18:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033355659999999999, 2024-09-18T00:18:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392074+0000", 2024-09-18T00:18:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392074+0000", 2024-09-18T00:18:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392075+0000", 2024-09-18T00:18:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:18:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392101+0000", 2024-09-18T00:18:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6001999999999999e-05 2024-09-18T00:18:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392125+0000", 2024-09-18T00:18:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3771e-05 2024-09-18T00:18:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392138+0000", 2024-09-18T00:18:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3675e-05 2024-09-18T00:18:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392216+0000", 2024-09-18T00:18:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8054999999999997e-05 2024-09-18T00:18:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392245+0000", 2024-09-18T00:18:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8501999999999999e-05 2024-09-18T00:18:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395315+0000", 2024-09-18T00:18:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003069975 2024-09-18T00:18:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395410+0000", 2024-09-18T00:18:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5068000000000005e-05 2024-09-18T00:18:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:839 3.0 e30/23 3:0095abde:::rbd_data.112ae20746f5.0000000000000116:head v 30'4, mlcod=29'2)", 2024-09-18T00:18:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.774971+0000", 2024-09-18T00:18:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.613245246, 2024-09-18T00:18:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00246956, 2024-09-18T00:18:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774971+0000", 2024-09-18T00:18:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774971+0000", 2024-09-18T00:18:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774973+0000", 2024-09-18T00:18:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-18T00:18:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.775056+0000", 2024-09-18T00:18:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3228999999999999e-05 2024-09-18T00:18:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.775086+0000", 2024-09-18T00:18:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9836999999999999e-05 2024-09-18T00:18:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.775100+0000", 2024-09-18T00:18:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.456e-05 2024-09-18T00:18:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.775188+0000", 2024-09-18T00:18:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7698e-05 2024-09-18T00:18:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.775220+0000", 2024-09-18T00:18:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1495000000000003e-05 2024-09-18T00:18:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.777391+0000", 2024-09-18T00:18:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002170983 2024-09-18T00:18:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.777440+0000", 2024-09-18T00:18:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9804e-05 2024-09-18T00:18:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:870 3.0 e30/23 3:05335421:::rbd_data.112ae20746f5.0000000000000014:head v 30'5, mlcod=30'3)", 2024-09-18T00:18:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.955786+0000", 2024-09-18T00:18:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.432430171, 2024-09-18T00:18:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0044642019999999996, 2024-09-18T00:18:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955786+0000", 2024-09-18T00:18:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955786+0000", 2024-09-18T00:18:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955787+0000", 2024-09-18T00:18:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.271e-06 2024-09-18T00:18:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955824+0000", 2024-09-18T00:18:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6668999999999997e-05 2024-09-18T00:18:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955840+0000", 2024-09-18T00:18:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5977999999999998e-05 2024-09-18T00:18:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955847+0000", 2024-09-18T00:18:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1760000000000004e-06 2024-09-18T00:18:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955929+0000", 2024-09-18T00:18:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1568000000000002e-05 2024-09-18T00:18:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955961+0000", 2024-09-18T00:18:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2873000000000002e-05 2024-09-18T00:18:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960199+0000", 2024-09-18T00:18:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042373990000000002 2024-09-18T00:18:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960250+0000", 2024-09-18T00:18:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1267999999999998e-05 2024-09-18T00:18:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:1563 3.4 e32/23 3:20d2ba28:::rbd_header.112ae20746f5:head v 32'87, mlcod=31'85)", 2024-09-18T00:18:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:15.426077+0000", 2024-09-18T00:18:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.9621385579999999, 2024-09-18T00:18:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021311279999999999, 2024-09-18T00:18:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.426077+0000", 2024-09-18T00:18:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.426077+0000", 2024-09-18T00:18:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.426078+0000", 2024-09-18T00:18:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.32e-07 2024-09-18T00:18:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.426085+0000", 2024-09-18T00:18:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.469e-06 2024-09-18T00:18:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.426109+0000", 2024-09-18T00:18:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.387e-05 2024-09-18T00:18:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.426120+0000", 2024-09-18T00:18:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1066999999999999e-05 2024-09-18T00:18:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.426235+0000", 2024-09-18T00:18:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000114502 2024-09-18T00:18:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.426269+0000", 2024-09-18T00:18:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4374e-05 2024-09-18T00:18:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.428164+0000", 2024-09-18T00:18:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018946309999999999 2024-09-18T00:18:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.428209+0000", 2024-09-18T00:18:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4783000000000001e-05 2024-09-18T00:18:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:17.479 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:17.479 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:17.479 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:18:17.480 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:17.479+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:17.481 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:18:17.580 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:17.579+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:17.646 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:17.646 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:17.661 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:18:17.677 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:17.677 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:17.695 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:18:17.695 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:18:17.695 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:17.691+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:17.713 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:17.713 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:17.730 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:17.730 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:17.751 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:17.751 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:17.772 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:18:17.772 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:18:17.782 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:17.779+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:17.793 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:17.793 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:17.808 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:18:17.817 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:17.817 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:17.882 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:17.879+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:17.963 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:17.963 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:17.979 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:18:17.982 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:17.979+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:18.084 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:18.079+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:18.115 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:18.115 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:18.115 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-18T00:18:18.129 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:18:18.185 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:18.183+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:18.191 INFO:teuthology.orchestra.run.smithi097.stdout:557 trunc from 0xea87965 to 0xd2adb00 2024-09-18T00:18:18.224 INFO:teuthology.orchestra.run.smithi097.stdout:558 write 0x9c6b8b3 thru 0x9c70208 (0x4956 bytes) 2024-09-18T00:18:18.235 INFO:teuthology.orchestra.run.smithi097.stdout:560 write 0xcbeed18 thru 0xcbf98f3 (0xabdc bytes) 2024-09-18T00:18:18.247 INFO:teuthology.orchestra.run.smithi097.stdout:561 read 0x332f6a2 thru 0x333910e (0x9a6d bytes) 2024-09-18T00:18:18.247 INFO:teuthology.orchestra.run.smithi097.stdout:562 write 0x7445fa1 thru 0x74495d0 (0x3630 bytes) 2024-09-18T00:18:18.256 INFO:teuthology.orchestra.run.smithi097.stdout:563 write 0xa16a75a thru 0xa16e71b (0x3fc2 bytes) 2024-09-18T00:18:18.263 INFO:teuthology.orchestra.run.smithi097.stdout:564 read 0xb557f3f thru 0xb560273 (0x8335 bytes) 2024-09-18T00:18:18.263 INFO:teuthology.orchestra.run.smithi097.stdout:565 read 0x9b05a5d thru 0x9b12ffe (0xd5a2 bytes) 2024-09-18T00:18:18.263 INFO:teuthology.orchestra.run.smithi097.stdout:567 read 0x5c64cf thru 0x5c8f9a (0x2acc bytes) 2024-09-18T00:18:18.264 INFO:teuthology.orchestra.run.smithi097.stdout:570 writesame 0x93b6f74 thru 0x93baa66 data_size 0x1(0x3af3 bytes) 2024-09-18T00:18:18.285 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:18.279+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:18.296 INFO:teuthology.orchestra.run.smithi097.stdout:571 writesame 0xb4f85 thru 0xbcfa0 data_size 0x3(0x801c bytes) 2024-09-18T00:18:18.312 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:18.313 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:18.327 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:18:18.331 INFO:teuthology.orchestra.run.smithi097.stdout:573 write 0xb25ebaa thru 0xb2637dd (0x4c34 bytes) 2024-09-18T00:18:18.339 INFO:teuthology.orchestra.run.smithi097.stdout:574 writesame 0xaa9f4c5 thru 0xaaadf0e data_size 0x1(0xea4a bytes) 2024-09-18T00:18:18.386 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:18.383+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:18.447 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:18:18.468 INFO:teuthology.orchestra.run.smithi097.stdout:575 write 0x1117001 thru 0x112352d (0xc52d bytes) 2024-09-18T00:18:18.477 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:18.478 INFO:teuthology.orchestra.run.smithi097.stdout:578 read 0xbfbf217 thru 0xbfc464a (0x5434 bytes) 2024-09-18T00:18:18.487 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:18.483+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:18.496 INFO:teuthology.orchestra.run.smithi097.stdout:579 trunc from 0xd2adb00 to 0x21b2c88 2024-09-18T00:18:18.498 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:18.498 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:18.516 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:18:18.588 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:18.583+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:18.637 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:18.638 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:18.651 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:18:18.689 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:18.687+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:18.790 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:18.791+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:18.815 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:18.816 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:18.829 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:18:18.891 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:18.887+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:18.952 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:18.952 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:18.966 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:18:18.992 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:18.987+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:19.092 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:19.091+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:19.135 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:19.135 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:19.149 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:18:19.194 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:19.191+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:19.266 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:19.267 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:19.280 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:18:19.294 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:19.291+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:19.395 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:19.391+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:19.447 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:19.447 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:19.462 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:18:19.463 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is set 2024-09-18T00:18:19.499 INFO:teuthology.orchestra.run.smithi097.stdout:580 punch from 0x2131796 to 0x213d4c6, (0xbd30 bytes) 2024-09-18T00:18:19.499 INFO:teuthology.orchestra.run.smithi097.stdout:582 read 0xadb05b thru 0xae328d (0x8233 bytes) 2024-09-18T00:18:19.500 INFO:teuthology.orchestra.run.smithi097.stdout:583 read 0x1febbc1 thru 0x1ff66a6 (0xaae6 bytes) 2024-09-18T00:18:19.500 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:19.495+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:19.526 INFO:teuthology.orchestra.run.smithi097.stdout:585 write 0xc44a894 thru 0xc451c11 (0x737e bytes) 2024-09-18T00:18:19.532 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:19.532 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:19.532 INFO:teuthology.orchestra.run.smithi097.stdout:590 writesame 0x8e773ce thru 0x8e7d8ad data_size 0x30(0x64e0 bytes) 2024-09-18T00:18:19.537 INFO:teuthology.orchestra.run.smithi097.stdout:591 write 0x75d874a thru 0x75dda03 (0x52ba bytes) 2024-09-18T00:18:19.542 INFO:teuthology.orchestra.run.smithi097.stdout:592 read 0xab7ad5b thru 0xab88b16 (0xddbc bytes) 2024-09-18T00:18:19.542 INFO:teuthology.orchestra.run.smithi097.stdout:593 read 0xb4a9b14 thru 0xb4ab1c4 (0x16b1 bytes) 2024-09-18T00:18:19.543 INFO:teuthology.orchestra.run.smithi097.stdout:594 write 0x77f441e thru 0x7802225 (0xde08 bytes) 2024-09-18T00:18:19.551 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:19.551 INFO:teuthology.orchestra.run.smithi097.stdout:596 writesame 0x6cb3ea4 thru 0x6cbf6c3 data_size 0x30(0xb820 bytes) 2024-09-18T00:18:19.557 INFO:teuthology.orchestra.run.smithi097.stdout:597 read 0x61b398d thru 0x61bbe3d (0x84b1 bytes) 2024-09-18T00:18:19.557 INFO:teuthology.orchestra.run.smithi097.stdout:598 write 0x5da0365 thru 0x5dad4bf (0xd15b bytes) 2024-09-18T00:18:19.563 INFO:teuthology.orchestra.run.smithi097.stdout:599 punch from 0x52d172a to 0x52dbc6b, (0xa541 bytes) 2024-09-18T00:18:19.563 INFO:teuthology.orchestra.run.smithi097.stdout:600 writesame 0x8125566 thru 0x8129a51 data_size 0x16(0x44ec bytes) 2024-09-18T00:18:19.570 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:19.570 INFO:teuthology.orchestra.run.smithi097.stdout:602 read 0x1a02728 thru 0x1a03ccb (0x15a4 bytes) 2024-09-18T00:18:19.570 INFO:teuthology.orchestra.run.smithi097.stdout:603 write 0x8c89750 thru 0x8c98dde (0xf68f bytes) 2024-09-18T00:18:19.583 INFO:teuthology.orchestra.run.smithi097.stdout:604 punch from 0x84d698c to 0x84e57b1, (0xee25 bytes) 2024-09-18T00:18:19.583 INFO:teuthology.orchestra.run.smithi097.stdout:605 write 0x1732398 thru 0x1737770 (0x53d9 bytes) 2024-09-18T00:18:19.592 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:19.592 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:19.592 INFO:teuthology.orchestra.run.smithi097.stdout:609 punch from 0x8f5472e to 0x8f63621, (0xeef3 bytes) 2024-09-18T00:18:19.592 INFO:teuthology.orchestra.run.smithi097.stdout:610 read 0xbc60600 thru 0xbc69e04 (0x9805 bytes) 2024-09-18T00:18:19.592 INFO:teuthology.orchestra.run.smithi097.stdout:611 punch from 0x51a8706 to 0x51b6ea6, (0xe7a0 bytes) 2024-09-18T00:18:19.592 INFO:teuthology.orchestra.run.smithi097.stdout:612 punch from 0x21d370f to 0x21dfcd8, (0xc5c9 bytes) 2024-09-18T00:18:19.593 INFO:teuthology.orchestra.run.smithi097.stdout:613 read 0x52fab11 thru 0x5302bac (0x809c bytes) 2024-09-18T00:18:19.595 INFO:teuthology.orchestra.run.smithi097.stdout:614 trunc from 0xc451c12 to 0xd8e331e 2024-09-18T00:18:19.597 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:19.595+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:19.602 INFO:teuthology.orchestra.run.smithi097.stdout:615 read 0x15b4790 thru 0x15c363d (0xeeae bytes) 2024-09-18T00:18:19.602 INFO:teuthology.orchestra.run.smithi097.stdout:616 read 0x2690169 thru 0x26920c0 (0x1f58 bytes) 2024-09-18T00:18:19.602 INFO:teuthology.orchestra.run.smithi097.stdout:617 read 0x77bd114 thru 0x77ca520 (0xd40d bytes) 2024-09-18T00:18:19.625 INFO:teuthology.orchestra.run.smithi097.stdout:618 trunc from 0xd8e331e to 0x232da14 2024-09-18T00:18:19.644 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:19.645 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:19.654 INFO:teuthology.orchestra.run.smithi097.stdout:619 write 0x1ca0079 thru 0x1ca6a16 (0x699e bytes) 2024-09-18T00:18:19.661 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:18:19.662 INFO:teuthology.orchestra.run.smithi097.stdout:620 read 0x230af5c thru 0x230d585 (0x262a bytes) 2024-09-18T00:18:19.662 INFO:teuthology.orchestra.run.smithi097.stdout:623 punch from 0x1585eb1 to 0x1589df4, (0x3f43 bytes) 2024-09-18T00:18:19.673 INFO:teuthology.orchestra.run.smithi097.stdout:624 write 0x30d3b0d thru 0x30d9593 (0x5a87 bytes) 2024-09-18T00:18:19.681 INFO:teuthology.orchestra.run.smithi097.stdout:625 punch from 0x4a5402 to 0x4abc63, (0x6861 bytes) 2024-09-18T00:18:19.681 INFO:teuthology.orchestra.run.smithi097.stdout:626 read 0x12fd95d thru 0x130b051 (0xd6f5 bytes) 2024-09-18T00:18:19.682 INFO:teuthology.orchestra.run.smithi097.stdout:627 punch from 0x1220033 to 0x1222b14, (0x2ae1 bytes) 2024-09-18T00:18:19.682 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:19.682 INFO:teuthology.orchestra.run.smithi097.stdout:629 punch from 0xed2994 to 0xee0527, (0xdb93 bytes) 2024-09-18T00:18:19.698 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:19.695+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:19.717 INFO:teuthology.orchestra.run.smithi097.stdout:631 writesame 0xd72fa86 thru 0xd73f336 data_size 0x6b(0xf8b1 bytes) 2024-09-18T00:18:19.718 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:19.718 INFO:teuthology.orchestra.run.smithi097.stdout:633 read 0x98917d5 thru 0x9895c95 (0x44c1 bytes) 2024-09-18T00:18:19.724 INFO:teuthology.orchestra.run.smithi097.stdout:634 trunc from 0xd73f337 to 0xaa946b0 2024-09-18T00:18:19.733 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:19.754 INFO:teuthology.orchestra.run.smithi097.stdout:637 trunc from 0xaa946b0 to 0x1cc3a3e 2024-09-18T00:18:19.791 INFO:teuthology.orchestra.run.smithi097.stdout:638 punch from 0x665021 to 0x6692cf, (0x42ae bytes) 2024-09-18T00:18:19.792 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:19.792 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:19.798 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:19.795+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:19.806 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:18:19.823 INFO:teuthology.orchestra.run.smithi097.stdout:639 trunc from 0x1cc3a3e to 0xea6a76e 2024-09-18T00:18:19.838 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:18:19.841 INFO:teuthology.orchestra.run.smithi097.stdout:640 trunc from 0xea6a76e to 0xac77c2a 2024-09-18T00:18:19.849 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:19.849 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:19.854 INFO:teuthology.orchestra.run.smithi097.stdout:643 trunc from 0xac77c2a to 0xe34e0c6 2024-09-18T00:18:19.862 INFO:teuthology.orchestra.run.smithi097.stdout:644 punch from 0xae9b2ba to 0xaea8733, (0xd479 bytes) 2024-09-18T00:18:19.862 INFO:teuthology.orchestra.run.smithi097.stdout:647 write 0x72185c thru 0x728801 (0x6fa6 bytes) 2024-09-18T00:18:19.869 INFO:teuthology.orchestra.run.smithi097.stdout:648 write 0x345453f thru 0x345ab6f (0x6631 bytes) 2024-09-18T00:18:19.875 INFO:teuthology.orchestra.run.smithi097.stdout:649 punch from 0x5811cca to 0x5816d6a, (0x50a0 bytes) 2024-09-18T00:18:19.875 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:19.877 INFO:teuthology.orchestra.run.smithi097.stdout:652 trunc from 0xe34e0c6 to 0xcff319a 2024-09-18T00:18:19.893 INFO:teuthology.orchestra.run.smithi097.stdout:653 trunc from 0xcff319a to 0x7b6773e 2024-09-18T00:18:19.899 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:19.895+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:19.902 INFO:teuthology.orchestra.run.smithi097.stdout:655 punch from 0x7b618a6 to 0x7b6773e, (0x5e98 bytes) 2024-09-18T00:18:19.902 INFO:teuthology.orchestra.run.smithi097.stdout:656 writesame 0xb16bc7 thru 0xb20b02 data_size 0x56(0x9f3c bytes) 2024-09-18T00:18:19.924 INFO:teuthology.orchestra.run.smithi097.stdout:657 trunc from 0x7b6773e to 0xeaede57 2024-09-18T00:18:19.933 INFO:teuthology.orchestra.run.smithi097.stdout:658 punch from 0x33f07b0 to 0x33f6b3d, (0x638d bytes) 2024-09-18T00:18:19.933 INFO:teuthology.orchestra.run.smithi097.stdout:659 write 0xa9ac497 thru 0xa9ae7c7 (0x2331 bytes) 2024-09-18T00:18:19.937 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:19.937 INFO:teuthology.orchestra.run.smithi097.stdout:661 punch from 0x8defb32 to 0x8defd20, (0x1ee bytes) 2024-09-18T00:18:19.937 INFO:teuthology.orchestra.run.smithi097.stdout:662 punch from 0x79be211 to 0x79cbf7c, (0xdd6b bytes) 2024-09-18T00:18:19.941 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:19.941 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:19.948 INFO:teuthology.orchestra.run.smithi097.stdout:663 trunc from 0xeaede57 to 0x8f97eea 2024-09-18T00:18:19.956 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:18:19.962 INFO:teuthology.orchestra.run.smithi097.stdout:664 write 0x143337b thru 0x1435a44 (0x26ca bytes) 2024-09-18T00:18:19.976 INFO:teuthology.orchestra.run.smithi097.stdout:665 writesame 0x94ecd08 thru 0x94f94b4 data_size 0xb(0xc7ad bytes) 2024-09-18T00:18:19.976 INFO:teuthology.orchestra.run.smithi097.stdout:666 writesame 0x393569 thru 0x39af0d data_size 0x13(0x79a5 bytes) 2024-09-18T00:18:19.984 INFO:teuthology.orchestra.run.smithi097.stdout:668 write 0xf8f662 thru 0xf97618 (0x7fb7 bytes) 2024-09-18T00:18:19.991 INFO:teuthology.orchestra.run.smithi097.stdout:670 writesame 0x67c9d28 thru 0x67ca41b data_size 0xa(0x6f4 bytes) 2024-09-18T00:18:19.997 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:19.997 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:19.997 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:19.998 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:20.002 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:19.999+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:20.003 INFO:teuthology.orchestra.run.smithi097.stdout:674 trunc from 0x94f94b5 to 0x4dd36e5 2024-09-18T00:18:20.015 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:18:20.016 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:18:20.031 INFO:teuthology.orchestra.run.smithi097.stdout:675 writesame 0x87dc2ae thru 0x87e836c data_size 0x35(0xc0bf bytes) 2024-09-18T00:18:20.031 INFO:teuthology.orchestra.run.smithi097.stdout:676 write 0x4ae2e63 thru 0x4af19c1 (0xeb5f bytes) 2024-09-18T00:18:20.032 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:20.033 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:20.044 INFO:teuthology.orchestra.run.smithi097.stdout:678 read 0x2b3655e thru 0x2b37e7e (0x1921 bytes) 2024-09-18T00:18:20.051 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:20.051 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:20.057 INFO:teuthology.orchestra.run.smithi097.stdout:679 write 0x9a68833 thru 0x9a76d26 (0xe4f4 bytes) 2024-09-18T00:18:20.068 INFO:teuthology.orchestra.run.smithi097.stdout:680 punch from 0xeb175a to 0xebeb8a, (0xd430 bytes) 2024-09-18T00:18:20.068 INFO:teuthology.orchestra.run.smithi097.stdout:681 read 0x397cd34 thru 0x398ac5b (0xdf28 bytes) 2024-09-18T00:18:20.068 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:20.068 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:20.068 INFO:teuthology.orchestra.run.smithi097.stdout:684 read 0x879de13 thru 0x87abd4a (0xdf38 bytes) 2024-09-18T00:18:20.068 INFO:teuthology.orchestra.run.smithi097.stdout:685 write 0x81c0ec8 thru 0x81c50fe (0x4237 bytes) 2024-09-18T00:18:20.074 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:20.074 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:20.076 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:20.076 INFO:teuthology.orchestra.run.smithi097.stdout:688 punch from 0x2368e0b to 0x2372731, (0x9926 bytes) 2024-09-18T00:18:20.084 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:20.085 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:20.085 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:20.085 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:20.095 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:18:20.095 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:18:20.097 INFO:teuthology.orchestra.run.smithi097.stdout:689 write 0xe810056 thru 0xe818ee6 (0x8e91 bytes) 2024-09-18T00:18:20.099 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:18:20.117 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:20.118 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:20.118 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:20.115+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:20.121 INFO:teuthology.orchestra.run.smithi097.stdout:690 trunc from 0xe818ee7 to 0x38a8bfe 2024-09-18T00:18:20.141 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:20.141 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:20.147 INFO:teuthology.orchestra.run.smithi097.stdout:691 read 0x319688 thru 0x31afae (0x1927 bytes) 2024-09-18T00:18:20.147 INFO:teuthology.orchestra.run.smithi097.stdout:692 punch from 0x9b9848 to 0x9bc80f, (0x2fc7 bytes) 2024-09-18T00:18:20.173 INFO:teuthology.orchestra.run.smithi097.stdout:693 write 0xbbf4160 thru 0xbbfd777 (0x9618 bytes) 2024-09-18T00:18:20.181 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:20.197 INFO:teuthology.orchestra.run.smithi097.stdout:696 write 0xda4504a thru 0xda517c9 (0xc780 bytes) 2024-09-18T00:18:20.202 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:20.199+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:20.208 INFO:teuthology.orchestra.run.smithi097.stdout:697 write 0x12d9e2d thru 0x12df51b (0x56ef bytes) 2024-09-18T00:18:20.232 INFO:teuthology.orchestra.run.smithi097.stdout:699 trunc from 0xda517ca to 0x88e89cb 2024-09-18T00:18:20.251 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:20.251 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:20.256 INFO:teuthology.orchestra.run.smithi097.stdout:700 read 0x80796ab thru 0x8085ebc (0xc812 bytes) 2024-09-18T00:18:20.256 INFO:teuthology.orchestra.run.smithi097.stdout:701 writesame 0x539542b thru 0x53a3681 data_size 0x1(0xe257 bytes) 2024-09-18T00:18:20.265 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:18:20.309 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:20.307+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:20.340 INFO:teuthology.orchestra.run.smithi097.stdout:702 read 0x50c4217 thru 0x50d061e (0xc408 bytes) 2024-09-18T00:18:20.340 INFO:teuthology.orchestra.run.smithi097.stdout:703 punch from 0x7cf1144 to 0x7cfd609, (0xc4c5 bytes) 2024-09-18T00:18:20.340 INFO:teuthology.orchestra.run.smithi097.stdout:704 read 0x71d2959 thru 0x71d81b3 (0x585b bytes) 2024-09-18T00:18:20.341 INFO:teuthology.orchestra.run.smithi097.stdout:705 trunc from 0x88e89cb to 0x93c5f4f 2024-09-18T00:18:20.373 INFO:teuthology.orchestra.run.smithi097.stdout:706 write 0xcc139d1 thru 0xcc17f62 (0x4592 bytes) 2024-09-18T00:18:20.379 INFO:teuthology.orchestra.run.smithi097.stdout:711 writesame 0xbe3eb03 thru 0xbe40255 data_size 0x1(0x1753 bytes) 2024-09-18T00:18:20.394 INFO:teuthology.orchestra.run.smithi097.stdout:712 write 0xaae4ba4 thru 0xaaed914 (0x8d71 bytes) 2024-09-18T00:18:20.404 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:20.403+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:20.420 INFO:teuthology.orchestra.run.smithi097.stdout:716 trunc from 0xcc17f63 to 0x9a6abc6 2024-09-18T00:18:20.441 INFO:teuthology.orchestra.run.smithi097.stdout:717 punch from 0x1ebc517 to 0x1ec4bab, (0x8694 bytes) 2024-09-18T00:18:20.442 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:20.442 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:20.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:20.442 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:20.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.442 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:36 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'22, mlcod=17'18)", 2024-09-18T00:18:20.442 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:51.563151+0000", 2024-09-18T00:18:20.442 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.877715032000001, 2024-09-18T00:18:20.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067203280000000002, 2024-09-18T00:18:20.442 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.442 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.442 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.443 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563151+0000", 2024-09-18T00:18:20.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563151+0000", 2024-09-18T00:18:20.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563153+0000", 2024-09-18T00:18:20.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.42e-06 2024-09-18T00:18:20.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563163+0000", 2024-09-18T00:18:20.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.077e-05 2024-09-18T00:18:20.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563191+0000", 2024-09-18T00:18:20.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.724e-05 2024-09-18T00:18:20.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563204+0000", 2024-09-18T00:18:20.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3266e-05 2024-09-18T00:18:20.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563319+0000", 2024-09-18T00:18:20.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115343 2024-09-18T00:18:20.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.563371+0000", 2024-09-18T00:18:20.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1922000000000002e-05 2024-09-18T00:18:20.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.569760+0000", 2024-09-18T00:18:20.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006388721 2024-09-18T00:18:20.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.569872+0000", 2024-09-18T00:18:20.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011164599999999999 2024-09-18T00:18:20.446 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.446 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.446 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.447 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:37 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'23, mlcod=17'18)", 2024-09-18T00:18:20.447 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:51.564518+0000", 2024-09-18T00:18:20.447 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.876347966000001, 2024-09-18T00:18:20.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005870994, 2024-09-18T00:18:20.447 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.447 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.447 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.447 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564518+0000", 2024-09-18T00:18:20.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564518+0000", 2024-09-18T00:18:20.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564520+0000", 2024-09-18T00:18:20.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8300000000000001e-06 2024-09-18T00:18:20.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564532+0000", 2024-09-18T00:18:20.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1754e-05 2024-09-18T00:18:20.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564560+0000", 2024-09-18T00:18:20.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8486000000000001e-05 2024-09-18T00:18:20.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564573+0000", 2024-09-18T00:18:20.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3069e-05 2024-09-18T00:18:20.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564660+0000", 2024-09-18T00:18:20.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6263999999999999e-05 2024-09-18T00:18:20.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.564695+0000", 2024-09-18T00:18:20.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5296000000000002e-05 2024-09-18T00:18:20.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.570284+0000", 2024-09-18T00:18:20.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055890030000000004 2024-09-18T00:18:20.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:51.570389+0000", 2024-09-18T00:18:20.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010529199999999999 2024-09-18T00:18:20.451 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.451 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.451 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.451 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:131 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'89, mlcod=17'85)", 2024-09-18T00:18:20.451 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.319608+0000", 2024-09-18T00:18:20.451 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.121257877000001, 2024-09-18T00:18:20.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0058413620000000001, 2024-09-18T00:18:20.451 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.451 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.452 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.452 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319608+0000", 2024-09-18T00:18:20.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319608+0000", 2024-09-18T00:18:20.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319609+0000", 2024-09-18T00:18:20.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3e-07 2024-09-18T00:18:20.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319676+0000", 2024-09-18T00:18:20.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6889000000000003e-05 2024-09-18T00:18:20.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319694+0000", 2024-09-18T00:18:20.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7703e-05 2024-09-18T00:18:20.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.319706+0000", 2024-09-18T00:18:20.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2577e-05 2024-09-18T00:18:20.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320272+0000", 2024-09-18T00:18:20.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056591599999999997 2024-09-18T00:18:20.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320306+0000", 2024-09-18T00:18:20.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4116000000000003e-05 2024-09-18T00:18:20.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325386+0000", 2024-09-18T00:18:20.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050793349999999999 2024-09-18T00:18:20.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325450+0000", 2024-09-18T00:18:20.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4196000000000007e-05 2024-09-18T00:18:20.455 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.455 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.455 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.456 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:132 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'90, mlcod=17'85)", 2024-09-18T00:18:20.456 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.320067+0000", 2024-09-18T00:18:20.456 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.120799344000002, 2024-09-18T00:18:20.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0054358590000000004, 2024-09-18T00:18:20.456 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.456 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.456 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.456 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320067+0000", 2024-09-18T00:18:20.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320067+0000", 2024-09-18T00:18:20.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320068+0000", 2024-09-18T00:18:20.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0100000000000001e-06 2024-09-18T00:18:20.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320079+0000", 2024-09-18T00:18:20.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1146e-05 2024-09-18T00:18:20.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320101+0000", 2024-09-18T00:18:20.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1994e-05 2024-09-18T00:18:20.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320110+0000", 2024-09-18T00:18:20.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3330000000000003e-06 2024-09-18T00:18:20.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321623+0000", 2024-09-18T00:18:20.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001512981 2024-09-18T00:18:20.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321656+0000", 2024-09-18T00:18:20.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2539999999999997e-05 2024-09-18T00:18:20.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325455+0000", 2024-09-18T00:18:20.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0037992880000000001 2024-09-18T00:18:20.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325503+0000", 2024-09-18T00:18:20.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7567e-05 2024-09-18T00:18:20.460 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.460 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.460 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.460 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:134 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'92, mlcod=17'85)", 2024-09-18T00:18:20.460 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.320847+0000", 2024-09-18T00:18:20.460 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.120019469999999, 2024-09-18T00:18:20.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0068502509999999999, 2024-09-18T00:18:20.460 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.460 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.460 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.461 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320847+0000", 2024-09-18T00:18:20.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320847+0000", 2024-09-18T00:18:20.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320848+0000", 2024-09-18T00:18:20.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1629999999999999e-06 2024-09-18T00:18:20.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320971+0000", 2024-09-18T00:18:20.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012338599999999999 2024-09-18T00:18:20.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.320990+0000", 2024-09-18T00:18:20.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9009999999999999e-05 2024-09-18T00:18:20.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321000+0000", 2024-09-18T00:18:20.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9399999999999997e-06 2024-09-18T00:18:20.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322866+0000", 2024-09-18T00:18:20.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001866089 2024-09-18T00:18:20.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322902+0000", 2024-09-18T00:18:20.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5312999999999999e-05 2024-09-18T00:18:20.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327608+0000", 2024-09-18T00:18:20.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047063000000000001 2024-09-18T00:18:20.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327697+0000", 2024-09-18T00:18:20.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9049999999999996e-05 2024-09-18T00:18:20.464 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.464 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.464 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.464 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:135 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'93, mlcod=17'85)", 2024-09-18T00:18:20.464 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.321689+0000", 2024-09-18T00:18:20.465 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.11917682, 2024-09-18T00:18:20.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0082046600000000008, 2024-09-18T00:18:20.465 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.465 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.465 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.465 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321689+0000", 2024-09-18T00:18:20.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321689+0000", 2024-09-18T00:18:20.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321691+0000", 2024-09-18T00:18:20.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0229999999999999e-06 2024-09-18T00:18:20.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321706+0000", 2024-09-18T00:18:20.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4513e-05 2024-09-18T00:18:20.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321727+0000", 2024-09-18T00:18:20.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0939999999999999e-05 2024-09-18T00:18:20.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.321745+0000", 2024-09-18T00:18:20.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7986999999999999e-05 2024-09-18T00:18:20.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324259+0000", 2024-09-18T00:18:20.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025143069999999999 2024-09-18T00:18:20.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324291+0000", 2024-09-18T00:18:20.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2116999999999998e-05 2024-09-18T00:18:20.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329824+0000", 2024-09-18T00:18:20.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055326300000000002 2024-09-18T00:18:20.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329894+0000", 2024-09-18T00:18:20.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0142999999999996e-05 2024-09-18T00:18:20.468 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.469 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.469 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.469 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:136 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'94, mlcod=17'85)", 2024-09-18T00:18:20.469 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.322041+0000", 2024-09-18T00:18:20.469 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.118825633, 2024-09-18T00:18:20.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0079092060000000002, 2024-09-18T00:18:20.469 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.469 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.469 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.470 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322041+0000", 2024-09-18T00:18:20.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322041+0000", 2024-09-18T00:18:20.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322042+0000", 2024-09-18T00:18:20.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.22e-06 2024-09-18T00:18:20.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322053+0000", 2024-09-18T00:18:20.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1584e-05 2024-09-18T00:18:20.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322073+0000", 2024-09-18T00:18:20.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0023000000000001e-05 2024-09-18T00:18:20.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322092+0000", 2024-09-18T00:18:20.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8063e-05 2024-09-18T00:18:20.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324848+0000", 2024-09-18T00:18:20.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027561420000000001 2024-09-18T00:18:20.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324877+0000", 2024-09-18T00:18:20.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9003e-05 2024-09-18T00:18:20.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329900+0000", 2024-09-18T00:18:20.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050233650000000001 2024-09-18T00:18:20.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329950+0000", 2024-09-18T00:18:20.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9805999999999997e-05 2024-09-18T00:18:20.473 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.473 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.473 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.473 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:137 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'95, mlcod=17'85)", 2024-09-18T00:18:20.474 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.322465+0000", 2024-09-18T00:18:20.474 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.118400929, 2024-09-18T00:18:20.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0075526680000000002, 2024-09-18T00:18:20.474 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.474 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.474 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.474 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322465+0000", 2024-09-18T00:18:20.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322465+0000", 2024-09-18T00:18:20.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322467+0000", 2024-09-18T00:18:20.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5999999999999999e-06 2024-09-18T00:18:20.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322612+0000", 2024-09-18T00:18:20.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014553499999999999 2024-09-18T00:18:20.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322631+0000", 2024-09-18T00:18:20.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8627e-05 2024-09-18T00:18:20.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.322645+0000", 2024-09-18T00:18:20.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4109e-05 2024-09-18T00:18:20.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325646+0000", 2024-09-18T00:18:20.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0030007179999999999 2024-09-18T00:18:20.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325674+0000", 2024-09-18T00:18:20.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7733e-05 2024-09-18T00:18:20.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.329958+0000", 2024-09-18T00:18:20.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042841329999999999 2024-09-18T00:18:20.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330018+0000", 2024-09-18T00:18:20.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0213000000000001e-05 2024-09-18T00:18:20.478 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.478 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.478 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.478 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:138 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'96, mlcod=17'85)", 2024-09-18T00:18:20.478 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.323237+0000", 2024-09-18T00:18:20.478 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.117629782000002, 2024-09-18T00:18:20.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0068313540000000004, 2024-09-18T00:18:20.478 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.478 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.478 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.478 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323237+0000", 2024-09-18T00:18:20.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323237+0000", 2024-09-18T00:18:20.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323238+0000", 2024-09-18T00:18:20.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.127e-06 2024-09-18T00:18:20.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323249+0000", 2024-09-18T00:18:20.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1157e-05 2024-09-18T00:18:20.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323277+0000", 2024-09-18T00:18:20.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8178999999999999e-05 2024-09-18T00:18:20.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323287+0000", 2024-09-18T00:18:20.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9699999999999994e-06 2024-09-18T00:18:20.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327148+0000", 2024-09-18T00:18:20.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038609109999999999 2024-09-18T00:18:20.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327178+0000", 2024-09-18T00:18:20.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0156000000000001e-05 2024-09-18T00:18:20.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330022+0000", 2024-09-18T00:18:20.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028444579999999998 2024-09-18T00:18:20.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330068+0000", 2024-09-18T00:18:20.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5395999999999997e-05 2024-09-18T00:18:20.482 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.482 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.482 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.482 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:139 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'86)", 2024-09-18T00:18:20.482 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.323792+0000", 2024-09-18T00:18:20.482 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.117074132999999, 2024-09-18T00:18:20.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063243850000000001, 2024-09-18T00:18:20.483 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.483 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.483 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.483 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323792+0000", 2024-09-18T00:18:20.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323792+0000", 2024-09-18T00:18:20.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323793+0000", 2024-09-18T00:18:20.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4399999999999998e-07 2024-09-18T00:18:20.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323806+0000", 2024-09-18T00:18:20.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3346000000000001e-05 2024-09-18T00:18:20.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323824+0000", 2024-09-18T00:18:20.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.789e-05 2024-09-18T00:18:20.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.323836+0000", 2024-09-18T00:18:20.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2082999999999999e-05 2024-09-18T00:18:20.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327773+0000", 2024-09-18T00:18:20.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039365099999999998 2024-09-18T00:18:20.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.327802+0000", 2024-09-18T00:18:20.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8617000000000001e-05 2024-09-18T00:18:20.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330072+0000", 2024-09-18T00:18:20.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022704219999999998 2024-09-18T00:18:20.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330117+0000", 2024-09-18T00:18:20.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4573000000000002e-05 2024-09-18T00:18:20.486 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.486 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.487 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.487 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:140 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'86)", 2024-09-18T00:18:20.487 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.324263+0000", 2024-09-18T00:18:20.487 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.116603680000001, 2024-09-18T00:18:20.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0066665350000000003, 2024-09-18T00:18:20.487 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.487 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.487 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.487 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324263+0000", 2024-09-18T00:18:20.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324263+0000", 2024-09-18T00:18:20.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324264+0000", 2024-09-18T00:18:20.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-18T00:18:20.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324387+0000", 2024-09-18T00:18:20.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012346899999999999 2024-09-18T00:18:20.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324402+0000", 2024-09-18T00:18:20.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5087e-05 2024-09-18T00:18:20.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.324412+0000", 2024-09-18T00:18:20.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5969999999999998e-06 2024-09-18T00:18:20.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.328364+0000", 2024-09-18T00:18:20.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039523559999999998 2024-09-18T00:18:20.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.328397+0000", 2024-09-18T00:18:20.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2733e-05 2024-09-18T00:18:20.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330864+0000", 2024-09-18T00:18:20.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0024676670000000002 2024-09-18T00:18:20.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330929+0000", 2024-09-18T00:18:20.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4665999999999994e-05 2024-09-18T00:18:20.491 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.491 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.491 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.491 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:141 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'86)", 2024-09-18T00:18:20.491 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.325030+0000", 2024-09-18T00:18:20.491 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.115836701999999, 2024-09-18T00:18:20.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006603462, 2024-09-18T00:18:20.491 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.491 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.492 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.492 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325030+0000", 2024-09-18T00:18:20.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325030+0000", 2024-09-18T00:18:20.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325031+0000", 2024-09-18T00:18:20.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.53e-06 2024-09-18T00:18:20.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325042+0000", 2024-09-18T00:18:20.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0506000000000001e-05 2024-09-18T00:18:20.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325080+0000", 2024-09-18T00:18:20.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8300000000000003e-05 2024-09-18T00:18:20.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.325098+0000", 2024-09-18T00:18:20.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8539999999999999e-05 2024-09-18T00:18:20.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330164+0000", 2024-09-18T00:18:20.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050653709999999999 2024-09-18T00:18:20.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.330194+0000", 2024-09-18T00:18:20.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0275999999999999e-05 2024-09-18T00:18:20.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.331562+0000", 2024-09-18T00:18:20.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0013680330000000001 2024-09-18T00:18:20.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.331633+0000", 2024-09-18T00:18:20.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0906000000000002e-05 2024-09-18T00:18:20.495 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.495 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.495 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.496 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:142 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'100, mlcod=17'98)", 2024-09-18T00:18:20.496 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.332949+0000", 2024-09-18T00:18:20.496 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.107917007000001, 2024-09-18T00:18:20.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059501479999999997, 2024-09-18T00:18:20.496 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.496 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.496 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.496 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.332949+0000", 2024-09-18T00:18:20.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.332949+0000", 2024-09-18T00:18:20.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.332951+0000", 2024-09-18T00:18:20.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8640000000000001e-06 2024-09-18T00:18:20.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333024+0000", 2024-09-18T00:18:20.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3112000000000001e-05 2024-09-18T00:18:20.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333050+0000", 2024-09-18T00:18:20.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5673e-05 2024-09-18T00:18:20.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333063+0000", 2024-09-18T00:18:20.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2644e-05 2024-09-18T00:18:20.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333228+0000", 2024-09-18T00:18:20.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016579799999999999 2024-09-18T00:18:20.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333267+0000", 2024-09-18T00:18:20.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8189999999999999e-05 2024-09-18T00:18:20.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338834+0000", 2024-09-18T00:18:20.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055676679999999996 2024-09-18T00:18:20.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338899+0000", 2024-09-18T00:18:20.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5198999999999997e-05 2024-09-18T00:18:20.500 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.500 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.500 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.500 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4157.0:143 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'101, mlcod=17'98)", 2024-09-18T00:18:20.500 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:52.333556+0000", 2024-09-18T00:18:20.500 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.107309952000001, 2024-09-18T00:18:20.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005386053, 2024-09-18T00:18:20.501 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.501 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.501 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.501 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333556+0000", 2024-09-18T00:18:20.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333556+0000", 2024-09-18T00:18:20.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333558+0000", 2024-09-18T00:18:20.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:18:20.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333677+0000", 2024-09-18T00:18:20.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119196 2024-09-18T00:18:20.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333698+0000", 2024-09-18T00:18:20.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1087e-05 2024-09-18T00:18:20.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.333708+0000", 2024-09-18T00:18:20.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0125999999999999e-05 2024-09-18T00:18:20.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334689+0000", 2024-09-18T00:18:20.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00098056899999999993 2024-09-18T00:18:20.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.334719+0000", 2024-09-18T00:18:20.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9963000000000001e-05 2024-09-18T00:18:20.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338907+0000", 2024-09-18T00:18:20.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041881619999999996 2024-09-18T00:18:20.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:52.338942+0000", 2024-09-18T00:18:20.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5660000000000001e-05 2024-09-18T00:18:20.505 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.505 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.505 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.505 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T00:18:20.505 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:07.405976+0000", 2024-09-18T00:18:20.505 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.034890651, 2024-09-18T00:18:20.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0063813710000000003, 2024-09-18T00:18:20.505 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.505 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.505 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.506 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:20.506 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:20.506 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:20.506 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11 2024-09-18T00:18:20.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.506 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.405976+0000", 2024-09-18T00:18:20.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.405976+0000", 2024-09-18T00:18:20.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.405979+0000", 2024-09-18T00:18:20.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9009999999999998e-06 2024-09-18T00:18:20.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.405998+0000", 2024-09-18T00:18:20.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9653e-05 2024-09-18T00:18:20.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.406027+0000", 2024-09-18T00:18:20.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8442e-05 2024-09-18T00:18:20.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.406041+0000", 2024-09-18T00:18:20.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4273e-05 2024-09-18T00:18:20.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.406141+0000", 2024-09-18T00:18:20.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010008199999999999 2024-09-18T00:18:20.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:18:20.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.406147+0000", 2024-09-18T00:18:20.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9340000000000003e-06 2024-09-18T00:18:20.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.408986+0000", 2024-09-18T00:18:20.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028392959999999998 2024-09-18T00:18:20.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.409279+0000", 2024-09-18T00:18:20.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000292596 2024-09-18T00:18:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.409615+0000", 2024-09-18T00:18:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033606 2024-09-18T00:18:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.411318+0000", 2024-09-18T00:18:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017026890000000001 2024-09-18T00:18:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:20.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.412269+0000", 2024-09-18T00:18:20.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00095177200000000003 2024-09-18T00:18:20.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.412319+0000", 2024-09-18T00:18:20.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9975999999999997e-05 2024-09-18T00:18:20.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:07.412357+0000", 2024-09-18T00:18:20.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7697000000000002e-05 2024-09-18T00:18:20.511 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.511 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.511 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.512 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:231 3.17 3:ee570356:::rbd_data.112a45c8e167.0000000000000037:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1398888~44282] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T00:18:20.512 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.796692+0000", 2024-09-18T00:18:20.512 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.6441739949999992, 2024-09-18T00:18:20.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0077415899999999996, 2024-09-18T00:18:20.512 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.512 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.512 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.512 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:20.512 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:20.512 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:20.512 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 231 2024-09-18T00:18:20.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.513 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796692+0000", 2024-09-18T00:18:20.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796692+0000", 2024-09-18T00:18:20.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796694+0000", 2024-09-18T00:18:20.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.291e-06 2024-09-18T00:18:20.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796736+0000", 2024-09-18T00:18:20.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2082999999999997e-05 2024-09-18T00:18:20.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796747+0000", 2024-09-18T00:18:20.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1144999999999999e-05 2024-09-18T00:18:20.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796752+0000", 2024-09-18T00:18:20.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5439999999999998e-06 2024-09-18T00:18:20.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796844+0000", 2024-09-18T00:18:20.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1847000000000003e-05 2024-09-18T00:18:20.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.796952+0000", 2024-09-18T00:18:20.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108223 2024-09-18T00:18:20.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:20.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.797040+0000", 2024-09-18T00:18:20.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7416000000000005e-05 2024-09-18T00:18:20.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:20.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.802075+0000", 2024-09-18T00:18:20.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050354390000000001 2024-09-18T00:18:20.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:20.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.804151+0000", 2024-09-18T00:18:20.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0020752769999999999 2024-09-18T00:18:20.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.804314+0000", 2024-09-18T00:18:20.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016367899999999999 2024-09-18T00:18:20.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.804434+0000", 2024-09-18T00:18:20.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119646 2024-09-18T00:18:20.517 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.517 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.517 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.518 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:763 3.7 e30/27 3:e4fee1a8:::rbd_data.112ae20746f5.00000000000001cb:head v 30'6, mlcod=29'4)", 2024-09-18T00:18:20.518 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.458804+0000", 2024-09-18T00:18:20.518 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.9820626279999995, 2024-09-18T00:18:20.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0060225469999999996, 2024-09-18T00:18:20.518 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.518 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.518 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.518 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458804+0000", 2024-09-18T00:18:20.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458804+0000", 2024-09-18T00:18:20.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458804+0000", 2024-09-18T00:18:20.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7599999999999998e-07 2024-09-18T00:18:20.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458809+0000", 2024-09-18T00:18:20.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1200000000000001e-06 2024-09-18T00:18:20.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458821+0000", 2024-09-18T00:18:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2176e-05 2024-09-18T00:18:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458830+0000", 2024-09-18T00:18:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1859999999999992e-06 2024-09-18T00:18:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458929+0000", 2024-09-18T00:18:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9661999999999995e-05 2024-09-18T00:18:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458950+0000", 2024-09-18T00:18:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0424e-05 2024-09-18T00:18:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.464761+0000", 2024-09-18T00:18:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0058110389999999996 2024-09-18T00:18:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.464826+0000", 2024-09-18T00:18:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5563999999999998e-05 2024-09-18T00:18:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2230 3.17 3:ef492888:::rbd_data.112aa7ba4578.000000000000014f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:18:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.17485532100000001, 2024-09-18T00:18:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075065283999999996, 2024-09-18T00:18:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.522 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2230 2024-09-18T00:18:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266014+0000", 2024-09-18T00:18:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:18:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266029+0000", 2024-09-18T00:18:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4616e-05 2024-09-18T00:18:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266053+0000", 2024-09-18T00:18:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4369999999999999e-05 2024-09-18T00:18:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266067+0000", 2024-09-18T00:18:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4243999999999999e-05 2024-09-18T00:18:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266157+0000", 2024-09-18T00:18:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9909999999999998e-05 2024-09-18T00:18:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266327+0000", 2024-09-18T00:18:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016981200000000001 2024-09-18T00:18:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269180+0000", 2024-09-18T00:18:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028526559999999999 2024-09-18T00:18:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340955+0000", 2024-09-18T00:18:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.071774904 2024-09-18T00:18:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340974+0000", 2024-09-18T00:18:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8824000000000001e-05 2024-09-18T00:18:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341028+0000", 2024-09-18T00:18:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4456999999999997e-05 2024-09-18T00:18:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341076+0000", 2024-09-18T00:18:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8284000000000001e-05 2024-09-18T00:18:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2229 3.10 e36/23 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head v 36'8, mlcod=35'6)", 2024-09-18T00:18:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.269628+0000", 2024-09-18T00:18:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.17123850700000001, 2024-09-18T00:18:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013356332, 2024-09-18T00:18:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.528 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269628+0000", 2024-09-18T00:18:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269628+0000", 2024-09-18T00:18:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269630+0000", 2024-09-18T00:18:20.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2299999999999998e-06 2024-09-18T00:18:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.281469+0000", 2024-09-18T00:18:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011838589 2024-09-18T00:18:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.281505+0000", 2024-09-18T00:18:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6412999999999998e-05 2024-09-18T00:18:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.281523+0000", 2024-09-18T00:18:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8122999999999999e-05 2024-09-18T00:18:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.281695+0000", 2024-09-18T00:18:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017173500000000001 2024-09-18T00:18:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.281748+0000", 2024-09-18T00:18:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3059999999999997e-05 2024-09-18T00:18:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.282951+0000", 2024-09-18T00:18:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001203345 2024-09-18T00:18:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.282984+0000", 2024-09-18T00:18:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2836999999999997e-05 2024-09-18T00:18:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2230 3.17 e36/23 ondisk, result = 0)", 2024-09-18T00:18:20.532 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.15751174000000001, 2024-09-18T00:18:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057609645000000001, 2024-09-18T00:18:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283356+0000", 2024-09-18T00:18:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.44e-06 2024-09-18T00:18:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283362+0000", 2024-09-18T00:18:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2539999999999998e-06 2024-09-18T00:18:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283382+0000", 2024-09-18T00:18:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9896000000000001e-05 2024-09-18T00:18:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283398+0000", 2024-09-18T00:18:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5489999999999999e-05 2024-09-18T00:18:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340936+0000", 2024-09-18T00:18:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057538649999999997 2024-09-18T00:18:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340947+0000", 2024-09-18T00:18:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0981e-05 2024-09-18T00:18:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340964+0000", 2024-09-18T00:18:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6934000000000001e-05 2024-09-18T00:18:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.536 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.536 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:20.537 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:18:20.537 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T00:18:20.538 INFO:teuthology.orchestra.run.smithi097.stdout:719 trunc from 0x9a6abc6 to 0x5321ce5 2024-09-18T00:18:20.539 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:18:20.540 INFO:teuthology.orchestra.run.smithi097.stdout:721 write 0x8e9829f thru 0x8ea2f41 (0xaca3 bytes) 2024-09-18T00:18:20.540 INFO:teuthology.orchestra.run.smithi097.stdout:723 punch from 0x64d9fc1 to 0x64e6fad, (0xcfec bytes) 2024-09-18T00:18:20.540 INFO:teuthology.orchestra.run.smithi097.stdout:724 writesame 0x88272f2 thru 0x882cd6e data_size 0x29(0x5a7d bytes) 2024-09-18T00:18:20.540 INFO:teuthology.orchestra.run.smithi097.stdout:725 writesame 0xd64722e thru 0xd654161 data_size 0x4(0xcf34 bytes) 2024-09-18T00:18:20.540 INFO:teuthology.orchestra.run.smithi097.stdout:726 write 0x110632 thru 0x11e56b (0xdf3a bytes) 2024-09-18T00:18:20.540 INFO:teuthology.orchestra.run.smithi097.stdout:727 write 0x2387ea5 thru 0x2388cf0 (0xe4c bytes) 2024-09-18T00:18:20.541 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:20.535+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:20.551 INFO:teuthology.orchestra.run.smithi097.stdout:728 trunc from 0xd654162 to 0xe7c991d 2024-09-18T00:18:20.562 INFO:teuthology.orchestra.run.smithi097.stdout:730 read 0xc95ce5c thru 0xc961921 (0x4ac6 bytes) 2024-09-18T00:18:20.583 INFO:teuthology.orchestra.run.smithi097.stdout:731 trunc from 0xe7c991d to 0x167dc1e 2024-09-18T00:18:20.633 INFO:teuthology.orchestra.run.smithi097.stdout:732 write 0x218a9f3 thru 0x218ee5a (0x4468 bytes) 2024-09-18T00:18:20.638 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:20.635+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:20.642 INFO:teuthology.orchestra.run.smithi097.stdout:733 trunc from 0x218ee5b to 0x13df16c 2024-09-18T00:18:20.670 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:20.670 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:20.670 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:20.670 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:20.688 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:18:20.694 INFO:teuthology.orchestra.run.smithi097.stdout:735 writesame 0x3d6cc99 thru 0x3d7a12c data_size 0x1(0xd494 bytes) 2024-09-18T00:18:20.694 INFO:teuthology.orchestra.run.smithi097.stdout:736 read 0x1c729ee thru 0x1c7a520 (0x7b33 bytes) 2024-09-18T00:18:20.721 INFO:teuthology.orchestra.run.smithi097.stdout:737 writesame 0xe398b3b thru 0xe3a640f data_size 0x1(0xd8d5 bytes) 2024-09-18T00:18:20.721 INFO:teuthology.orchestra.run.smithi097.stdout:739 punch from 0x6fb98a0 to 0x6fc8916, (0xf076 bytes) 2024-09-18T00:18:20.722 INFO:teuthology.orchestra.run.smithi097.stdout:741 read 0xaea0af6 thru 0xaead1fa (0xc705 bytes) 2024-09-18T00:18:20.722 INFO:teuthology.orchestra.run.smithi097.stdout:743 write 0x403ad8d thru 0x4044d52 (0x9fc6 bytes) 2024-09-18T00:18:20.730 INFO:teuthology.orchestra.run.smithi097.stdout:745 writesame 0x6c8835c thru 0x6c9607c data_size 0x1(0xdd21 bytes) 2024-09-18T00:18:20.739 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:20.735+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:20.814 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:20.814 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:20.814 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:20.814 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:20.814 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:20.828 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:18:20.840 INFO:teuthology.orchestra.run.smithi097.stdout:746 read 0x478e40b thru 0x4794af3 (0x66e9 bytes) 2024-09-18T00:18:20.841 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:20.839+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:20.868 INFO:teuthology.orchestra.run.smithi097.stdout:748 trunc from 0xe3a6410 to 0x1531230 2024-09-18T00:18:20.927 INFO:teuthology.orchestra.run.smithi097.stdout:749 writesame 0xbdcdb0f thru 0xbdd2634 data_size 0x1(0x4b26 bytes) 2024-09-18T00:18:20.927 INFO:teuthology.orchestra.run.smithi097.stdout:751 read 0x9bec1b1 thru 0x9beeda5 (0x2bf5 bytes) 2024-09-18T00:18:20.927 INFO:teuthology.orchestra.run.smithi097.stdout:758 read 0x69cfef9 thru 0x69dde40 (0xdf48 bytes) 2024-09-18T00:18:20.927 INFO:teuthology.orchestra.run.smithi097.stdout:759 read 0xb320e9b thru 0xb3284bc (0x7622 bytes) 2024-09-18T00:18:20.927 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:20.927 INFO:teuthology.orchestra.run.smithi097.stdout:762 punch from 0xa029f1a to 0xa031e6a, (0x7f50 bytes) 2024-09-18T00:18:20.927 INFO:teuthology.orchestra.run.smithi097.stdout:764 read 0x8f42cdb thru 0x8f48586 (0x58ac bytes) 2024-09-18T00:18:20.928 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:20.928 INFO:teuthology.orchestra.run.smithi097.stdout:766 trunc from 0xbdd2635 to 0xc18115c 2024-09-18T00:18:20.940 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:20.940 INFO:teuthology.orchestra.run.smithi097.stdout:770 write 0x22b4af2 thru 0x22bbef5 (0x7404 bytes) 2024-09-18T00:18:20.941 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:20.939+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:20.951 INFO:teuthology.orchestra.run.smithi097.stdout:772 read 0xa0f07db thru 0xa0fce7c (0xc6a2 bytes) 2024-09-18T00:18:20.969 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:220 3.19 3:9e8c0c63:::rbd_data.112a45c8e167.0000000000000031:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T00:18:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.761391+0000", 2024-09-18T00:18:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.204769236000001, 2024-09-18T00:18:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062183000000000004, 2024-09-18T00:18:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 220 2024-09-18T00:18:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761391+0000", 2024-09-18T00:18:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761391+0000", 2024-09-18T00:18:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761392+0000", 2024-09-18T00:18:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.838e-06 2024-09-18T00:18:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761407+0000", 2024-09-18T00:18:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4538999999999999e-05 2024-09-18T00:18:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761421+0000", 2024-09-18T00:18:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3896e-05 2024-09-18T00:18:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761427+0000", 2024-09-18T00:18:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2519999999999996e-06 2024-09-18T00:18:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761507+0000", 2024-09-18T00:18:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9909999999999999e-05 2024-09-18T00:18:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761697+0000", 2024-09-18T00:18:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000189834 2024-09-18T00:18:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-18T00:18:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.762095+0000", 2024-09-18T00:18:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00039823799999999998 2024-09-18T00:18:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767289+0000", 2024-09-18T00:18:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0051936930000000001 2024-09-18T00:18:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767536+0000", 2024-09-18T00:18:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024718899999999998 2024-09-18T00:18:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767573+0000", 2024-09-18T00:18:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6792000000000003e-05 2024-09-18T00:18:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767609+0000", 2024-09-18T00:18:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6118999999999998e-05 2024-09-18T00:18:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:220 3.19 e28/23 ondisk, result = 0)", 2024-09-18T00:18:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.763702+0000", 2024-09-18T00:18:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.202457779, 2024-09-18T00:18:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035976879999999999, 2024-09-18T00:18:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763702+0000", 2024-09-18T00:18:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763702+0000", 2024-09-18T00:18:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763703+0000", 2024-09-18T00:18:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5099999999999999e-07 2024-09-18T00:18:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763707+0000", 2024-09-18T00:18:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7450000000000003e-06 2024-09-18T00:18:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763720+0000", 2024-09-18T00:18:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2215000000000001e-05 2024-09-18T00:18:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.763725+0000", 2024-09-18T00:18:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.604e-06 2024-09-18T00:18:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767267+0000", 2024-09-18T00:18:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035417729999999998 2024-09-18T00:18:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767279+0000", 2024-09-18T00:18:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2267e-05 2024-09-18T00:18:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767300+0000", 2024-09-18T00:18:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0333e-05 2024-09-18T00:18:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:372 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.457128+0000", 2024-09-18T00:18:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.5090313129999995, 2024-09-18T00:18:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003527082, 2024-09-18T00:18:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 372 2024-09-18T00:18:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457128+0000", 2024-09-18T00:18:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457128+0000", 2024-09-18T00:18:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457131+0000", 2024-09-18T00:18:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9739999999999998e-06 2024-09-18T00:18:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457146+0000", 2024-09-18T00:18:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4516e-05 2024-09-18T00:18:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457178+0000", 2024-09-18T00:18:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2475999999999999e-05 2024-09-18T00:18:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457199+0000", 2024-09-18T00:18:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:18:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457348+0000", 2024-09-18T00:18:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014881200000000001 2024-09-18T00:18:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457602+0000", 2024-09-18T00:18:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000253732 2024-09-18T00:18:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.457977+0000", 2024-09-18T00:18:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037547299999999997 2024-09-18T00:18:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.459553+0000", 2024-09-18T00:18:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001576261 2024-09-18T00:18:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460497+0000", 2024-09-18T00:18:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094395500000000001 2024-09-18T00:18:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460588+0000", 2024-09-18T00:18:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0443000000000001e-05 2024-09-18T00:18:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460656+0000", 2024-09-18T00:18:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7650000000000005e-05 2024-09-18T00:18:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:373 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.set_stripe_unit_count in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.460868+0000", 2024-09-18T00:18:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.5052914049999995, 2024-09-18T00:18:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031188050000000001, 2024-09-18T00:18:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 373 2024-09-18T00:18:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460868+0000", 2024-09-18T00:18:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460868+0000", 2024-09-18T00:18:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460870+0000", 2024-09-18T00:18:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.733e-06 2024-09-18T00:18:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460880+0000", 2024-09-18T00:18:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5429999999999999e-06 2024-09-18T00:18:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460913+0000", 2024-09-18T00:18:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3333000000000001e-05 2024-09-18T00:18:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.460937+0000", 2024-09-18T00:18:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3836999999999999e-05 2024-09-18T00:18:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.461083+0000", 2024-09-18T00:18:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145952 2024-09-18T00:18:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.461327+0000", 2024-09-18T00:18:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024425100000000002 2024-09-18T00:18:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.461895+0000", 2024-09-18T00:18:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00056844600000000001 2024-09-18T00:18:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.462901+0000", 2024-09-18T00:18:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001005939 2024-09-18T00:18:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.463800+0000", 2024-09-18T00:18:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00089826999999999995 2024-09-18T00:18:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.463885+0000", 2024-09-18T00:18:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5768999999999998e-05 2024-09-18T00:18:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.463987+0000", 2024-09-18T00:18:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010173199999999999 2024-09-18T00:18:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:387 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [watch watch cookie 140686016265056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.485197+0000", 2024-09-18T00:18:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.4809630810000005, 2024-09-18T00:18:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029145590000000002, 2024-09-18T00:18:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 387 2024-09-18T00:18:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485197+0000", 2024-09-18T00:18:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485197+0000", 2024-09-18T00:18:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485199+0000", 2024-09-18T00:18:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4849999999999999e-06 2024-09-18T00:18:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485209+0000", 2024-09-18T00:18:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0251999999999999e-05 2024-09-18T00:18:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485238+0000", 2024-09-18T00:18:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8708e-05 2024-09-18T00:18:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485255+0000", 2024-09-18T00:18:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7011e-05 2024-09-18T00:18:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485400+0000", 2024-09-18T00:18:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000144504 2024-09-18T00:18:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.485654+0000", 2024-09-18T00:18:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025417400000000002 2024-09-18T00:18:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.486056+0000", 2024-09-18T00:18:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000402147 2024-09-18T00:18:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.487262+0000", 2024-09-18T00:18:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0012063480000000001 2024-09-18T00:18:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.487908+0000", 2024-09-18T00:18:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00064552100000000003 2024-09-18T00:18:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.487993+0000", 2024-09-18T00:18:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5263e-05 2024-09-18T00:18:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488111+0000", 2024-09-18T00:18:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000118146 2024-09-18T00:18:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:388 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.parent_attach in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.488320+0000", 2024-09-18T00:18:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.4778397309999995, 2024-09-18T00:18:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0030081399999999999, 2024-09-18T00:18:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 388 2024-09-18T00:18:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488320+0000", 2024-09-18T00:18:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488320+0000", 2024-09-18T00:18:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488322+0000", 2024-09-18T00:18:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7430000000000001e-06 2024-09-18T00:18:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488331+0000", 2024-09-18T00:18:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5880000000000004e-06 2024-09-18T00:18:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488356+0000", 2024-09-18T00:18:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4403e-05 2024-09-18T00:18:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488368+0000", 2024-09-18T00:18:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2690999999999999e-05 2024-09-18T00:18:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488485+0000", 2024-09-18T00:18:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011662099999999999 2024-09-18T00:18:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.488623+0000", 2024-09-18T00:18:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000137472 2024-09-18T00:18:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.489032+0000", 2024-09-18T00:18:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00040919699999999998 2024-09-18T00:18:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.490129+0000", 2024-09-18T00:18:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001096765 2024-09-18T00:18:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.491175+0000", 2024-09-18T00:18:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0010461380000000001 2024-09-18T00:18:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.491259+0000", 2024-09-18T00:18:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4625999999999999e-05 2024-09-18T00:18:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.491328+0000", 2024-09-18T00:18:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8895999999999999e-05 2024-09-18T00:18:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:554 3.8 e29/23 3:14210294:::rbd_data.112ae20746f5.0000000000000133:head v 29'1, mlcod=0'0)", 2024-09-18T00:18:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.879349+0000", 2024-09-18T00:18:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0868108089999993, 2024-09-18T00:18:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043514469999999996, 2024-09-18T00:18:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879349+0000", 2024-09-18T00:18:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879349+0000", 2024-09-18T00:18:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879351+0000", 2024-09-18T00:18:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8050000000000001e-06 2024-09-18T00:18:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879392+0000", 2024-09-18T00:18:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1054999999999999e-05 2024-09-18T00:18:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879413+0000", 2024-09-18T00:18:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0784999999999998e-05 2024-09-18T00:18:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879425+0000", 2024-09-18T00:18:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2896e-05 2024-09-18T00:18:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879540+0000", 2024-09-18T00:18:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011473500000000001 2024-09-18T00:18:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879579+0000", 2024-09-18T00:18:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8578999999999998e-05 2024-09-18T00:18:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.883623+0000", 2024-09-18T00:18:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040441959999999999 2024-09-18T00:18:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.883700+0000", 2024-09-18T00:18:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7396000000000002e-05 2024-09-18T00:18:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:668 3.1a e29/23 3:59ee8e8b:::rbd_data.112ae20746f5.0000000000000031:head v 29'3, mlcod=25'1)", 2024-09-18T00:18:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.170930+0000", 2024-09-18T00:18:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.7952296699999994, 2024-09-18T00:18:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052043310000000004, 2024-09-18T00:18:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.170930+0000", 2024-09-18T00:18:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.170930+0000", 2024-09-18T00:18:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.170932+0000", 2024-09-18T00:18:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4699999999999999e-06 2024-09-18T00:18:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171038+0000", 2024-09-18T00:18:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106205 2024-09-18T00:18:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171061+0000", 2024-09-18T00:18:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3329999999999999e-05 2024-09-18T00:18:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171072+0000", 2024-09-18T00:18:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0806e-05 2024-09-18T00:18:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171201+0000", 2024-09-18T00:18:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012925799999999999 2024-09-18T00:18:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171239+0000", 2024-09-18T00:18:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7866e-05 2024-09-18T00:18:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.176016+0000", 2024-09-18T00:18:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047769930000000002 2024-09-18T00:18:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.176134+0000", 2024-09-18T00:18:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000118403 2024-09-18T00:18:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:692 3.0 3:03fab4ad:::rbd_data.112ae20746f5.0000000000000057:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.238016+0000", 2024-09-18T00:18:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.7281439600000006, 2024-09-18T00:18:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0087112969999999998, 2024-09-18T00:18:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 692 2024-09-18T00:18:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238016+0000", 2024-09-18T00:18:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238016+0000", 2024-09-18T00:18:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238017+0000", 2024-09-18T00:18:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.257e-06 2024-09-18T00:18:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238023+0000", 2024-09-18T00:18:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5470000000000004e-06 2024-09-18T00:18:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238046+0000", 2024-09-18T00:18:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3088e-05 2024-09-18T00:18:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238054+0000", 2024-09-18T00:18:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3569999999999998e-06 2024-09-18T00:18:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238135+0000", 2024-09-18T00:18:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0814000000000002e-05 2024-09-18T00:18:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238331+0000", 2024-09-18T00:18:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019594 2024-09-18T00:18:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238606+0000", 2024-09-18T00:18:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027473200000000003 2024-09-18T00:18:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.240010+0000", 2024-09-18T00:18:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001404329 2024-09-18T00:18:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246563+0000", 2024-09-18T00:18:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0065535630000000001 2024-09-18T00:18:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246643+0000", 2024-09-18T00:18:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9468999999999994e-05 2024-09-18T00:18:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246727+0000", 2024-09-18T00:18:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4201000000000002e-05 2024-09-18T00:18:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:733 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.365564+0000", 2024-09-18T00:18:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.6005961600000003, 2024-09-18T00:18:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038647540000000002, 2024-09-18T00:18:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 733 2024-09-18T00:18:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365564+0000", 2024-09-18T00:18:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365564+0000", 2024-09-18T00:18:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365566+0000", 2024-09-18T00:18:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1909999999999999e-06 2024-09-18T00:18:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365589+0000", 2024-09-18T00:18:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3096999999999999e-05 2024-09-18T00:18:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365616+0000", 2024-09-18T00:18:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7577e-05 2024-09-18T00:18:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365628+0000", 2024-09-18T00:18:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1804000000000001e-05 2024-09-18T00:18:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365708+0000", 2024-09-18T00:18:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9657e-05 2024-09-18T00:18:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365797+0000", 2024-09-18T00:18:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8812000000000003e-05 2024-09-18T00:18:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.365976+0000", 2024-09-18T00:18:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000178853 2024-09-18T00:18:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.367005+0000", 2024-09-18T00:18:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001029196 2024-09-18T00:18:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.369312+0000", 2024-09-18T00:18:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0023069079999999999 2024-09-18T00:18:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.369375+0000", 2024-09-18T00:18:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2823000000000005e-05 2024-09-18T00:18:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.369428+0000", 2024-09-18T00:18:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3835999999999998e-05 2024-09-18T00:18:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:746 3.0 3:03157719:::rbd_data.112ae20746f5.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.389598+0000", 2024-09-18T00:18:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.5765621930000009, 2024-09-18T00:18:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006052516, 2024-09-18T00:18:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 746 2024-09-18T00:18:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389598+0000", 2024-09-18T00:18:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389598+0000", 2024-09-18T00:18:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389599+0000", 2024-09-18T00:18:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.844e-06 2024-09-18T00:18:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389609+0000", 2024-09-18T00:18:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1039999999999996e-06 2024-09-18T00:18:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389623+0000", 2024-09-18T00:18:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4384e-05 2024-09-18T00:18:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389632+0000", 2024-09-18T00:18:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5589999999999994e-06 2024-09-18T00:18:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389715+0000", 2024-09-18T00:18:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2185e-05 2024-09-18T00:18:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389923+0000", 2024-09-18T00:18:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000208214 2024-09-18T00:18:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.390263+0000", 2024-09-18T00:18:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033984 2024-09-18T00:18:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392457+0000", 2024-09-18T00:18:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021944180000000001 2024-09-18T00:18:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395565+0000", 2024-09-18T00:18:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031083450000000002 2024-09-18T00:18:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395616+0000", 2024-09-18T00:18:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0949000000000003e-05 2024-09-18T00:18:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395650+0000", 2024-09-18T00:18:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3674000000000003e-05 2024-09-18T00:18:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:749 3.1a e29/23 3:5e11336d:::rbd_data.112ae20746f5.000000000000004e:head v 29'4, mlcod=26'2)", 2024-09-18T00:18:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.403303+0000", 2024-09-18T00:18:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.562857223, 2024-09-18T00:18:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0044570729999999998, 2024-09-18T00:18:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403303+0000", 2024-09-18T00:18:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403303+0000", 2024-09-18T00:18:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403303+0000", 2024-09-18T00:18:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6300000000000004e-07 2024-09-18T00:18:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403446+0000", 2024-09-18T00:18:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014246500000000001 2024-09-18T00:18:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403461+0000", 2024-09-18T00:18:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4904e-05 2024-09-18T00:18:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403470+0000", 2024-09-18T00:18:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9020000000000005e-06 2024-09-18T00:18:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403582+0000", 2024-09-18T00:18:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112161 2024-09-18T00:18:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.403610+0000", 2024-09-18T00:18:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7671e-05 2024-09-18T00:18:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.407712+0000", 2024-09-18T00:18:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041027449999999997 2024-09-18T00:18:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.407760+0000", 2024-09-18T00:18:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7361999999999998e-05 2024-09-18T00:18:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:754 3.1a e30/23 3:5e11336d:::rbd_data.112ae20746f5.000000000000004e:head v 30'5, mlcod=29'3)", 2024-09-18T00:18:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.459551+0000", 2024-09-18T00:18:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.5066090089999999, 2024-09-18T00:18:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029016850000000002, 2024-09-18T00:18:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459551+0000", 2024-09-18T00:18:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459551+0000", 2024-09-18T00:18:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459552+0000", 2024-09-18T00:18:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9300000000000006e-07 2024-09-18T00:18:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459557+0000", 2024-09-18T00:18:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3700000000000003e-06 2024-09-18T00:18:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459584+0000", 2024-09-18T00:18:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.709e-05 2024-09-18T00:18:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459593+0000", 2024-09-18T00:18:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4009999999999997e-06 2024-09-18T00:18:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459650+0000", 2024-09-18T00:18:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7436999999999999e-05 2024-09-18T00:18:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459671+0000", 2024-09-18T00:18:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0704999999999999e-05 2024-09-18T00:18:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.462395+0000", 2024-09-18T00:18:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027243990000000002 2024-09-18T00:18:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.462452+0000", 2024-09-18T00:18:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7290000000000002e-05 2024-09-18T00:18:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:839 3.0 3:0095abde:::rbd_data.112ae20746f5.0000000000000116:head [set-alloc-hint object_size 524288 write_size 524288,write 81590~25537] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.774325+0000", 2024-09-18T00:18:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.1918348590000001, 2024-09-18T00:18:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033986020000000001, 2024-09-18T00:18:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 839 2024-09-18T00:18:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774325+0000", 2024-09-18T00:18:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774325+0000", 2024-09-18T00:18:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774326+0000", 2024-09-18T00:18:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2419999999999999e-06 2024-09-18T00:18:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774342+0000", 2024-09-18T00:18:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5586999999999998e-05 2024-09-18T00:18:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774357+0000", 2024-09-18T00:18:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4986e-05 2024-09-18T00:18:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774367+0000", 2024-09-18T00:18:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9909999999999997e-06 2024-09-18T00:18:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774446+0000", 2024-09-18T00:18:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9419000000000007e-05 2024-09-18T00:18:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774611+0000", 2024-09-18T00:18:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016517600000000001 2024-09-18T00:18:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774804+0000", 2024-09-18T00:18:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019286300000000001 2024-09-18T00:18:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.777346+0000", 2024-09-18T00:18:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025422399999999999 2024-09-18T00:18:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.777606+0000", 2024-09-18T00:18:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025912400000000001 2024-09-18T00:18:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.777670+0000", 2024-09-18T00:18:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4785999999999999e-05 2024-09-18T00:18:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.777724+0000", 2024-09-18T00:18:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3188e-05 2024-09-18T00:18:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:870 3.0 3:05335421:::rbd_data.112ae20746f5.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~43670] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.955002+0000", 2024-09-18T00:18:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0111580349999993, 2024-09-18T00:18:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055249590000000003, 2024-09-18T00:18:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 870 2024-09-18T00:18:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955002+0000", 2024-09-18T00:18:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955002+0000", 2024-09-18T00:18:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955004+0000", 2024-09-18T00:18:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0760000000000001e-06 2024-09-18T00:18:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955038+0000", 2024-09-18T00:18:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4008999999999999e-05 2024-09-18T00:18:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955053+0000", 2024-09-18T00:18:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5410999999999999e-05 2024-09-18T00:18:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955060+0000", 2024-09-18T00:18:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7340000000000002e-06 2024-09-18T00:18:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955139+0000", 2024-09-18T00:18:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8733999999999995e-05 2024-09-18T00:18:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955356+0000", 2024-09-18T00:18:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021757600000000001 2024-09-18T00:18:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955658+0000", 2024-09-18T00:18:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00030183500000000003 2024-09-18T00:18:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960123+0000", 2024-09-18T00:18:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0044650269999999999 2024-09-18T00:18:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960423+0000", 2024-09-18T00:18:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000300346 2024-09-18T00:18:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960476+0000", 2024-09-18T00:18:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.295e-05 2024-09-18T00:18:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960527+0000", 2024-09-18T00:18:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0260999999999999e-05 2024-09-18T00:18:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:1563 3.4 3:20d2ba28:::rbd_header.112ae20746f5:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T00:18:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:15.425150+0000", 2024-09-18T00:18:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.541009463, 2024-09-18T00:18:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0034638830000000001, 2024-09-18T00:18:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 1563 2024-09-18T00:18:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425150+0000", 2024-09-18T00:18:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425150+0000", 2024-09-18T00:18:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425152+0000", 2024-09-18T00:18:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.018e-06 2024-09-18T00:18:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425159+0000", 2024-09-18T00:18:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0319999999999998e-06 2024-09-18T00:18:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425177+0000", 2024-09-18T00:18:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7915999999999999e-05 2024-09-18T00:18:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425189+0000", 2024-09-18T00:18:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1280999999999999e-05 2024-09-18T00:18:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425275+0000", 2024-09-18T00:18:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6277000000000001e-05 2024-09-18T00:18:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425408+0000", 2024-09-18T00:18:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013278199999999999 2024-09-18T00:18:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.425923+0000", 2024-09-18T00:18:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00051550100000000002 2024-09-18T00:18:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.427067+0000", 2024-09-18T00:18:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001144396 2024-09-18T00:18:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.428466+0000", 2024-09-18T00:18:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0013989060000000001 2024-09-18T00:18:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.428547+0000", 2024-09-18T00:18:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0549000000000004e-05 2024-09-18T00:18:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.428614+0000", 2024-09-18T00:18:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7224999999999994e-05 2024-09-18T00:18:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2344 3.4 3:20ffa5cc:::rbd_data.112aa7ba4578.000000000000021d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.593153+0000", 2024-09-18T00:18:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.37300646399999998, 2024-09-18T00:18:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039942249999999997, 2024-09-18T00:18:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2344 2024-09-18T00:18:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593153+0000", 2024-09-18T00:18:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593153+0000", 2024-09-18T00:18:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593156+0000", 2024-09-18T00:18:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4789999999999999e-06 2024-09-18T00:18:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593162+0000", 2024-09-18T00:18:21.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0229999999999998e-06 2024-09-18T00:18:21.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593180+0000", 2024-09-18T00:18:21.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8297000000000001e-05 2024-09-18T00:18:21.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593191+0000", 2024-09-18T00:18:21.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1381e-05 2024-09-18T00:18:21.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593262+0000", 2024-09-18T00:18:21.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0366999999999999e-05 2024-09-18T00:18:21.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593344+0000", 2024-09-18T00:18:21.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2149000000000002e-05 2024-09-18T00:18:21.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:21.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593464+0000", 2024-09-18T00:18:21.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011951300000000001 2024-09-18T00:18:21.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:21.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.594137+0000", 2024-09-18T00:18:21.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00067384200000000002 2024-09-18T00:18:21.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:21.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.597072+0000", 2024-09-18T00:18:21.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002934557 2024-09-18T00:18:21.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.597113+0000", 2024-09-18T00:18:21.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.159e-05 2024-09-18T00:18:21.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.597148+0000", 2024-09-18T00:18:21.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4026999999999998e-05 2024-09-18T00:18:21.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.064 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.065 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2381 3.0 3:01782307:::rbd_data.112aa7ba4578.000000000000004f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:21.065 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.661880+0000", 2024-09-18T00:18:21.065 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.30427993199999998, 2024-09-18T00:18:21.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040755189999999997, 2024-09-18T00:18:21.065 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.065 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.065 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:21.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:21.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:21.065 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2381 2024-09-18T00:18:21.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.066 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.661880+0000", 2024-09-18T00:18:21.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.661880+0000", 2024-09-18T00:18:21.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.661881+0000", 2024-09-18T00:18:21.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2979999999999999e-06 2024-09-18T00:18:21.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.661888+0000", 2024-09-18T00:18:21.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5560000000000004e-06 2024-09-18T00:18:21.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.661907+0000", 2024-09-18T00:18:21.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8892000000000001e-05 2024-09-18T00:18:21.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.661915+0000", 2024-09-18T00:18:21.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8910000000000001e-06 2024-09-18T00:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.661974+0000", 2024-09-18T00:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8341000000000003e-05 2024-09-18T00:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662081+0000", 2024-09-18T00:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107503 2024-09-18T00:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:21.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662240+0000", 2024-09-18T00:18:21.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015828000000000001 2024-09-18T00:18:21.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:21.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.665803+0000", 2024-09-18T00:18:21.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035632939999999998 2024-09-18T00:18:21.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:21.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.665857+0000", 2024-09-18T00:18:21.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4200999999999998e-05 2024-09-18T00:18:21.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.665912+0000", 2024-09-18T00:18:21.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4642000000000003e-05 2024-09-18T00:18:21.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.665955+0000", 2024-09-18T00:18:21.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3621000000000001e-05 2024-09-18T00:18:21.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.070 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.070 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2383 3.0 3:01e056be:::rbd_data.112aa7ba4578.0000000000000051:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:21.070 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.664211+0000", 2024-09-18T00:18:21.071 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.30194838800000001, 2024-09-18T00:18:21.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003049029, 2024-09-18T00:18:21.071 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.071 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.071 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:21.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:21.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:21.071 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2383 2024-09-18T00:18:21.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.071 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664211+0000", 2024-09-18T00:18:21.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664211+0000", 2024-09-18T00:18:21.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664213+0000", 2024-09-18T00:18:21.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4479999999999999e-06 2024-09-18T00:18:21.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664226+0000", 2024-09-18T00:18:21.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3399e-05 2024-09-18T00:18:21.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664241+0000", 2024-09-18T00:18:21.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5264000000000002e-05 2024-09-18T00:18:21.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664250+0000", 2024-09-18T00:18:21.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6240000000000008e-06 2024-09-18T00:18:21.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664307+0000", 2024-09-18T00:18:21.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7260999999999999e-05 2024-09-18T00:18:21.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664396+0000", 2024-09-18T00:18:21.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9114000000000001e-05 2024-09-18T00:18:21.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:21.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664510+0000", 2024-09-18T00:18:21.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113416 2024-09-18T00:18:21.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:21.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.666032+0000", 2024-09-18T00:18:21.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0015221620000000001 2024-09-18T00:18:21.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:21.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.667193+0000", 2024-09-18T00:18:21.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011605529999999999 2024-09-18T00:18:21.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.667226+0000", 2024-09-18T00:18:21.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2960999999999998e-05 2024-09-18T00:18:21.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.667260+0000", 2024-09-18T00:18:21.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4826999999999997e-05 2024-09-18T00:18:21.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.076 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.077 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2418 3.1a e37/23 3:59a8b40d:::rbd_data.112aa7ba4578.00000000000001b0:head v 37'19, mlcod=36'17)", 2024-09-18T00:18:21.077 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.748732+0000", 2024-09-18T00:18:21.077 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.217427811, 2024-09-18T00:18:21.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038097109999999999, 2024-09-18T00:18:21.077 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.077 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.077 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.077 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.748732+0000", 2024-09-18T00:18:21.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.748732+0000", 2024-09-18T00:18:21.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.748734+0000", 2024-09-18T00:18:21.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0660000000000002e-06 2024-09-18T00:18:21.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.751085+0000", 2024-09-18T00:18:21.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0023505370000000002 2024-09-18T00:18:21.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.751106+0000", 2024-09-18T00:18:21.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1846000000000001e-05 2024-09-18T00:18:21.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.751118+0000", 2024-09-18T00:18:21.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1566000000000001e-05 2024-09-18T00:18:21.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.751206+0000", 2024-09-18T00:18:21.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7774000000000004e-05 2024-09-18T00:18:21.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.751237+0000", 2024-09-18T00:18:21.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1361e-05 2024-09-18T00:18:21.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.752491+0000", 2024-09-18T00:18:21.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0012539859999999999 2024-09-18T00:18:21.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.752542+0000", 2024-09-18T00:18:21.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0575000000000003e-05 2024-09-18T00:18:21.081 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.081 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.081 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.081 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.081 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.081 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:21.082 INFO:teuthology.orchestra.run.smithi097.stdout:773 trunc from 0xc18115c to 0x111567 2024-09-18T00:18:21.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:18:21.083 INFO:teuthology.orchestra.run.smithi097.stdout:774 write 0x56aa3bd thru 0x56b4103 (0x9d47 bytes) 2024-09-18T00:18:21.083 INFO:teuthology.orchestra.run.smithi097.stdout:775 read 0x1319f26 thru 0x132753b (0xd616 bytes) 2024-09-18T00:18:21.084 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:21.079+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:21.090 INFO:teuthology.orchestra.run.smithi097.stdout:776 trunc from 0x56b4104 to 0xe8f6a88 2024-09-18T00:18:21.104 INFO:teuthology.orchestra.run.smithi097.stdout:777 read 0x5452997 thru 0x5461cf1 (0xf35b bytes) 2024-09-18T00:18:21.104 INFO:teuthology.orchestra.run.smithi097.stdout:778 punch from 0x92b21a5 to 0x92bd237, (0xb092 bytes) 2024-09-18T00:18:21.104 INFO:teuthology.orchestra.run.smithi097.stdout:779 punch from 0xa854265 to 0xa85e6dd, (0xa478 bytes) 2024-09-18T00:18:21.104 INFO:teuthology.orchestra.run.smithi097.stdout:782 read 0x9be38da thru 0x9be76cd (0x3df4 bytes) 2024-09-18T00:18:21.113 INFO:teuthology.orchestra.run.smithi097.stdout:783 trunc from 0xe8f6a88 to 0xaacb611 2024-09-18T00:18:21.127 INFO:teuthology.orchestra.run.smithi097.stdout:785 punch from 0x72df88e to 0x72e810b, (0x887d bytes) 2024-09-18T00:18:21.127 INFO:teuthology.orchestra.run.smithi097.stdout:786 read 0x226ee7c thru 0x226efda (0x15f bytes) 2024-09-18T00:18:21.127 INFO:teuthology.orchestra.run.smithi097.stdout:787 read 0x7e28c52 thru 0x7e369fb (0xddaa bytes) 2024-09-18T00:18:21.128 INFO:teuthology.orchestra.run.smithi097.stdout:789 write 0x3db0909 thru 0x3dbc205 (0xb8fd bytes) 2024-09-18T00:18:21.142 INFO:teuthology.orchestra.run.smithi097.stdout:790 read 0x6469ff8 thru 0x64762fd (0xc306 bytes) 2024-09-18T00:18:21.158 INFO:teuthology.orchestra.run.smithi097.stdout:791 write 0xbef2430 thru 0xbef5522 (0x30f3 bytes) 2024-09-18T00:18:21.179 INFO:teuthology.orchestra.run.smithi097.stdout:793 write 0xc3d4a69 thru 0xc3df3bb (0xa953 bytes) 2024-09-18T00:18:21.183 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:21.179+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:21.190 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:21.190 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:21.207 INFO:teuthology.orchestra.run.smithi097.stdout:796 write 0xe5cba61 thru 0xe5d6204 (0xa7a4 bytes) 2024-09-18T00:18:21.219 INFO:teuthology.orchestra.run.smithi097.stdout:798 read 0xc3f2b7f thru 0xc3f33e1 (0x863 bytes) 2024-09-18T00:18:21.219 INFO:teuthology.orchestra.run.smithi097.stdout:799 writesame 0x1e3b6dd thru 0x1e423e1 data_size 0x15(0x6d05 bytes) 2024-09-18T00:18:21.221 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:21.221 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:21.222 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:21.230 INFO:teuthology.orchestra.run.smithi097.stdout:800 write 0x859191 thru 0x85e938 (0x57a8 bytes) 2024-09-18T00:18:21.236 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:18:21.239 INFO:teuthology.orchestra.run.smithi097.stdout:802 read 0x69d5fe7 thru 0x69deea9 (0x8ec3 bytes) 2024-09-18T00:18:21.239 INFO:teuthology.orchestra.run.smithi097.stdout:803 read 0x38a064a thru 0x38a8c21 (0x85d8 bytes) 2024-09-18T00:18:21.239 INFO:teuthology.orchestra.run.smithi097.stdout:804 read 0x7043399 thru 0x704dcc9 (0xa931 bytes) 2024-09-18T00:18:21.239 INFO:teuthology.orchestra.run.smithi097.stdout:809 read 0x7911f0c thru 0x791e7ea (0xc8df bytes) 2024-09-18T00:18:21.240 INFO:teuthology.orchestra.run.smithi097.stdout:811 read 0xe0b10c1 thru 0xe0b35d7 (0x2517 bytes) 2024-09-18T00:18:21.240 INFO:teuthology.orchestra.run.smithi097.stdout:812 write 0x66cf11 thru 0x66e8ae (0x199e bytes) 2024-09-18T00:18:21.246 INFO:teuthology.orchestra.run.smithi097.stdout:813 read 0xf1cbaa thru 0xf1d131 (0x588 bytes) 2024-09-18T00:18:21.262 INFO:teuthology.orchestra.run.smithi097.stdout:814 trunc from 0xe5d6205 to 0x3284198 2024-09-18T00:18:21.284 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:21.279+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:21.301 INFO:teuthology.orchestra.run.smithi097.stdout:815 writesame 0x47b8e98 thru 0x47bb655 data_size 0x1(0x27be bytes) 2024-09-18T00:18:21.301 INFO:teuthology.orchestra.run.smithi097.stdout:818 punch from 0x44a500 to 0x44c2c5, (0x1dc5 bytes) 2024-09-18T00:18:21.324 INFO:teuthology.orchestra.run.smithi097.stdout:819 write 0xbb6a68f thru 0xbb75e4d (0xb7bf bytes) 2024-09-18T00:18:21.334 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:21.335 INFO:teuthology.orchestra.run.smithi097.stdout:822 trunc from 0xbb75e4e to 0xd065a4b 2024-09-18T00:18:21.349 INFO:teuthology.orchestra.run.smithi097.stdout:823 punch from 0x7619f7f to 0x761e712, (0x4793 bytes) 2024-09-18T00:18:21.349 INFO:teuthology.orchestra.run.smithi097.stdout:824 write 0x39e0ab thru 0x39ea57 (0x9ad bytes) 2024-09-18T00:18:21.356 INFO:teuthology.orchestra.run.smithi097.stdout:826 read 0x9f585da thru 0x9f6656a (0xdf91 bytes) 2024-09-18T00:18:21.366 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:21.366 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:21.366 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:21.366 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:21.366 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:21.371 INFO:teuthology.orchestra.run.smithi097.stdout:829 writesame 0xdc3fb48 thru 0xdc4953a data_size 0x57(0x99f3 bytes) 2024-09-18T00:18:21.371 INFO:teuthology.orchestra.run.smithi097.stdout:830 write 0xa3631b1 thru 0xa367cb9 (0x4b09 bytes) 2024-09-18T00:18:21.380 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:18:21.380 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:21.380 INFO:teuthology.orchestra.run.smithi097.stdout:832 write 0x8df902b thru 0x8dfb689 (0x265f bytes) 2024-09-18T00:18:21.385 INFO:teuthology.orchestra.run.smithi097.stdout:833 punch from 0x9ff040c to 0x9ff7d47, (0x793b bytes) 2024-09-18T00:18:21.385 INFO:teuthology.orchestra.run.smithi097.stdout:834 punch from 0x47950a4 to 0x4795689, (0x5e5 bytes) 2024-09-18T00:18:21.385 INFO:teuthology.orchestra.run.smithi097.stdout:835 punch from 0x24c0e51 to 0x24c6c43, (0x5df2 bytes) 2024-09-18T00:18:21.385 INFO:teuthology.orchestra.run.smithi097.stdout:836 punch from 0xcac150b to 0xcace686, (0xd17b bytes) 2024-09-18T00:18:21.385 INFO:teuthology.orchestra.run.smithi097.stdout:837 read 0xa8d1303 thru 0xa8d62bc (0x4fba bytes) 2024-09-18T00:18:21.385 INFO:teuthology.orchestra.run.smithi097.stdout:839 write 0xb058f63 thru 0xb05d40b (0x44a9 bytes) 2024-09-18T00:18:21.386 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:21.379+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:21.391 INFO:teuthology.orchestra.run.smithi097.stdout:840 write 0x231c307 thru 0x23216ee (0x53e8 bytes) 2024-09-18T00:18:21.401 INFO:teuthology.orchestra.run.smithi097.stdout:841 trunc from 0xdc4953b to 0xe81b56e 2024-09-18T00:18:21.415 INFO:teuthology.orchestra.run.smithi097.stdout:842 write 0x7e14fd0 thru 0x7e16e7f (0x1eb0 bytes) 2024-09-18T00:18:21.422 INFO:teuthology.orchestra.run.smithi097.stdout:843 punch from 0x4838d35 to 0x4847bb0, (0xee7b bytes) 2024-09-18T00:18:21.422 INFO:teuthology.orchestra.run.smithi097.stdout:845 read 0x270a83b thru 0x2714fbf (0xa785 bytes) 2024-09-18T00:18:21.422 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:21.422 INFO:teuthology.orchestra.run.smithi097.stdout:848 write 0xe5c30dc thru 0xe5c7b7b (0x4aa0 bytes) 2024-09-18T00:18:21.430 INFO:teuthology.orchestra.run.smithi097.stdout:849 write 0x755d5e3 thru 0x756082b (0x3249 bytes) 2024-09-18T00:18:21.440 INFO:teuthology.orchestra.run.smithi097.stdout:850 write 0x1ae0b4c thru 0x1aedb4f (0xd004 bytes) 2024-09-18T00:18:21.452 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:21.452 INFO:teuthology.orchestra.run.smithi097.stdout:853 read 0x9cfc933 thru 0x9d074d9 (0xaba7 bytes) 2024-09-18T00:18:21.452 INFO:teuthology.orchestra.run.smithi097.stdout:854 read 0xbd0f23e thru 0xbd162e2 (0x70a5 bytes) 2024-09-18T00:18:21.452 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:21.467 INFO:teuthology.orchestra.run.smithi097.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T00:18:21.477 INFO:teuthology.orchestra.run.smithi097.stdout:857 trunc from 0xe81b56e to 0x2fbcb20 2024-09-18T00:18:21.485 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:21.479+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:21.487 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:18:21.495 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T00:18:21.503 INFO:teuthology.orchestra.run.smithi097.stdout:858 punch from 0x16a00b to 0x174954, (0xa949 bytes) 2024-09-18T00:18:21.521 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:21.521 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:21.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:21.522 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:21.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.522 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4263.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-18T00:18:21.522 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:55.481299+0000", 2024-09-18T00:18:21.522 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.038840255, 2024-09-18T00:18:21.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036304219999999999, 2024-09-18T00:18:21.522 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.522 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.522 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.522 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481299+0000", 2024-09-18T00:18:21.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481299+0000", 2024-09-18T00:18:21.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481300+0000", 2024-09-18T00:18:21.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3e-07 2024-09-18T00:18:21.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481307+0000", 2024-09-18T00:18:21.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4959999999999999e-06 2024-09-18T00:18:21.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481330+0000", 2024-09-18T00:18:21.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3363999999999999e-05 2024-09-18T00:18:21.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481347+0000", 2024-09-18T00:18:21.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6263e-05 2024-09-18T00:18:21.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481428+0000", 2024-09-18T00:18:21.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1612000000000004e-05 2024-09-18T00:18:21.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481495+0000", 2024-09-18T00:18:21.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6439999999999994e-05 2024-09-18T00:18:21.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.484828+0000", 2024-09-18T00:18:21.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033337140000000002 2024-09-18T00:18:21.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.484929+0000", 2024-09-18T00:18:21.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100803 2024-09-18T00:18:21.526 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.526 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.526 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.526 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:285 3.9 e28/23 3:94c9a3ca:::rbd_data.112a45c8e167.000000000000001c:head v 28'1, mlcod=0'0)", 2024-09-18T00:18:21.526 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.892623+0000", 2024-09-18T00:18:21.527 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.627515558000001, 2024-09-18T00:18:21.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040991170000000002, 2024-09-18T00:18:21.527 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.527 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.527 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.527 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892623+0000", 2024-09-18T00:18:21.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892623+0000", 2024-09-18T00:18:21.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892655+0000", 2024-09-18T00:18:21.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1377000000000001e-05 2024-09-18T00:18:21.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892819+0000", 2024-09-18T00:18:21.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000164275 2024-09-18T00:18:21.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892851+0000", 2024-09-18T00:18:21.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1696000000000003e-05 2024-09-18T00:18:21.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892873+0000", 2024-09-18T00:18:21.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.26e-05 2024-09-18T00:18:21.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892958+0000", 2024-09-18T00:18:21.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4245000000000004e-05 2024-09-18T00:18:21.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892975+0000", 2024-09-18T00:18:21.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7240000000000001e-05 2024-09-18T00:18:21.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.896666+0000", 2024-09-18T00:18:21.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003691083 2024-09-18T00:18:21.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.896723+0000", 2024-09-18T00:18:21.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6601000000000002e-05 2024-09-18T00:18:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:318 3.9 e28/23 3:945ebdd9:::rbd_data.112a45c8e167.0000000000000012:head v 28'3, mlcod=28'1)", 2024-09-18T00:18:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.996252+0000", 2024-09-18T00:18:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.523886715, 2024-09-18T00:18:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0024507719999999999, 2024-09-18T00:18:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.996252+0000", 2024-09-18T00:18:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.996252+0000", 2024-09-18T00:18:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.996254+0000", 2024-09-18T00:18:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6440000000000001e-06 2024-09-18T00:18:21.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.996280+0000", 2024-09-18T00:18:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6480000000000001e-05 2024-09-18T00:18:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.996299+0000", 2024-09-18T00:18:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.855e-05 2024-09-18T00:18:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.996308+0000", 2024-09-18T00:18:21.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7630000000000006e-06 2024-09-18T00:18:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.996404+0000", 2024-09-18T00:18:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6398999999999996e-05 2024-09-18T00:18:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.996446+0000", 2024-09-18T00:18:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1609000000000001e-05 2024-09-18T00:18:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.998595+0000", 2024-09-18T00:18:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002149134 2024-09-18T00:18:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.998703+0000", 2024-09-18T00:18:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108193 2024-09-18T00:18:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:459 3.9 e29/23 3:9496b897:::rbd_data.112ae20746f5.0000000000000161:head v 29'5, mlcod=28'3)", 2024-09-18T00:18:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.652427+0000", 2024-09-18T00:18:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.8677117760000002, 2024-09-18T00:18:21.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021551880000000002, 2024-09-18T00:18:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.652427+0000", 2024-09-18T00:18:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.652427+0000", 2024-09-18T00:18:21.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.652428+0000", 2024-09-18T00:18:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6399999999999998e-07 2024-09-18T00:18:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.652624+0000", 2024-09-18T00:18:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000196906 2024-09-18T00:18:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.652640+0000", 2024-09-18T00:18:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5406000000000001e-05 2024-09-18T00:18:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.652648+0000", 2024-09-18T00:18:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4419999999999998e-06 2024-09-18T00:18:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.652731+0000", 2024-09-18T00:18:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2610999999999999e-05 2024-09-18T00:18:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.652768+0000", 2024-09-18T00:18:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7347e-05 2024-09-18T00:18:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.654519+0000", 2024-09-18T00:18:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0017512680000000001 2024-09-18T00:18:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.654582+0000", 2024-09-18T00:18:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2943999999999999e-05 2024-09-18T00:18:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.539 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:568 3.13 e29/27 3:c87503f4:::rbd_data.112ae20746f5.00000000000001c5:head v 29'2, mlcod=0'0)", 2024-09-18T00:18:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.912514+0000", 2024-09-18T00:18:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.6076254250000002, 2024-09-18T00:18:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026220229999999998, 2024-09-18T00:18:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.912514+0000", 2024-09-18T00:18:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.912514+0000", 2024-09-18T00:18:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.912516+0000", 2024-09-18T00:18:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.08e-06 2024-09-18T00:18:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.912670+0000", 2024-09-18T00:18:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000154129 2024-09-18T00:18:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.912698+0000", 2024-09-18T00:18:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8549000000000001e-05 2024-09-18T00:18:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.912711+0000", 2024-09-18T00:18:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3077e-05 2024-09-18T00:18:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.912828+0000", 2024-09-18T00:18:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000116966 2024-09-18T00:18:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.912870+0000", 2024-09-18T00:18:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1896e-05 2024-09-18T00:18:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.915044+0000", 2024-09-18T00:18:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002173866 2024-09-18T00:18:21.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.915136+0000", 2024-09-18T00:18:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1459999999999995e-05 2024-09-18T00:18:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:692 3.0 e29/23 3:03fab4ad:::rbd_data.112ae20746f5.0000000000000057:head v 29'1, mlcod=0'0)", 2024-09-18T00:18:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.2813020700000006, 2024-09-18T00:18:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074841819999999998, 2024-09-18T00:18:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.544 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238838+0000", 2024-09-18T00:18:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.187e-06 2024-09-18T00:18:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238874+0000", 2024-09-18T00:18:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5380999999999999e-05 2024-09-18T00:18:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238893+0000", 2024-09-18T00:18:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999999e-05 2024-09-18T00:18:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238906+0000", 2024-09-18T00:18:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3011e-05 2024-09-18T00:18:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238982+0000", 2024-09-18T00:18:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6018000000000003e-05 2024-09-18T00:18:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.239006+0000", 2024-09-18T00:18:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3683e-05 2024-09-18T00:18:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246227+0000", 2024-09-18T00:18:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072210989999999999 2024-09-18T00:18:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246321+0000", 2024-09-18T00:18:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4263000000000002e-05 2024-09-18T00:18:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.548 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:746 3.0 e29/23 3:03157719:::rbd_data.112ae20746f5.0000000000000004:head v 29'2, mlcod=0'0)", 2024-09-18T00:18:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.392074+0000", 2024-09-18T00:18:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.1280645969999998, 2024-09-18T00:18:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033355659999999999, 2024-09-18T00:18:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392074+0000", 2024-09-18T00:18:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392074+0000", 2024-09-18T00:18:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392075+0000", 2024-09-18T00:18:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:18:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392101+0000", 2024-09-18T00:18:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6001999999999999e-05 2024-09-18T00:18:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392125+0000", 2024-09-18T00:18:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3771e-05 2024-09-18T00:18:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392138+0000", 2024-09-18T00:18:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3675e-05 2024-09-18T00:18:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392216+0000", 2024-09-18T00:18:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8054999999999997e-05 2024-09-18T00:18:21.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392245+0000", 2024-09-18T00:18:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8501999999999999e-05 2024-09-18T00:18:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395315+0000", 2024-09-18T00:18:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003069975 2024-09-18T00:18:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395410+0000", 2024-09-18T00:18:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5068000000000005e-05 2024-09-18T00:18:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:839 3.0 e30/23 3:0095abde:::rbd_data.112ae20746f5.0000000000000116:head v 30'4, mlcod=29'2)", 2024-09-18T00:18:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.774971+0000", 2024-09-18T00:18:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.7451682589999997, 2024-09-18T00:18:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00246956, 2024-09-18T00:18:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.553 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774971+0000", 2024-09-18T00:18:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774971+0000", 2024-09-18T00:18:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.774973+0000", 2024-09-18T00:18:21.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-18T00:18:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.775056+0000", 2024-09-18T00:18:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3228999999999999e-05 2024-09-18T00:18:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.775086+0000", 2024-09-18T00:18:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9836999999999999e-05 2024-09-18T00:18:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.775100+0000", 2024-09-18T00:18:21.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.456e-05 2024-09-18T00:18:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.775188+0000", 2024-09-18T00:18:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7698e-05 2024-09-18T00:18:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.775220+0000", 2024-09-18T00:18:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1495000000000003e-05 2024-09-18T00:18:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.777391+0000", 2024-09-18T00:18:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002170983 2024-09-18T00:18:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.777440+0000", 2024-09-18T00:18:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9804e-05 2024-09-18T00:18:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:870 3.0 e30/23 3:05335421:::rbd_data.112ae20746f5.0000000000000014:head v 30'5, mlcod=30'3)", 2024-09-18T00:18:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.955786+0000", 2024-09-18T00:18:21.557 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.5643531839999998, 2024-09-18T00:18:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0044642019999999996, 2024-09-18T00:18:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955786+0000", 2024-09-18T00:18:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955786+0000", 2024-09-18T00:18:21.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955787+0000", 2024-09-18T00:18:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.271e-06 2024-09-18T00:18:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955824+0000", 2024-09-18T00:18:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6668999999999997e-05 2024-09-18T00:18:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955840+0000", 2024-09-18T00:18:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5977999999999998e-05 2024-09-18T00:18:21.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955847+0000", 2024-09-18T00:18:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1760000000000004e-06 2024-09-18T00:18:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955929+0000", 2024-09-18T00:18:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1568000000000002e-05 2024-09-18T00:18:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955961+0000", 2024-09-18T00:18:21.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2873000000000002e-05 2024-09-18T00:18:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960199+0000", 2024-09-18T00:18:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042373990000000002 2024-09-18T00:18:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960250+0000", 2024-09-18T00:18:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1267999999999998e-05 2024-09-18T00:18:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:1563 3.4 e32/23 3:20d2ba28:::rbd_header.112ae20746f5:head v 32'87, mlcod=31'85)", 2024-09-18T00:18:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:15.426077+0000", 2024-09-18T00:18:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.0940615710000001, 2024-09-18T00:18:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021311279999999999, 2024-09-18T00:18:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.426077+0000", 2024-09-18T00:18:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.426077+0000", 2024-09-18T00:18:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.426078+0000", 2024-09-18T00:18:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.32e-07 2024-09-18T00:18:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.426085+0000", 2024-09-18T00:18:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.469e-06 2024-09-18T00:18:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.426109+0000", 2024-09-18T00:18:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.387e-05 2024-09-18T00:18:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.426120+0000", 2024-09-18T00:18:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1066999999999999e-05 2024-09-18T00:18:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.426235+0000", 2024-09-18T00:18:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000114502 2024-09-18T00:18:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.426269+0000", 2024-09-18T00:18:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4374e-05 2024-09-18T00:18:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.428164+0000", 2024-09-18T00:18:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018946309999999999 2024-09-18T00:18:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:15.428209+0000", 2024-09-18T00:18:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4783000000000001e-05 2024-09-18T00:18:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2344 3.4 e37/23 3:20ffa5cc:::rbd_data.112aa7ba4578.000000000000021d:head v 37'95, mlcod=36'93)", 2024-09-18T00:18:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.593553+0000", 2024-09-18T00:18:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.92658557799999997, 2024-09-18T00:18:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033906819999999999, 2024-09-18T00:18:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593553+0000", 2024-09-18T00:18:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593553+0000", 2024-09-18T00:18:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593555+0000", 2024-09-18T00:18:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.643e-06 2024-09-18T00:18:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593567+0000", 2024-09-18T00:18:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2289999999999999e-05 2024-09-18T00:18:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593616+0000", 2024-09-18T00:18:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8406999999999999e-05 2024-09-18T00:18:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593629+0000", 2024-09-18T00:18:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2774e-05 2024-09-18T00:18:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593696+0000", 2024-09-18T00:18:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7587000000000002e-05 2024-09-18T00:18:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593720+0000", 2024-09-18T00:18:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3638e-05 2024-09-18T00:18:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.596897+0000", 2024-09-18T00:18:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003176772 2024-09-18T00:18:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.596944+0000", 2024-09-18T00:18:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7571000000000002e-05 2024-09-18T00:18:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2381 3.0 e37/23 3:01782307:::rbd_data.112aa7ba4578.000000000000004f:head v 37'16, mlcod=35'14)", 2024-09-18T00:18:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.662312+0000", 2024-09-18T00:18:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.85782691799999999, 2024-09-18T00:18:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033464179999999999, 2024-09-18T00:18:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662312+0000", 2024-09-18T00:18:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662312+0000", 2024-09-18T00:18:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662313+0000", 2024-09-18T00:18:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1900000000000001e-07 2024-09-18T00:18:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662319+0000", 2024-09-18T00:18:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4670000000000001e-06 2024-09-18T00:18:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662330+0000", 2024-09-18T00:18:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.023e-05 2024-09-18T00:18:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662336+0000", 2024-09-18T00:18:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.691e-06 2024-09-18T00:18:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662427+0000", 2024-09-18T00:18:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0216000000000005e-05 2024-09-18T00:18:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662448+0000", 2024-09-18T00:18:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1795000000000001e-05 2024-09-18T00:18:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.665606+0000", 2024-09-18T00:18:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031574799999999998 2024-09-18T00:18:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.665659+0000", 2024-09-18T00:18:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2719999999999997e-05 2024-09-18T00:18:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2383 3.0 e37/23 3:01e056be:::rbd_data.112aa7ba4578.0000000000000051:head v 37'17, mlcod=35'14)", 2024-09-18T00:18:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.664593+0000", 2024-09-18T00:18:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.85554635199999995, 2024-09-18T00:18:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0024915670000000001, 2024-09-18T00:18:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664593+0000", 2024-09-18T00:18:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664593+0000", 2024-09-18T00:18:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664593+0000", 2024-09-18T00:18:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7499999999999999e-07 2024-09-18T00:18:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664600+0000", 2024-09-18T00:18:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2589999999999997e-06 2024-09-18T00:18:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664612+0000", 2024-09-18T00:18:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2274e-05 2024-09-18T00:18:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664619+0000", 2024-09-18T00:18:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4569999999999999e-06 2024-09-18T00:18:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664674+0000", 2024-09-18T00:18:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4561999999999997e-05 2024-09-18T00:18:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.664698+0000", 2024-09-18T00:18:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4457999999999999e-05 2024-09-18T00:18:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.667055+0000", 2024-09-18T00:18:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002356478 2024-09-18T00:18:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.667084+0000", 2024-09-18T00:18:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9303999999999999e-05 2024-09-18T00:18:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2455 3.9 e37/23 3:93e40d20:::rbd_data.112aa7ba4578.000000000000008a:head v 37'16, mlcod=36'14)", 2024-09-18T00:18:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.948469+0000", 2024-09-18T00:18:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.57167050500000005, 2024-09-18T00:18:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028601339999999999, 2024-09-18T00:18:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.948469+0000", 2024-09-18T00:18:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.948469+0000", 2024-09-18T00:18:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.948471+0000", 2024-09-18T00:18:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6129999999999998e-06 2024-09-18T00:18:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.948541+0000", 2024-09-18T00:18:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9832999999999994e-05 2024-09-18T00:18:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.948577+0000", 2024-09-18T00:18:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5849999999999997e-05 2024-09-18T00:18:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.948596+0000", 2024-09-18T00:18:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8802999999999999e-05 2024-09-18T00:18:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.948741+0000", 2024-09-18T00:18:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145618 2024-09-18T00:18:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.948773+0000", 2024-09-18T00:18:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1885999999999999e-05 2024-09-18T00:18:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.951234+0000", 2024-09-18T00:18:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002460552 2024-09-18T00:18:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.951329+0000", 2024-09-18T00:18:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4979e-05 2024-09-18T00:18:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2460 3.0 e37/23 3:016548b0:::rbd_data.112aa7ba4578.0000000000000018:head v 37'18, mlcod=37'16)", 2024-09-18T00:18:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.985197+0000", 2024-09-18T00:18:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.53494202899999999, 2024-09-18T00:18:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004264894, 2024-09-18T00:18:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985197+0000", 2024-09-18T00:18:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985197+0000", 2024-09-18T00:18:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985198+0000", 2024-09-18T00:18:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.342e-06 2024-09-18T00:18:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985210+0000", 2024-09-18T00:18:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1783999999999999e-05 2024-09-18T00:18:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985231+0000", 2024-09-18T00:18:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0769000000000001e-05 2024-09-18T00:18:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985241+0000", 2024-09-18T00:18:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0266e-05 2024-09-18T00:18:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985398+0000", 2024-09-18T00:18:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156701 2024-09-18T00:18:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985447+0000", 2024-09-18T00:18:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9391000000000002e-05 2024-09-18T00:18:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989422+0000", 2024-09-18T00:18:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039745470000000001 2024-09-18T00:18:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989462+0000", 2024-09-18T00:18:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0093999999999999e-05 2024-09-18T00:18:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2469 3.0 e37/23 3:02fdd8b9:::rbd_data.112aa7ba4578.0000000000000044:head v 37'19, mlcod=37'16)", 2024-09-18T00:18:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.989106+0000", 2024-09-18T00:18:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.53103281099999999, 2024-09-18T00:18:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004408843, 2024-09-18T00:18:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989106+0000", 2024-09-18T00:18:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989106+0000", 2024-09-18T00:18:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989107+0000", 2024-09-18T00:18:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.088e-06 2024-09-18T00:18:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989115+0000", 2024-09-18T00:18:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8590000000000005e-06 2024-09-18T00:18:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989126+0000", 2024-09-18T00:18:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1144e-05 2024-09-18T00:18:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989134+0000", 2024-09-18T00:18:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4229999999999999e-06 2024-09-18T00:18:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989215+0000", 2024-09-18T00:18:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1551000000000006e-05 2024-09-18T00:18:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989237+0000", 2024-09-18T00:18:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1832999999999999e-05 2024-09-18T00:18:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.993479+0000", 2024-09-18T00:18:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042420119999999999 2024-09-18T00:18:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.993515+0000", 2024-09-18T00:18:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5933000000000003e-05 2024-09-18T00:18:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2471 3.4 e37/23 3:26bbbe4b:::rbd_data.112aa7ba4578.000000000000004e:head v 37'98, mlcod=37'95)", 2024-09-18T00:18:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.989721+0000", 2024-09-18T00:18:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.53041791199999999, 2024-09-18T00:18:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043834369999999996, 2024-09-18T00:18:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989721+0000", 2024-09-18T00:18:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989721+0000", 2024-09-18T00:18:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989722+0000", 2024-09-18T00:18:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3300000000000001e-07 2024-09-18T00:18:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989728+0000", 2024-09-18T00:18:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:18:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989741+0000", 2024-09-18T00:18:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3444999999999999e-05 2024-09-18T00:18:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989750+0000", 2024-09-18T00:18:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7369999999999995e-06 2024-09-18T00:18:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989807+0000", 2024-09-18T00:18:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6971000000000001e-05 2024-09-18T00:18:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989826+0000", 2024-09-18T00:18:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8918000000000001e-05 2024-09-18T00:18:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994077+0000", 2024-09-18T00:18:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042507719999999999 2024-09-18T00:18:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994105+0000", 2024-09-18T00:18:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7885000000000002e-05 2024-09-18T00:18:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2472 3.0 e37/23 3:01782307:::rbd_data.112aa7ba4578.000000000000004f:head v 37'20, mlcod=37'17)", 2024-09-18T00:18:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.990421+0000", 2024-09-18T00:18:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.52971781799999995, 2024-09-18T00:18:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00366118, 2024-09-18T00:18:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990421+0000", 2024-09-18T00:18:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990421+0000", 2024-09-18T00:18:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990422+0000", 2024-09-18T00:18:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8599999999999998e-07 2024-09-18T00:18:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990428+0000", 2024-09-18T00:18:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.908e-06 2024-09-18T00:18:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990438+0000", 2024-09-18T00:18:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0244000000000001e-05 2024-09-18T00:18:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990445+0000", 2024-09-18T00:18:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7170000000000002e-06 2024-09-18T00:18:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990492+0000", 2024-09-18T00:18:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7644e-05 2024-09-18T00:18:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990513+0000", 2024-09-18T00:18:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.02e-05 2024-09-18T00:18:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994045+0000", 2024-09-18T00:18:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035321699999999998 2024-09-18T00:18:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994082+0000", 2024-09-18T00:18:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7611000000000003e-05 2024-09-18T00:18:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2474 3.0 e37/23 3:01e056be:::rbd_data.112aa7ba4578.0000000000000051:head v 37'21, mlcod=37'17)", 2024-09-18T00:18:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.991077+0000", 2024-09-18T00:18:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.52906167699999995, 2024-09-18T00:18:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0030287030000000002, 2024-09-18T00:18:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991077+0000", 2024-09-18T00:18:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991077+0000", 2024-09-18T00:18:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991078+0000", 2024-09-18T00:18:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0699999999999996e-07 2024-09-18T00:18:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991085+0000", 2024-09-18T00:18:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5339999999999996e-06 2024-09-18T00:18:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991098+0000", 2024-09-18T00:18:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3471e-05 2024-09-18T00:18:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991106+0000", 2024-09-18T00:18:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.311e-06 2024-09-18T00:18:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991152+0000", 2024-09-18T00:18:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6458999999999997e-05 2024-09-18T00:18:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991164+0000", 2024-09-18T00:18:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2448e-05 2024-09-18T00:18:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994085+0000", 2024-09-18T00:18:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029206150000000001 2024-09-18T00:18:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994106+0000", 2024-09-18T00:18:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0958000000000002e-05 2024-09-18T00:18:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2477 3.9 e37/23 3:93e40d20:::rbd_data.112aa7ba4578.000000000000008a:head v 37'17, mlcod=36'15)", 2024-09-18T00:18:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.991598+0000", 2024-09-18T00:18:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.52854094200000001, 2024-09-18T00:18:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025379970000000002, 2024-09-18T00:18:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991598+0000", 2024-09-18T00:18:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991598+0000", 2024-09-18T00:18:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991599+0000", 2024-09-18T00:18:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1999999999999999e-07 2024-09-18T00:18:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991603+0000", 2024-09-18T00:18:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3189999999999996e-06 2024-09-18T00:18:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991612+0000", 2024-09-18T00:18:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5799999999999992e-06 2024-09-18T00:18:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991617+0000", 2024-09-18T00:18:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1390000000000003e-06 2024-09-18T00:18:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991716+0000", 2024-09-18T00:18:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8759999999999996e-05 2024-09-18T00:18:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991752+0000", 2024-09-18T00:18:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6022999999999997e-05 2024-09-18T00:18:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994113+0000", 2024-09-18T00:18:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002361663 2024-09-18T00:18:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994136+0000", 2024-09-18T00:18:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2892999999999999e-05 2024-09-18T00:18:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:21.610 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:21.611 INFO:teuthology.orchestra.run.smithi097.stdout:859 writesame 0xa926ffb thru 0xa92df4c data_size 0x1(0x6f52 bytes) 2024-09-18T00:18:21.611 INFO:teuthology.orchestra.run.smithi097.stdout:861 read 0xa1b24dd thru 0xa1c0a33 (0xe557 bytes) 2024-09-18T00:18:21.611 INFO:teuthology.orchestra.run.smithi097.stdout:862 read 0x8adc391 thru 0x8adf949 (0x35b9 bytes) 2024-09-18T00:18:21.611 INFO:teuthology.orchestra.run.smithi097.stdout:863 writesame 0x2a437f0 thru 0x2a4eaf5 data_size 0x5(0xb306 bytes) 2024-09-18T00:18:21.612 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:18:21.613 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:21.614 INFO:teuthology.orchestra.run.smithi097.stdout:865 punch from 0x14b5d4e to 0x14bbfaf, (0x6261 bytes) 2024-09-18T00:18:21.614 INFO:teuthology.orchestra.run.smithi097.stdout:866 trunc from 0xa92df4d to 0x8c9e035 2024-09-18T00:18:21.614 INFO:teuthology.orchestra.run.smithi097.stdout:867 write 0xd3c1e4f thru 0xd3ced24 (0xced6 bytes) 2024-09-18T00:18:21.614 INFO:teuthology.orchestra.run.smithi097.stdout:868 punch from 0xab662d3 to 0xab6c4b6, (0x61e3 bytes) 2024-09-18T00:18:21.614 INFO:teuthology.orchestra.run.smithi097.stdout:869 write 0x68d38b3 thru 0x68dea09 (0xb157 bytes) 2024-09-18T00:18:21.614 INFO:teuthology.orchestra.run.smithi097.stdout:871 write 0xac6fffb thru 0xac77fac (0x7fb2 bytes) 2024-09-18T00:18:21.614 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:21.614 INFO:teuthology.orchestra.run.smithi097.stdout:876 read 0x572252d thru 0x572b492 (0x8f66 bytes) 2024-09-18T00:18:21.614 INFO:teuthology.orchestra.run.smithi097.stdout:877 punch from 0x6c46ac3 to 0x6c4a7de, (0x3d1b bytes) 2024-09-18T00:18:21.614 INFO:teuthology.orchestra.run.smithi097.stdout:878 punch from 0x68b203c to 0x68b474f, (0x2713 bytes) 2024-09-18T00:18:21.614 INFO:teuthology.orchestra.run.smithi097.stdout:879 read 0x215bdcb thru 0x2162bbc (0x6df2 bytes) 2024-09-18T00:18:21.614 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:21.614 INFO:teuthology.orchestra.run.smithi097.stdout:881 read 0x36c3b52 thru 0x36c6142 (0x25f1 bytes) 2024-09-18T00:18:21.614 INFO:teuthology.orchestra.run.smithi097.stdout:884 read 0xb206655 thru 0xb20b50e (0x4eba bytes) 2024-09-18T00:18:21.615 INFO:teuthology.orchestra.run.smithi097.stdout:885 punch from 0x335660b to 0x335ff4f, (0x9944 bytes) 2024-09-18T00:18:21.615 INFO:teuthology.orchestra.run.smithi097.stdout:886 read 0xa9b9566 thru 0xa9bc6ef (0x318a bytes) 2024-09-18T00:18:21.615 INFO:teuthology.orchestra.run.smithi097.stdout:887 trunc from 0xd3ced25 to 0xcf3adeb 2024-09-18T00:18:21.615 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:21.607+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:21.626 INFO:teuthology.orchestra.run.smithi097.stdout:888 write 0xcdce2ac thru 0xcdd43f5 (0x614a bytes) 2024-09-18T00:18:21.643 INFO:teuthology.orchestra.run.smithi097.stdout:889 trunc from 0xcf3adeb to 0x7d454e5 2024-09-18T00:18:21.666 INFO:teuthology.orchestra.run.smithi097.stdout:890 read 0x4593f8f thru 0x459e45a (0xa4cc bytes) 2024-09-18T00:18:21.666 INFO:teuthology.orchestra.run.smithi097.stdout:891 write 0x4d60acd thru 0x4d62896 (0x1dca bytes) 2024-09-18T00:18:21.673 INFO:teuthology.orchestra.run.smithi097.stdout:892 punch from 0x97a7e1 to 0x989e7e, (0xf69d bytes) 2024-09-18T00:18:21.673 INFO:teuthology.orchestra.run.smithi097.stdout:893 write 0x54eba66 thru 0x54f8eeb (0xd486 bytes) 2024-09-18T00:18:21.686 INFO:teuthology.orchestra.run.smithi097.stdout:895 read 0x565b499 thru 0x5669d35 (0xe89d bytes) 2024-09-18T00:18:21.708 INFO:teuthology.orchestra.run.smithi097.stdout:897 write 0xdca3ad9 thru 0xdca9e1e (0x6346 bytes) 2024-09-18T00:18:21.712 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:21.707+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:21.716 INFO:teuthology.orchestra.run.smithi097.stdout:898 writesame 0x85ba3ec thru 0x85beb93 data_size 0x4(0x47a8 bytes) 2024-09-18T00:18:21.728 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:21.728 INFO:teuthology.orchestra.run.smithi097.stdout:901 writesame 0x36c7e19 thru 0x36ca888 data_size 0x1c(0x2a70 bytes) 2024-09-18T00:18:21.746 INFO:teuthology.orchestra.run.smithi097.stdout:902 trunc from 0xdca9e1f to 0x849ec6c 2024-09-18T00:18:21.749 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:21.750 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:21.750 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:21.750 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:21.766 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:18:21.770 INFO:teuthology.orchestra.run.smithi097.stdout:903 writesame 0x1e2f7bb thru 0x1e388ce data_size 0x1e(0x9114 bytes) 2024-09-18T00:18:21.783 INFO:teuthology.orchestra.run.smithi097.stdout:905 writesame 0xeed026 thru 0xef44e0 data_size 0x7(0x74bb bytes) 2024-09-18T00:18:21.803 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:21.803 INFO:teuthology.orchestra.run.smithi097.stdout:907 read 0x3598e thru 0x3f720 (0x9d93 bytes) 2024-09-18T00:18:21.803 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:21.803 INFO:teuthology.orchestra.run.smithi097.stdout:909 punch from 0x66382ac to 0x664211a, (0x9e6e bytes) 2024-09-18T00:18:21.813 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:21.807+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:21.826 INFO:teuthology.orchestra.run.smithi097.stdout:910 write 0xc42f11e thru 0xc437050 (0x7f33 bytes) 2024-09-18T00:18:21.835 INFO:teuthology.orchestra.run.smithi097.stdout:911 write 0x468d238 thru 0x468f13f (0x1f08 bytes) 2024-09-18T00:18:21.842 INFO:teuthology.orchestra.run.smithi097.stdout:912 read 0xa4103f5 thru 0xa41c546 (0xc152 bytes) 2024-09-18T00:18:21.843 INFO:teuthology.orchestra.run.smithi097.stdout:913 read 0x679b234 thru 0x67a5ba7 (0xa974 bytes) 2024-09-18T00:18:21.843 INFO:teuthology.orchestra.run.smithi097.stdout:914 write 0x79c61dd thru 0x79ca45c (0x4280 bytes) 2024-09-18T00:18:21.853 INFO:teuthology.orchestra.run.smithi097.stdout:915 write 0x2cb7f3c thru 0x2cbf6b3 (0x7778 bytes) 2024-09-18T00:18:21.864 INFO:teuthology.orchestra.run.smithi097.stdout:916 writesame 0x4cc7492 thru 0x4ccc97d data_size 0x1(0x54ec bytes) 2024-09-18T00:18:21.903 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:21.903 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:21.903 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:21.903 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:21.904 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:21.914 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:21.911+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:21.918 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:18:21.953 INFO:teuthology.orchestra.run.smithi097.stdout:917 writesame 0x26cd79d thru 0x26d5627 data_size 0x37(0x7e8b bytes) 2024-09-18T00:18:21.965 INFO:teuthology.orchestra.run.smithi097.stdout:918 trunc from 0xc437051 to 0xc5481e8 2024-09-18T00:18:21.980 INFO:teuthology.orchestra.run.smithi097.stdout:919 write 0xbfe059a thru 0xbfee098 (0xdaff bytes) 2024-09-18T00:18:21.992 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:21.992 INFO:teuthology.orchestra.run.smithi097.stdout:923 read 0xbee0caa thru 0xbee84c8 (0x781f bytes) 2024-09-18T00:18:21.996 INFO:teuthology.orchestra.run.smithi097.stdout:925 trunc from 0xc5481e8 to 0x9f28e5e 2024-09-18T00:18:22.015 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:22.011+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:22.030 INFO:teuthology.orchestra.run.smithi097.stdout:926 trunc from 0x9f28e5e to 0x302da37 2024-09-18T00:18:22.086 INFO:teuthology.orchestra.run.smithi097.stdout:927 writesame 0x80711b0 thru 0x807fc9e data_size 0x1(0xeaef bytes) 2024-09-18T00:18:22.086 INFO:teuthology.orchestra.run.smithi097.stdout:928 read 0x2b75177 thru 0x2b76c32 (0x1abc bytes) 2024-09-18T00:18:22.086 INFO:teuthology.orchestra.run.smithi097.stdout:931 write 0x11b2f04 thru 0x11b6803 (0x3900 bytes) 2024-09-18T00:18:22.087 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:22.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:224 3.15 3:ae53dbea:::rbd_data.112a45c8e167.0000000000000014:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1837333~47262] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T00:18:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.778326+0000", 2024-09-18T00:18:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 11.30573618, 2024-09-18T00:18:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0073850679999999998, 2024-09-18T00:18:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:22.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 224 2024-09-18T00:18:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.778326+0000", 2024-09-18T00:18:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.778326+0000", 2024-09-18T00:18:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.778328+0000", 2024-09-18T00:18:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.973e-06 2024-09-18T00:18:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.778349+0000", 2024-09-18T00:18:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1437000000000002e-05 2024-09-18T00:18:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.778389+0000", 2024-09-18T00:18:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0136000000000003e-05 2024-09-18T00:18:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.778413+0000", 2024-09-18T00:18:22.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3923000000000001e-05 2024-09-18T00:18:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.778496+0000", 2024-09-18T00:18:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2415000000000003e-05 2024-09-18T00:18:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.778595+0000", 2024-09-18T00:18:22.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9831e-05 2024-09-18T00:18:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.778679+0000", 2024-09-18T00:18:22.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3117999999999999e-05 2024-09-18T00:18:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.781429+0000", 2024-09-18T00:18:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027506499999999999 2024-09-18T00:18:22.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.785645+0000", 2024-09-18T00:18:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042158040000000001 2024-09-18T00:18:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.785682+0000", 2024-09-18T00:18:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7044e-05 2024-09-18T00:18:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.785711+0000", 2024-09-18T00:18:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8736999999999999e-05 2024-09-18T00:18:22.096 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:484 3.b 3:d213724a:::rbd_data.112ae20746f5.000000000000001a:head [stat,write 2856~54588] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.695416+0000", 2024-09-18T00:18:22.097 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.388646264, 2024-09-18T00:18:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0051065809999999998, 2024-09-18T00:18:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:22.098 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 484 2024-09-18T00:18:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.695416+0000", 2024-09-18T00:18:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.695416+0000", 2024-09-18T00:18:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.695418+0000", 2024-09-18T00:18:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3199999999999998e-06 2024-09-18T00:18:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.695492+0000", 2024-09-18T00:18:22.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3959000000000001e-05 2024-09-18T00:18:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.695526+0000", 2024-09-18T00:18:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4212999999999999e-05 2024-09-18T00:18:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.695560+0000", 2024-09-18T00:18:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4279999999999997e-05 2024-09-18T00:18:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.695649+0000", 2024-09-18T00:18:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8599999999999999e-05 2024-09-18T00:18:22.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.695720+0000", 2024-09-18T00:18:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0877999999999995e-05 2024-09-18T00:18:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.695834+0000", 2024-09-18T00:18:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000114182 2024-09-18T00:18:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.699754+0000", 2024-09-18T00:18:22.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039203739999999999 2024-09-18T00:18:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.700442+0000", 2024-09-18T00:18:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00068720399999999996 2024-09-18T00:18:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.700486+0000", 2024-09-18T00:18:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3958999999999997e-05 2024-09-18T00:18:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.700522+0000", 2024-09-18T00:18:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6612000000000001e-05 2024-09-18T00:18:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:529 3.15 3:ac3cbbc6:::rbd_data.112ae20746f5.00000000000000c4:head [set-alloc-hint object_size 524288 write_size 524288,write 458503~55345] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.819485+0000", 2024-09-18T00:18:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.264576669, 2024-09-18T00:18:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093508949999999997, 2024-09-18T00:18:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.109 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 529 2024-09-18T00:18:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.819485+0000", 2024-09-18T00:18:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.819485+0000", 2024-09-18T00:18:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.819487+0000", 2024-09-18T00:18:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1169999999999998e-06 2024-09-18T00:18:22.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.819517+0000", 2024-09-18T00:18:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9459999999999999e-05 2024-09-18T00:18:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.819536+0000", 2024-09-18T00:18:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999999e-05 2024-09-18T00:18:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.819548+0000", 2024-09-18T00:18:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1141999999999999e-05 2024-09-18T00:18:22.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.819652+0000", 2024-09-18T00:18:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104845 2024-09-18T00:18:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.819851+0000", 2024-09-18T00:18:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199039 2024-09-18T00:18:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.820067+0000", 2024-09-18T00:18:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000215992 2024-09-18T00:18:22.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.824105+0000", 2024-09-18T00:18:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00403726 2024-09-18T00:18:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.828648+0000", 2024-09-18T00:18:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045436850000000004 2024-09-18T00:18:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.828754+0000", 2024-09-18T00:18:22.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105702 2024-09-18T00:18:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.828836+0000", 2024-09-18T00:18:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2113000000000005e-05 2024-09-18T00:18:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.119 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:554 3.8 3:14210294:::rbd_data.112ae20746f5.0000000000000133:head [set-alloc-hint object_size 524288 write_size 524288,write 496641~27647] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.878492+0000", 2024-09-18T00:18:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.205570197, 2024-09-18T00:18:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0058120619999999998, 2024-09-18T00:18:22.120 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 554 2024-09-18T00:18:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.121 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.878492+0000", 2024-09-18T00:18:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.878492+0000", 2024-09-18T00:18:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.878495+0000", 2024-09-18T00:18:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4029999999999999e-06 2024-09-18T00:18:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.878531+0000", 2024-09-18T00:18:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6075999999999998e-05 2024-09-18T00:18:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.878571+0000", 2024-09-18T00:18:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9446999999999997e-05 2024-09-18T00:18:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.878597+0000", 2024-09-18T00:18:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6129999999999999e-05 2024-09-18T00:18:22.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.878738+0000", 2024-09-18T00:18:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000141445 2024-09-18T00:18:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.878960+0000", 2024-09-18T00:18:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00022156199999999999 2024-09-18T00:18:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-18T00:18:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.879169+0000", 2024-09-18T00:18:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020968399999999999 2024-09-18T00:18:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.881715+0000", 2024-09-18T00:18:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025456950000000002 2024-09-18T00:18:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:22.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.883999+0000", 2024-09-18T00:18:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022841850000000002 2024-09-18T00:18:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.884168+0000", 2024-09-18T00:18:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000168383 2024-09-18T00:18:22.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.884304+0000", 2024-09-18T00:18:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000136052 2024-09-18T00:18:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:599 3.15 3:ac22c048:::rbd_data.112ae20746f5.0000000000000110:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.980706+0000", 2024-09-18T00:18:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.103356328, 2024-09-18T00:18:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017399257000000001, 2024-09-18T00:18:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.131 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 599 2024-09-18T00:18:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980706+0000", 2024-09-18T00:18:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980706+0000", 2024-09-18T00:18:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980708+0000", 2024-09-18T00:18:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5919999999999999e-06 2024-09-18T00:18:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980722+0000", 2024-09-18T00:18:22.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.364e-05 2024-09-18T00:18:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980752+0000", 2024-09-18T00:18:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9794999999999998e-05 2024-09-18T00:18:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980778+0000", 2024-09-18T00:18:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6494000000000001e-05 2024-09-18T00:18:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980885+0000", 2024-09-18T00:18:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107104 2024-09-18T00:18:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.981079+0000", 2024-09-18T00:18:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019375699999999999 2024-09-18T00:18:22.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.981866+0000", 2024-09-18T00:18:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00078716099999999998 2024-09-18T00:18:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.997852+0000", 2024-09-18T00:18:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015986305999999999 2024-09-18T00:18:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.997957+0000", 2024-09-18T00:18:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104085 2024-09-18T00:18:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.998037+0000", 2024-09-18T00:18:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0098000000000005e-05 2024-09-18T00:18:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.998105+0000", 2024-09-18T00:18:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8225000000000005e-05 2024-09-18T00:18:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:599 3.15 e29/27 ondisk, result = 0)", 2024-09-18T00:18:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.985755+0000", 2024-09-18T00:18:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.098307258, 2024-09-18T00:18:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012114039, 2024-09-18T00:18:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.143 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.985755+0000", 2024-09-18T00:18:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.985755+0000", 2024-09-18T00:18:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.985761+0000", 2024-09-18T00:18:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2879999999999998e-06 2024-09-18T00:18:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.985769+0000", 2024-09-18T00:18:22.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2150000000000001e-06 2024-09-18T00:18:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.985795+0000", 2024-09-18T00:18:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6203e-05 2024-09-18T00:18:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.985808+0000", 2024-09-18T00:18:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2904e-05 2024-09-18T00:18:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.997823+0000", 2024-09-18T00:18:22.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012014413 2024-09-18T00:18:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.997840+0000", 2024-09-18T00:18:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7073000000000001e-05 2024-09-18T00:18:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.997869+0000", 2024-09-18T00:18:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8943000000000001e-05 2024-09-18T00:18:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:640 3.7 3:e0810bc8:::rbd_data.112ae20746f5.0000000000000019:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.104726+0000", 2024-09-18T00:18:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.9793354999999995, 2024-09-18T00:18:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017067268, 2024-09-18T00:18:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.151 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 640 2024-09-18T00:18:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104726+0000", 2024-09-18T00:18:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104726+0000", 2024-09-18T00:18:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104728+0000", 2024-09-18T00:18:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.336e-06 2024-09-18T00:18:22.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104735+0000", 2024-09-18T00:18:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6149999999999999e-06 2024-09-18T00:18:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104753+0000", 2024-09-18T00:18:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7362000000000001e-05 2024-09-18T00:18:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104761+0000", 2024-09-18T00:18:22.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5609999999999999e-06 2024-09-18T00:18:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104842+0000", 2024-09-18T00:18:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0526999999999997e-05 2024-09-18T00:18:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104916+0000", 2024-09-18T00:18:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3724000000000001e-05 2024-09-18T00:18:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:22.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.106201+0000", 2024-09-18T00:18:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001285046 2024-09-18T00:18:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121677+0000", 2024-09-18T00:18:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01547665 2024-09-18T00:18:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121696+0000", 2024-09-18T00:18:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8304999999999998e-05 2024-09-18T00:18:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121744+0000", 2024-09-18T00:18:22.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8415000000000003e-05 2024-09-18T00:18:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121794+0000", 2024-09-18T00:18:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9727e-05 2024-09-18T00:18:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:640 3.7 e29/27 ondisk, result = 0)", 2024-09-18T00:18:22.161 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.110269+0000", 2024-09-18T00:18:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.9737925310000008, 2024-09-18T00:18:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011417017999999999, 2024-09-18T00:18:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.110269+0000", 2024-09-18T00:18:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.110269+0000", 2024-09-18T00:18:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.110270+0000", 2024-09-18T00:18:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9399999999999993e-07 2024-09-18T00:18:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.110274+0000", 2024-09-18T00:18:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7570000000000002e-06 2024-09-18T00:18:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.110289+0000", 2024-09-18T00:18:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4542000000000001e-05 2024-09-18T00:18:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.110299+0000", 2024-09-18T00:18:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0188e-05 2024-09-18T00:18:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121660+0000", 2024-09-18T00:18:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01136116 2024-09-18T00:18:22.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121670+0000", 2024-09-18T00:18:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0053e-05 2024-09-18T00:18:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121686+0000", 2024-09-18T00:18:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6324000000000002e-05 2024-09-18T00:18:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:763 3.7 3:e4fee1a8:::rbd_data.112ae20746f5.00000000000001cb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.458069+0000", 2024-09-18T00:18:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.6259932809999995, 2024-09-18T00:18:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0071133539999999997, 2024-09-18T00:18:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 763 2024-09-18T00:18:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.166 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458069+0000", 2024-09-18T00:18:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458069+0000", 2024-09-18T00:18:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458071+0000", 2024-09-18T00:18:22.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.441e-06 2024-09-18T00:18:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458081+0000", 2024-09-18T00:18:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8430000000000005e-06 2024-09-18T00:18:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458109+0000", 2024-09-18T00:18:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7954e-05 2024-09-18T00:18:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458121+0000", 2024-09-18T00:18:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.202e-05 2024-09-18T00:18:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458213+0000", 2024-09-18T00:18:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1811000000000006e-05 2024-09-18T00:18:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458301+0000", 2024-09-18T00:18:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7955999999999996e-05 2024-09-18T00:18:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.458414+0000", 2024-09-18T00:18:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113817 2024-09-18T00:18:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.459300+0000", 2024-09-18T00:18:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00088518600000000005 2024-09-18T00:18:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.465011+0000", 2024-09-18T00:18:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057109750000000001 2024-09-18T00:18:22.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.465112+0000", 2024-09-18T00:18:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101733 2024-09-18T00:18:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.465182+0000", 2024-09-18T00:18:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9617999999999997e-05 2024-09-18T00:18:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.171 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.2838952579999994, 2024-09-18T00:18:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:18:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:18:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:18:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:18:22.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:18:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:18:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:18:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:18:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:18:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:18:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:18:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:18:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:18:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:18:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:18:22.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:18:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:18:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:18:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:18:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:18:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:22.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:18:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:18:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:18:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:18:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:18:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:22.177 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.2676873850000003, 2024-09-18T00:18:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:18:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:22.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:18:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:18:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:18:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:18:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:18:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:18:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:18:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:18:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:18:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:18:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:18:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:18:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:18:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:18:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:891 3.15 3:afdffb8d:::rbd_data.112ae20746f5.0000000000000199:head [set-alloc-hint object_size 524288 write_size 524288,write 102199~62064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:13.018521+0000", 2024-09-18T00:18:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0655413140000007, 2024-09-18T00:18:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0096613600000000008, 2024-09-18T00:18:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:22.182 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 891 2024-09-18T00:18:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:13.018521+0000", 2024-09-18T00:18:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:13.018521+0000", 2024-09-18T00:18:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:13.018523+0000", 2024-09-18T00:18:22.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.176e-06 2024-09-18T00:18:22.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:13.018566+0000", 2024-09-18T00:18:22.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3619999999999999e-05 2024-09-18T00:18:22.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:13.018590+0000", 2024-09-18T00:18:22.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3637000000000001e-05 2024-09-18T00:18:22.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:13.018603+0000", 2024-09-18T00:18:22.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2713e-05 2024-09-18T00:18:22.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:13.018665+0000", 2024-09-18T00:18:22.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2220999999999999e-05 2024-09-18T00:18:22.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:13.018772+0000", 2024-09-18T00:18:22.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107297 2024-09-18T00:18:22.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:22.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:13.018868+0000", 2024-09-18T00:18:22.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5503999999999999e-05 2024-09-18T00:18:22.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:22.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:13.021981+0000", 2024-09-18T00:18:22.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031128029999999999 2024-09-18T00:18:22.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:13.028085+0000", 2024-09-18T00:18:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0061046659999999999 2024-09-18T00:18:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:13.028126+0000", 2024-09-18T00:18:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0361999999999997e-05 2024-09-18T00:18:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:13.028182+0000", 2024-09-18T00:18:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6360999999999998e-05 2024-09-18T00:18:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.187 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2208 3.15 e36/32 3:ad31ca9e:::rbd_data.112aa7ba4578.000000000000004b:head v 36'24, mlcod=35'22)", 2024-09-18T00:18:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.216243+0000", 2024-09-18T00:18:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.867819363, 2024-09-18T00:18:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074706039999999996, 2024-09-18T00:18:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.216243+0000", 2024-09-18T00:18:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.216243+0000", 2024-09-18T00:18:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.216244+0000", 2024-09-18T00:18:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1379999999999999e-06 2024-09-18T00:18:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.216603+0000", 2024-09-18T00:18:22.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035968799999999999 2024-09-18T00:18:22.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.216618+0000", 2024-09-18T00:18:22.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4123e-05 2024-09-18T00:18:22.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.216627+0000", 2024-09-18T00:18:22.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1290000000000002e-06 2024-09-18T00:18:22.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.216752+0000", 2024-09-18T00:18:22.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000125547 2024-09-18T00:18:22.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.216814+0000", 2024-09-18T00:18:22.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1705999999999995e-05 2024-09-18T00:18:22.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:22.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.223634+0000", 2024-09-18T00:18:22.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0068197570000000001 2024-09-18T00:18:22.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.223713+0000", 2024-09-18T00:18:22.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9516000000000002e-05 2024-09-18T00:18:22.192 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.192 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.192 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.193 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2258 3.2 3:47d2461c:::rbd_data.112aa7ba4578.00000000000002a9:head [set-alloc-hint object_size 262144 write_size 262144,write 229376~22805] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:22.193 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.405797+0000", 2024-09-18T00:18:22.193 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.6782652339999999, 2024-09-18T00:18:22.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0082914340000000003, 2024-09-18T00:18:22.193 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.193 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.193 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:22.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:22.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:22.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:22.193 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2258 2024-09-18T00:18:22.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.193 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.405797+0000", 2024-09-18T00:18:22.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.405797+0000", 2024-09-18T00:18:22.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.405800+0000", 2024-09-18T00:18:22.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7299999999999999e-06 2024-09-18T00:18:22.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.405867+0000", 2024-09-18T00:18:22.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6530000000000002e-05 2024-09-18T00:18:22.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.405912+0000", 2024-09-18T00:18:22.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5012000000000002e-05 2024-09-18T00:18:22.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.405926+0000", 2024-09-18T00:18:22.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3981999999999999e-05 2024-09-18T00:18:22.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.406018+0000", 2024-09-18T00:18:22.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1854999999999994e-05 2024-09-18T00:18:22.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:18:22.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.406026+0000", 2024-09-18T00:18:22.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2660000000000001e-06 2024-09-18T00:18:22.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.409330+0000", 2024-09-18T00:18:22.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033041590000000001 2024-09-18T00:18:22.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.409487+0000", 2024-09-18T00:18:22.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015645399999999999 2024-09-18T00:18:22.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:22.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.409655+0000", 2024-09-18T00:18:22.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016868800000000001 2024-09-18T00:18:22.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:22.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.413775+0000", 2024-09-18T00:18:22.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041197830000000001 2024-09-18T00:18:22.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:22.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.413938+0000", 2024-09-18T00:18:22.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000162566 2024-09-18T00:18:22.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:22.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.414034+0000", 2024-09-18T00:18:22.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6663999999999995e-05 2024-09-18T00:18:22.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.414088+0000", 2024-09-18T00:18:22.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3745000000000001e-05 2024-09-18T00:18:22.199 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.199 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.199 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.199 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2313 3.7 3:e32f6735:::rbd_data.112aa7ba4578.0000000000000004:head [stat,write 165426~31182] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:22.199 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.533139+0000", 2024-09-18T00:18:22.199 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.5509228390000001, 2024-09-18T00:18:22.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0071360199999999999, 2024-09-18T00:18:22.199 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.199 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.199 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:22.200 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:22.200 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:22.200 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:22.200 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2313 2024-09-18T00:18:22.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.200 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533139+0000", 2024-09-18T00:18:22.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533139+0000", 2024-09-18T00:18:22.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533141+0000", 2024-09-18T00:18:22.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8780000000000001e-06 2024-09-18T00:18:22.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533175+0000", 2024-09-18T00:18:22.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4211999999999997e-05 2024-09-18T00:18:22.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533192+0000", 2024-09-18T00:18:22.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6745e-05 2024-09-18T00:18:22.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533202+0000", 2024-09-18T00:18:22.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7759999999999992e-06 2024-09-18T00:18:22.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533278+0000", 2024-09-18T00:18:22.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6507000000000005e-05 2024-09-18T00:18:22.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533369+0000", 2024-09-18T00:18:22.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1112000000000005e-05 2024-09-18T00:18:22.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:22.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533471+0000", 2024-09-18T00:18:22.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101934 2024-09-18T00:18:22.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:22.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.535664+0000", 2024-09-18T00:18:22.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021927219999999998 2024-09-18T00:18:22.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:22.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.540008+0000", 2024-09-18T00:18:22.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043437329999999998 2024-09-18T00:18:22.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:22.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.540162+0000", 2024-09-18T00:18:22.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015423499999999999 2024-09-18T00:18:22.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.540275+0000", 2024-09-18T00:18:22.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011316599999999999 2024-09-18T00:18:22.205 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.205 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.205 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.205 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2314 3.1e 3:785f9b7b:::rbd_data.112aa7ba4578.0000000000000005:head [stat,write 163840~25964] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:22.205 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.533228+0000", 2024-09-18T00:18:22.205 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.55083379, 2024-09-18T00:18:22.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006776018, 2024-09-18T00:18:22.205 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.205 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.205 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:22.205 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:22.206 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:22.206 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:22.206 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2314 2024-09-18T00:18:22.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.206 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533228+0000", 2024-09-18T00:18:22.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533228+0000", 2024-09-18T00:18:22.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533229+0000", 2024-09-18T00:18:22.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0999999999999998e-07 2024-09-18T00:18:22.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533257+0000", 2024-09-18T00:18:22.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8742e-05 2024-09-18T00:18:22.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533266+0000", 2024-09-18T00:18:22.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0460000000000006e-06 2024-09-18T00:18:22.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533271+0000", 2024-09-18T00:18:22.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3940000000000004e-06 2024-09-18T00:18:22.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533376+0000", 2024-09-18T00:18:22.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105486 2024-09-18T00:18:22.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533532+0000", 2024-09-18T00:18:22.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000155215 2024-09-18T00:18:22.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 6", 2024-09-18T00:18:22.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.533765+0000", 2024-09-18T00:18:22.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023361200000000001 2024-09-18T00:18:22.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:22.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.537499+0000", 2024-09-18T00:18:22.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003733924 2024-09-18T00:18:22.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:22.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.539897+0000", 2024-09-18T00:18:22.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0023975680000000001 2024-09-18T00:18:22.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:22.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.539953+0000", 2024-09-18T00:18:22.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6292000000000002e-05 2024-09-18T00:18:22.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.540004+0000", 2024-09-18T00:18:22.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1128999999999998e-05 2024-09-18T00:18:22.211 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.211 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.211 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.211 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2345 3.7 3:e1b0bdb4:::rbd_data.112aa7ba4578.000000000000023d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:22.211 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.593177+0000", 2024-09-18T00:18:22.211 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.490884734, 2024-09-18T00:18:22.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.005742741, 2024-09-18T00:18:22.211 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.212 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.212 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:22.212 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:22.212 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:22.212 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:22.212 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2345 2024-09-18T00:18:22.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.212 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593177+0000", 2024-09-18T00:18:22.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593177+0000", 2024-09-18T00:18:22.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593178+0000", 2024-09-18T00:18:22.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.248e-06 2024-09-18T00:18:22.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593184+0000", 2024-09-18T00:18:22.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8239999999999996e-06 2024-09-18T00:18:22.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593199+0000", 2024-09-18T00:18:22.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4898e-05 2024-09-18T00:18:22.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593208+0000", 2024-09-18T00:18:22.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4309999999999994e-06 2024-09-18T00:18:22.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593271+0000", 2024-09-18T00:18:22.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3235e-05 2024-09-18T00:18:22.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593342+0000", 2024-09-18T00:18:22.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1557000000000006e-05 2024-09-18T00:18:22.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:22.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593505+0000", 2024-09-18T00:18:22.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000162146 2024-09-18T00:18:22.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:22.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.598333+0000", 2024-09-18T00:18:22.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0048284340000000004 2024-09-18T00:18:22.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:22.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.598815+0000", 2024-09-18T00:18:22.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048224800000000001 2024-09-18T00:18:22.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:22.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.598878+0000", 2024-09-18T00:18:22.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2847000000000003e-05 2024-09-18T00:18:22.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.598920+0000", 2024-09-18T00:18:22.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1872999999999997e-05 2024-09-18T00:18:22.217 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.217 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.217 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.217 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2468 3.8 3:125ee6f7:::rbd_data.112aa7ba4578.0000000000000043:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:22.217 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.987498+0000", 2024-09-18T00:18:22.217 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.096564249, 2024-09-18T00:18:22.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006557462, 2024-09-18T00:18:22.217 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.217 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.218 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:22.218 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:22.218 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:22.218 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:22.218 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2468 2024-09-18T00:18:22.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.218 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.987498+0000", 2024-09-18T00:18:22.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.987498+0000", 2024-09-18T00:18:22.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.987499+0000", 2024-09-18T00:18:22.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.181e-06 2024-09-18T00:18:22.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.987504+0000", 2024-09-18T00:18:22.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9130000000000002e-06 2024-09-18T00:18:22.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.987517+0000", 2024-09-18T00:18:22.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3206e-05 2024-09-18T00:18:22.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.987524+0000", 2024-09-18T00:18:22.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.267e-06 2024-09-18T00:18:22.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.987642+0000", 2024-09-18T00:18:22.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117395 2024-09-18T00:18:22.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.987769+0000", 2024-09-18T00:18:22.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000127613 2024-09-18T00:18:22.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-18T00:18:22.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988073+0000", 2024-09-18T00:18:22.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000303858 2024-09-18T00:18:22.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:22.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989478+0000", 2024-09-18T00:18:22.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0014052730000000001 2024-09-18T00:18:22.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:22.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.993964+0000", 2024-09-18T00:18:22.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0044857849999999999 2024-09-18T00:18:22.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:22.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994012+0000", 2024-09-18T00:18:22.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7874000000000003e-05 2024-09-18T00:18:22.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994055+0000", 2024-09-18T00:18:22.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3096999999999998e-05 2024-09-18T00:18:22.223 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.223 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.223 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.223 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2797 3.8 3:16afcbf5:::rbd_data.112aa7ba4578.0000000000000217:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:18:22.223 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:21.721166+0000", 2024-09-18T00:18:22.223 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.36289618299999998, 2024-09-18T00:18:22.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085902059999999995, 2024-09-18T00:18:22.223 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.223 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.223 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:22.223 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:22.224 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:22.224 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:22.224 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2797 2024-09-18T00:18:22.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.224 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.721166+0000", 2024-09-18T00:18:22.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.721166+0000", 2024-09-18T00:18:22.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.721168+0000", 2024-09-18T00:18:22.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1320000000000001e-06 2024-09-18T00:18:22.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.721177+0000", 2024-09-18T00:18:22.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0280000000000006e-06 2024-09-18T00:18:22.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.721198+0000", 2024-09-18T00:18:22.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0806e-05 2024-09-18T00:18:22.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.721214+0000", 2024-09-18T00:18:22.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6358000000000002e-05 2024-09-18T00:18:22.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.721301+0000", 2024-09-18T00:18:22.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6685000000000001e-05 2024-09-18T00:18:22.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.721447+0000", 2024-09-18T00:18:22.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014640399999999999 2024-09-18T00:18:22.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-18T00:18:22.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.722083+0000", 2024-09-18T00:18:22.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00063603300000000004 2024-09-18T00:18:22.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:22.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.729509+0000", 2024-09-18T00:18:22.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074262019999999998 2024-09-18T00:18:22.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:22.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.729582+0000", 2024-09-18T00:18:22.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2459000000000005e-05 2024-09-18T00:18:22.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:22.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.729650+0000", 2024-09-18T00:18:22.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8396e-05 2024-09-18T00:18:22.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.729756+0000", 2024-09-18T00:18:22.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105703 2024-09-18T00:18:22.229 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.229 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.229 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.229 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2797 3.8 e38/23 ondisk, result = 0)", 2024-09-18T00:18:22.229 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:21.724376+0000", 2024-09-18T00:18:22.229 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.35968582399999999, 2024-09-18T00:18:22.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0051491710000000001, 2024-09-18T00:18:22.229 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:22.230 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:22.230 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:22.230 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:22.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:22.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.724376+0000", 2024-09-18T00:18:22.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:22.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.724376+0000", 2024-09-18T00:18:22.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:22.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:22.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.724377+0000", 2024-09-18T00:18:22.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1659999999999999e-06 2024-09-18T00:18:22.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:22.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.724384+0000", 2024-09-18T00:18:22.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3840000000000002e-06 2024-09-18T00:18:22.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:22.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.724402+0000", 2024-09-18T00:18:22.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8572000000000001e-05 2024-09-18T00:18:22.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:22.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.724412+0000", 2024-09-18T00:18:22.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3840000000000003e-06 2024-09-18T00:18:22.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:22.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.729474+0000", 2024-09-18T00:18:22.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050628269999999998 2024-09-18T00:18:22.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:22.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.729496+0000", 2024-09-18T00:18:22.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1274e-05 2024-09-18T00:18:22.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:22.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:22.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:22.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.729525+0000", 2024-09-18T00:18:22.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9564e-05 2024-09-18T00:18:22.233 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.233 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.233 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.233 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:22.233 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:22.233 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:22.234 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:18:22.235 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.235 INFO:teuthology.orchestra.run.smithi097.stdout:933 write 0x2b39acd thru 0x2b43a01 (0x9f35 bytes) 2024-09-18T00:18:22.236 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:18:22.237 INFO:teuthology.orchestra.run.smithi097.stdout:934 writesame 0x2ff42d8 thru 0x2ffd85f data_size 0x84(0x9588 bytes) 2024-09-18T00:18:22.237 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.237 INFO:teuthology.orchestra.run.smithi097.stdout:936 trunc from 0x807fc9f to 0x5eaa75a 2024-09-18T00:18:22.237 INFO:teuthology.orchestra.run.smithi097.stdout:937 read 0x4f76507 thru 0x4f7b8a1 (0x539b bytes) 2024-09-18T00:18:22.237 INFO:teuthology.orchestra.run.smithi097.stdout:940 trunc from 0x5eaa75a to 0x3cd4f64 2024-09-18T00:18:22.238 INFO:teuthology.orchestra.run.smithi097.stdout:941 writesame 0x7e4ca53 thru 0x7e529b0 data_size 0x1a(0x5f5e bytes) 2024-09-18T00:18:22.238 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.238 INFO:teuthology.orchestra.run.smithi097.stdout:943 write 0xe908653 thru 0xe918619 (0xffc7 bytes) 2024-09-18T00:18:22.238 INFO:teuthology.orchestra.run.smithi097.stdout:945 punch from 0xa955952 to 0xa95bf95, (0x6643 bytes) 2024-09-18T00:18:22.238 INFO:teuthology.orchestra.run.smithi097.stdout:946 read 0xb4fc3bd thru 0xb50a4b9 (0xe0fd bytes) 2024-09-18T00:18:22.238 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.238 INFO:teuthology.orchestra.run.smithi097.stdout:948 read 0xb12c559 thru 0xb133d8d (0x7835 bytes) 2024-09-18T00:18:22.238 INFO:teuthology.orchestra.run.smithi097.stdout:949 write 0xafd26b7 thru 0xafdf295 (0xcbdf bytes) 2024-09-18T00:18:22.238 INFO:teuthology.orchestra.run.smithi097.stdout:950 read 0x9313dbf thru 0x931be7a (0x80bc bytes) 2024-09-18T00:18:22.238 INFO:teuthology.orchestra.run.smithi097.stdout:951 write 0x167771c thru 0x167da3d (0x6322 bytes) 2024-09-18T00:18:22.238 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:22.231+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:22.245 INFO:teuthology.orchestra.run.smithi097.stdout:954 read 0xa464902 thru 0xa4742af (0xf9ae bytes) 2024-09-18T00:18:22.245 INFO:teuthology.orchestra.run.smithi097.stdout:956 punch from 0x3972871 to 0x3981262, (0xe9f1 bytes) 2024-09-18T00:18:22.246 INFO:teuthology.orchestra.run.smithi097.stdout:958 write 0x72fa8e5 thru 0x72fccee (0x240a bytes) 2024-09-18T00:18:22.253 INFO:teuthology.orchestra.run.smithi097.stdout:960 write 0xf38a79 thru 0xf467a0 (0xdd28 bytes) 2024-09-18T00:18:22.260 INFO:teuthology.orchestra.run.smithi097.stdout:961 punch from 0x8f25233 to 0x8f2ce4c, (0x7c19 bytes) 2024-09-18T00:18:22.261 INFO:teuthology.orchestra.run.smithi097.stdout:962 write 0x6d5aa6c thru 0x6d621f9 (0x778e bytes) 2024-09-18T00:18:22.268 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.269 INFO:teuthology.orchestra.run.smithi097.stdout:966 writesame 0x98c20f7 thru 0x98d1769 data_size 0x1(0xf673 bytes) 2024-09-18T00:18:22.335 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:22.331+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:22.360 INFO:teuthology.orchestra.run.smithi097.stdout:967 write 0x3e64b51 thru 0x3e64cbd (0x16d bytes) 2024-09-18T00:18:22.365 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:22.365 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:22.365 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:18:22.365 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:22.366 INFO:teuthology.orchestra.run.smithi097.stdout:968 read 0x5354834 thru 0x53553e7 (0xbb4 bytes) 2024-09-18T00:18:22.366 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.366 INFO:teuthology.orchestra.run.smithi097.stdout:972 write 0xb441b17 thru 0xb446c36 (0x5120 bytes) 2024-09-18T00:18:22.371 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.371 INFO:teuthology.orchestra.run.smithi097.stdout:974 writesame 0x7be93f1 thru 0x7be9481 data_size 0x1(0x91 bytes) 2024-09-18T00:18:22.376 INFO:teuthology.orchestra.run.smithi097.stdout:975 write 0x633b141 thru 0x634429f (0x915f bytes) 2024-09-18T00:18:22.380 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:18:22.385 INFO:teuthology.orchestra.run.smithi097.stdout:977 read 0xa0f0c38 thru 0xa0fd60f (0xc9d8 bytes) 2024-09-18T00:18:22.385 INFO:teuthology.orchestra.run.smithi097.stdout:979 writesame 0x9c4547d thru 0x9c45f7c data_size 0x2c(0xb00 bytes) 2024-09-18T00:18:22.390 INFO:teuthology.orchestra.run.smithi097.stdout:980 punch from 0x8df599c to 0x8e00087, (0xa6eb bytes) 2024-09-18T00:18:22.390 INFO:teuthology.orchestra.run.smithi097.stdout:981 read 0x2d5f9ca thru 0x2d69139 (0x9770 bytes) 2024-09-18T00:18:22.390 INFO:teuthology.orchestra.run.smithi097.stdout:983 writesame 0xe5f4177 thru 0xe5fd6dc data_size 0xd(0x9566 bytes) 2024-09-18T00:18:22.401 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:22.401 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:22.406 INFO:teuthology.orchestra.run.smithi097.stdout:984 trunc from 0xe91861a to 0x97b1005 2024-09-18T00:18:22.420 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:18:22.421 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:18:22.423 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.433 INFO:teuthology.orchestra.run.smithi097.stdout:988 write 0xa4d12be thru 0xa4d6757 (0x549a bytes) 2024-09-18T00:18:22.436 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:22.431+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:22.437 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:22.438 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:22.446 INFO:teuthology.orchestra.run.smithi097.stdout:989 write 0xd156c13 thru 0xd161961 (0xad4f bytes) 2024-09-18T00:18:22.452 INFO:teuthology.orchestra.run.smithi097.stdout:990 write 0x834f9dd thru 0x8354ad6 (0x50fa bytes) 2024-09-18T00:18:22.455 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:22.455 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:22.458 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.462 INFO:teuthology.orchestra.run.smithi097.stdout:993 trunc from 0xd161962 to 0xa22345e 2024-09-18T00:18:22.475 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:22.475 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:22.478 INFO:teuthology.orchestra.run.smithi097.stdout:994 read 0x2a5ffaa thru 0x2a67148 (0x719f bytes) 2024-09-18T00:18:22.478 INFO:teuthology.orchestra.run.smithi097.stdout:995 read 0x7d3e33b thru 0x7d494f5 (0xb1bb bytes) 2024-09-18T00:18:22.479 INFO:teuthology.orchestra.run.smithi097.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T00:18:22.480 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is unset 2024-09-18T00:18:22.485 INFO:teuthology.orchestra.run.smithi097.stdout:997 trunc from 0xa22345e to 0x68fcc33 2024-09-18T00:18:22.502 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:18:22.502 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:18:22.508 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.508 INFO:teuthology.orchestra.run.smithi097.stdout:999 read 0x4c630df thru 0x4c6793f (0x4861 bytes) 2024-09-18T00:18:22.508 INFO:teuthology.orchestra.run.smithi097.stdout:1000 read 0x4f493b8 thru 0x4f4ed3a (0x5983 bytes) 2024-09-18T00:18:22.508 INFO:teuthology.orchestra.run.smithi097.stdout:1001 writesame 0x54cd67e thru 0x54cf76c data_size 0x1(0x20ef bytes) 2024-09-18T00:18:22.509 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T00:18:22.515 INFO:teuthology.orchestra.run.smithi097.stdout:1003 read 0x27d73cb thru 0x27da263 (0x2e99 bytes) 2024-09-18T00:18:22.526 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:22.533 INFO:teuthology.orchestra.run.smithi097.stdout:1004 write 0xc643464 thru 0xc645700 (0x229d bytes) 2024-09-18T00:18:22.537 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:22.531+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:22.537 INFO:teuthology.orchestra.run.smithi097.stdout:1005 read 0xb37d1d5 thru 0xb38536d (0x8199 bytes) 2024-09-18T00:18:22.538 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.538 INFO:teuthology.orchestra.run.smithi097.stdout:1007 writesame 0x5eb70e thru 0x5ec3b1 data_size 0x2(0xca4 bytes) 2024-09-18T00:18:22.542 INFO:teuthology.orchestra.run.smithi097.stdout:1008 write 0x9d64021 thru 0x9d65c43 (0x1c23 bytes) 2024-09-18T00:18:22.545 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:22.546 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:22.547 INFO:teuthology.orchestra.run.smithi097.stdout:1009 write 0x9f9e50 thru 0xa0682a (0xc9db bytes) 2024-09-18T00:18:22.555 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.555 INFO:teuthology.orchestra.run.smithi097.stdout:1011 write 0x7080b6 thru 0x716cf0 (0xec3b bytes) 2024-09-18T00:18:22.563 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.563 INFO:teuthology.orchestra.run.smithi097.stdout:1014 read 0xa0cb56e thru 0xa0cdd30 (0x27c3 bytes) 2024-09-18T00:18:22.563 INFO:teuthology.orchestra.run.smithi097.stdout:1015 punch from 0x2eb926c to 0x2ebb853, (0x25e7 bytes) 2024-09-18T00:18:22.564 INFO:teuthology.orchestra.run.smithi097.stdout:1016 read 0x7259ba5 thru 0x72606e5 (0x6b41 bytes) 2024-09-18T00:18:22.564 INFO:teuthology.orchestra.run.smithi097.stdout:1017 write 0xc1aff10 thru 0xc1b1e93 (0x1f84 bytes) 2024-09-18T00:18:22.565 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:22.565 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:22.568 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.581 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:18:22.581 INFO:teuthology.orchestra.run.smithi097.stdout:1021 write 0xe3846e5 thru 0xe38f339 (0xac55 bytes) 2024-09-18T00:18:22.594 INFO:teuthology.orchestra.run.smithi097.stdout:1023 write 0xe3da947 thru 0xe3e9a50 (0xf10a bytes) 2024-09-18T00:18:22.597 INFO:teuthology.orchestra.run.smithi097.stdout:1025 read 0x5c3d51f thru 0x5c47200 (0x9ce2 bytes) 2024-09-18T00:18:22.597 INFO:teuthology.orchestra.run.smithi097.stdout:1026 read 0x48edc78 thru 0x48f68e7 (0x8c70 bytes) 2024-09-18T00:18:22.597 INFO:teuthology.orchestra.run.smithi097.stdout:1027 punch from 0x8b12ef2 to 0x8b1a3aa, (0x74b8 bytes) 2024-09-18T00:18:22.597 INFO:teuthology.orchestra.run.smithi097.stdout:1028 write 0xb7fe6e6 thru 0xb80a34e (0xbc69 bytes) 2024-09-18T00:18:22.606 INFO:teuthology.orchestra.run.smithi097.stdout:1029 write 0x3099eff thru 0x30a28b3 (0x89b5 bytes) 2024-09-18T00:18:22.614 INFO:teuthology.orchestra.run.smithi097.stdout:1031 write 0x1cbfc0b thru 0x1cc04a0 (0x896 bytes) 2024-09-18T00:18:22.619 INFO:teuthology.orchestra.run.smithi097.stdout:1032 read 0x7125b21 thru 0x7133954 (0xde34 bytes) 2024-09-18T00:18:22.619 INFO:teuthology.orchestra.run.smithi097.stdout:1034 punch from 0xb748b65 to 0xb74f555, (0x69f0 bytes) 2024-09-18T00:18:22.620 INFO:teuthology.orchestra.run.smithi097.stdout:1035 write 0x68efbc6 thru 0x68fcc09 (0xd044 bytes) 2024-09-18T00:18:22.628 INFO:teuthology.orchestra.run.smithi097.stdout:1036 punch from 0x93baace to 0x93ca8b6, (0xfde8 bytes) 2024-09-18T00:18:22.629 INFO:teuthology.orchestra.run.smithi097.stdout:1037 write 0xd7cfffc thru 0xd7da53e (0xa543 bytes) 2024-09-18T00:18:22.637 INFO:teuthology.orchestra.run.smithi097.stdout:1039 punch from 0x3a7133 to 0x3aff12, (0x8ddf bytes) 2024-09-18T00:18:22.637 INFO:teuthology.orchestra.run.smithi097.stdout:1040 read 0xab57b6c thru 0xab62266 (0xa6fb bytes) 2024-09-18T00:18:22.637 INFO:teuthology.orchestra.run.smithi097.stdout:1041 read 0x1e14fea thru 0x1e1e4e3 (0x94fa bytes) 2024-09-18T00:18:22.638 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.658 INFO:teuthology.orchestra.run.smithi097.stdout:1043 trunc from 0xe3e9a51 to 0x20c4e7 2024-09-18T00:18:22.666 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:22.663+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:22.684 INFO:teuthology.orchestra.run.smithi097.stdout:1044 punch from 0x11f67b to 0x12a3d6, (0xad5b bytes) 2024-09-18T00:18:22.709 INFO:teuthology.orchestra.run.smithi097.stdout:1047 write 0xc12c418 thru 0xc137f37 (0xbb20 bytes) 2024-09-18T00:18:22.713 INFO:teuthology.orchestra.run.smithi097.stdout:1048 writesame 0x35042f6 thru 0x350c104 data_size 0x1f(0x7e0f bytes) 2024-09-18T00:18:22.718 INFO:teuthology.orchestra.run.smithi097.stdout:1049 write 0xaeac9f thru 0xaecbcf (0x1f31 bytes) 2024-09-18T00:18:22.722 INFO:teuthology.orchestra.run.smithi097.stdout:1050 write 0x4c3fb72 thru 0x4c44efc (0x538b bytes) 2024-09-18T00:18:22.735 INFO:teuthology.orchestra.run.smithi097.stdout:1051 write 0xccd8616 thru 0xcce3265 (0xac50 bytes) 2024-09-18T00:18:22.738 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:22.735+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:22.741 INFO:teuthology.orchestra.run.smithi097.stdout:1052 read 0x9604d56 thru 0x960dc6b (0x8f16 bytes) 2024-09-18T00:18:22.741 INFO:teuthology.orchestra.run.smithi097.stdout:1053 flatten 2024-09-18T00:18:22.749 INFO:teuthology.orchestra.run.smithi097.stdout:1054 punch from 0x47d9a17 to 0x47e9119, (0xf702 bytes) 2024-09-18T00:18:22.749 INFO:teuthology.orchestra.run.smithi097.stdout:1055 write 0x9df0780 thru 0x9df575f (0x4fe0 bytes) 2024-09-18T00:18:22.755 INFO:teuthology.orchestra.run.smithi097.stdout:1056 write 0x5138c24 thru 0x514072d (0x7b0a bytes) 2024-09-18T00:18:22.757 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:22.758 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:22.763 INFO:teuthology.orchestra.run.smithi097.stdout:1057 write 0x53e2856 thru 0x53ed4b0 (0xac5b bytes) 2024-09-18T00:18:22.773 INFO:teuthology.orchestra.run.smithi097.stdout:1058 punch from 0x86e08be to 0x86e0d0e, (0x450 bytes) 2024-09-18T00:18:22.774 INFO:teuthology.orchestra.run.smithi097.stdout:1059 writesame 0x21e4bac thru 0x21f4960 data_size 0x1(0xfdb5 bytes) 2024-09-18T00:18:22.774 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:18:22.839 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:22.835+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:22.857 INFO:teuthology.orchestra.run.smithi097.stdout:1060 read 0xae78fd9 thru 0xae79836 (0x85e bytes) 2024-09-18T00:18:22.857 INFO:teuthology.orchestra.run.smithi097.stdout:1061 write 0x4278940 thru 0x428090f (0x7fd0 bytes) 2024-09-18T00:18:22.866 INFO:teuthology.orchestra.run.smithi097.stdout:1062 punch from 0x41cd604 to 0x41d6757, (0x9153 bytes) 2024-09-18T00:18:22.866 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.867 INFO:teuthology.orchestra.run.smithi097.stdout:1064 read 0x312ae18 thru 0x31333c9 (0x85b2 bytes) 2024-09-18T00:18:22.867 INFO:teuthology.orchestra.run.smithi097.stdout:1065 punch from 0x3e808d4 to 0x3e8a7dc, (0x9f08 bytes) 2024-09-18T00:18:22.867 INFO:teuthology.orchestra.run.smithi097.stdout:1066 write 0x635c8ca thru 0x6364ea0 (0x85d7 bytes) 2024-09-18T00:18:22.873 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:22.874 INFO:teuthology.orchestra.run.smithi097.stdout:1068 write 0x456b137 thru 0x457777f (0xc649 bytes) 2024-09-18T00:18:22.881 INFO:teuthology.orchestra.run.smithi097.stdout:1069 writesame 0xa6dd856 thru 0xa6e8de9 data_size 0x2(0xb594 bytes) 2024-09-18T00:18:22.927 INFO:teuthology.orchestra.run.smithi097.stdout:1071 read 0x2ace071 thru 0x2ad9efe (0xbe8e bytes) 2024-09-18T00:18:22.927 INFO:teuthology.orchestra.run.smithi097.stdout:1074 write 0x19c024f thru 0x19c6e99 (0x6c4b bytes) 2024-09-18T00:18:22.936 INFO:teuthology.orchestra.run.smithi097.stdout:1075 write 0x927e6e6 thru 0x928548e (0x6da9 bytes) 2024-09-18T00:18:22.939 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:22.935+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:22.947 INFO:teuthology.orchestra.run.smithi097.stdout:1076 write 0x5631d70 thru 0x56366d6 (0x4967 bytes) 2024-09-18T00:18:22.949 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:22.950 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:22.955 INFO:teuthology.orchestra.run.smithi097.stdout:1078 write 0x8bd2ddb thru 0x8bda4fe (0x7724 bytes) 2024-09-18T00:18:22.965 INFO:teuthology.orchestra.run.smithi097.stdout:1079 write 0xcb0596a thru 0xcb0f2c1 (0x9958 bytes) 2024-09-18T00:18:22.966 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:18:22.976 INFO:teuthology.orchestra.run.smithi097.stdout:1080 read 0x96a49ed thru 0x96b065f (0xbc73 bytes) 2024-09-18T00:18:22.981 INFO:teuthology.orchestra.run.smithi097.stdout:1081 trunc from 0xcce3266 to 0xaf905f7 2024-09-18T00:18:23.001 INFO:teuthology.orchestra.run.smithi097.stdout:1082 write 0x93e78b8 thru 0x93e825f (0x9a8 bytes) 2024-09-18T00:18:23.023 INFO:teuthology.orchestra.run.smithi097.stdout:1083 write 0xdb2c92c thru 0xdb2f798 (0x2e6d bytes) 2024-09-18T00:18:23.028 INFO:teuthology.orchestra.run.smithi097.stdout:1084 read 0x1ba72d0 thru 0x1bb6741 (0xf472 bytes) 2024-09-18T00:18:23.029 INFO:teuthology.orchestra.run.smithi097.stdout:1085 read 0x578cdab thru 0x5790334 (0x358a bytes) 2024-09-18T00:18:23.029 INFO:teuthology.orchestra.run.smithi097.stdout:1086 writesame 0x50e359b thru 0x50f187f data_size 0x1(0xe2e5 bytes) 2024-09-18T00:18:23.040 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:23.039+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:23.119 INFO:teuthology.orchestra.run.smithi097.stdout:1087 writesame 0x306504a thru 0x306ac5f data_size 0x3(0x5c16 bytes) 2024-09-18T00:18:23.132 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:23.132 INFO:teuthology.orchestra.run.smithi097.stdout:1092 punch from 0xa2a777d to 0xa2af04b, (0x78ce bytes) 2024-09-18T00:18:23.141 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:23.139+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:23.142 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:23.142 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:23.149 INFO:teuthology.orchestra.run.smithi097.stdout:1093 trunc from 0xdb2f799 to 0x71880ef 2024-09-18T00:18:23.157 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:18:23.170 INFO:teuthology.orchestra.run.smithi097.stdout:1097 write 0x643d3b9 thru 0x6443a3b (0x6683 bytes) 2024-09-18T00:18:23.176 INFO:teuthology.orchestra.run.smithi097.stdout:1099 read 0x24a0750 thru 0x24a4497 (0x3d48 bytes) 2024-09-18T00:18:23.198 INFO:teuthology.orchestra.run.smithi097.stdout:1100 write 0xbd9e415 thru 0xbda19d9 (0x35c5 bytes) 2024-09-18T00:18:23.214 INFO:teuthology.orchestra.run.smithi097.stdout:1101 write 0xce8c66b thru 0xce900ab (0x3a41 bytes) 2024-09-18T00:18:23.221 INFO:teuthology.orchestra.run.smithi097.stdout:1102 writesame 0x5096374 thru 0x5098960 data_size 0x13(0x25ed bytes) 2024-09-18T00:18:23.224 INFO:teuthology.orchestra.run.smithi097.stdout:1103 write 0x85b344a thru 0x85b3f07 (0xabe bytes) 2024-09-18T00:18:23.227 INFO:teuthology.orchestra.run.smithi097.stdout:1104 write 0x9b4107a thru 0x9b466a8 (0x562f bytes) 2024-09-18T00:18:23.235 INFO:teuthology.orchestra.run.smithi097.stdout:1105 writesame 0x3a5d88f thru 0x3a5e61e data_size 0x10(0xd90 bytes) 2024-09-18T00:18:23.240 INFO:teuthology.orchestra.run.smithi097.stdout:1107 write 0x626eebb thru 0x627a4be (0xb604 bytes) 2024-09-18T00:18:23.241 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:23.235+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:23.251 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:23.251 INFO:teuthology.orchestra.run.smithi097.stdout:1111 write 0xa731ccd thru 0xa73cb0c (0xae40 bytes) 2024-09-18T00:18:23.260 INFO:teuthology.orchestra.run.smithi097.stdout:1112 write 0x57882c4 thru 0x579489f (0xc5dc bytes) 2024-09-18T00:18:23.272 INFO:teuthology.orchestra.run.smithi097.stdout:1114 clone 3 order 24 su 4194304 sc 11 2024-09-18T00:18:23.292 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:23.292 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:23.306 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:18:23.350 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:23.347+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:23.444 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:23.439+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:23.474 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:23.475 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:23.482 INFO:teuthology.orchestra.run.smithi097.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T00:18:23.489 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:18:23.510 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T00:18:23.544 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:23.539+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:23.616 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:23.616 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:23.630 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:18:23.645 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:23.643+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:23.745 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:23.743+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:23.748 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-18T00:18:23.797 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:23.797 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:23.811 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:18:23.846 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:23.843+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:23.937 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:23.937 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:23.947 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:23.943+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:23.951 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:18:24.048 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:24.043+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:24.071 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:24.072 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:24.085 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:18:24.164 INFO:teuthology.orchestra.run.smithi097.stdout:1115 writesame 0x87dc682 thru 0x87dee09 data_size 0x37(0x2788 bytes) 2024-09-18T00:18:24.174 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:24.171+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:24.184 INFO:teuthology.orchestra.run.smithi097.stdout:1116 trunc from 0xce900ac to 0xe408539 2024-09-18T00:18:24.194 INFO:teuthology.orchestra.run.smithi097.stdout:truncating to largest ever: 0xed473b9 2024-09-18T00:18:24.195 INFO:teuthology.orchestra.run.smithi097.stdout:1117 trunc from 0xe408539 to 0xed473b9 2024-09-18T00:18:24.206 INFO:teuthology.orchestra.run.smithi097.stdout:1118 punch from 0x175fca2 to 0x17660ca, (0x6428 bytes) 2024-09-18T00:18:24.206 INFO:teuthology.orchestra.run.smithi097.stdout:1119 punch from 0xdd9a336 to 0xdd9e89b, (0x4565 bytes) 2024-09-18T00:18:24.206 INFO:teuthology.orchestra.run.smithi097.stdout:1120 write 0xfe6b63 thru 0xfee427 (0x78c5 bytes) 2024-09-18T00:18:24.219 INFO:teuthology.orchestra.run.smithi097.stdout:1121 read 0xe736ecb thru 0xe740eef (0xa025 bytes) 2024-09-18T00:18:24.219 INFO:teuthology.orchestra.run.smithi097.stdout:1122 write 0x9729c6c thru 0x9736e06 (0xd19b bytes) 2024-09-18T00:18:24.232 INFO:teuthology.orchestra.run.smithi097.stdout:1123 writesame 0xe9c0f41 thru 0xe9c4e66 data_size 0x2(0x3f26 bytes) 2024-09-18T00:18:24.250 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:24.247+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:24.254 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:24.254 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:24.269 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:18:24.272 INFO:teuthology.orchestra.run.smithi097.stdout:1124 writesame 0x45e5d88 thru 0x45e80ab data_size 0x1a(0x2324 bytes) 2024-09-18T00:18:24.289 INFO:teuthology.orchestra.run.smithi097.stdout:1125 write 0x61f217f thru 0x62020eb (0xff6d bytes) 2024-09-18T00:18:24.314 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:24.314 INFO:teuthology.orchestra.run.smithi097.stdout:1128 write 0x9916fff thru 0x991cfb3 (0x5fb5 bytes) 2024-09-18T00:18:24.328 INFO:teuthology.orchestra.run.smithi097.stdout:1129 read 0xd4e4747 thru 0xd4e848e (0x3d48 bytes) 2024-09-18T00:18:24.328 INFO:teuthology.orchestra.run.smithi097.stdout:1130 read 0x46d1bd5 thru 0x46daa1e (0x8e4a bytes) 2024-09-18T00:18:24.329 INFO:teuthology.orchestra.run.smithi097.stdout:1131 write 0x61f738c thru 0x61f8e4a (0x1abf bytes) 2024-09-18T00:18:24.332 INFO:teuthology.orchestra.run.smithi097.stdout:1132 trunc from 0xed473b9 to 0xe9edbfe 2024-09-18T00:18:24.348 INFO:teuthology.orchestra.run.smithi097.stdout:1135 write 0x25b4c96 thru 0x25b7755 (0x2ac0 bytes) 2024-09-18T00:18:24.350 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:24.347+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:24.366 INFO:teuthology.orchestra.run.smithi097.stdout:1136 write 0x6f2d62 thru 0x701ff1 (0xf290 bytes) 2024-09-18T00:18:24.380 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:24.381 INFO:teuthology.orchestra.run.smithi097.stdout:1138 clone 4 order 22 su 65536 sc 12 2024-09-18T00:18:24.406 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:24.406 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:24.420 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:18:24.465 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:24.463+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:24.491 INFO:teuthology.orchestra.run.smithi097.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T00:18:24.508 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:18:24.518 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T00:18:24.553 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:24.547+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:24.565 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:18:24.594 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:24.595 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:24.613 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:18:24.623 INFO:teuthology.orchestra.run.smithi097.stdout:leaving image image_client.0-clone3 intact 2024-09-18T00:18:24.654 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:24.647+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:24.754 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:24.754 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:24.755 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:24.751+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:24.763 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:24.764 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:24.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:18:24.781 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:18:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:18:24.798 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:24.798 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:24.817 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:24.818 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:24.840 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:24.840 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:24.856 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:24.851+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:24.861 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:18:24.861 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:18:24.883 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:24.883 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:24.887 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-18T00:18:24.898 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:24.899 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:24.899 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:24.899 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:24.904 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:24.905 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:24.916 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:18:24.957 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:24.951+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:25.057 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:25.055+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:25.091 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:25.092 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:25.092 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:25.092 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:25.092 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:25.107 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:18:25.158 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:25.155+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:25.236 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:25.236 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:25.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:25.236 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:25.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.236 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:220 3.19 3:9e8c0c63:::rbd_data.112a45c8e167.0000000000000031:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T00:18:25.237 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.761391+0000", 2024-09-18T00:18:25.237 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.473764022999999, 2024-09-18T00:18:25.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0062183000000000004, 2024-09-18T00:18:25.237 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.237 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.237 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.237 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:25.237 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:25.237 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:25.237 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 220 2024-09-18T00:18:25.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.237 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761391+0000", 2024-09-18T00:18:25.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761391+0000", 2024-09-18T00:18:25.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761392+0000", 2024-09-18T00:18:25.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.838e-06 2024-09-18T00:18:25.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761407+0000", 2024-09-18T00:18:25.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4538999999999999e-05 2024-09-18T00:18:25.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761421+0000", 2024-09-18T00:18:25.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3896e-05 2024-09-18T00:18:25.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761427+0000", 2024-09-18T00:18:25.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2519999999999996e-06 2024-09-18T00:18:25.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761507+0000", 2024-09-18T00:18:25.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9909999999999999e-05 2024-09-18T00:18:25.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.761697+0000", 2024-09-18T00:18:25.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000189834 2024-09-18T00:18:25.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-18T00:18:25.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.762095+0000", 2024-09-18T00:18:25.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00039823799999999998 2024-09-18T00:18:25.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:25.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767289+0000", 2024-09-18T00:18:25.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0051936930000000001 2024-09-18T00:18:25.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:25.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767536+0000", 2024-09-18T00:18:25.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00024718899999999998 2024-09-18T00:18:25.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767573+0000", 2024-09-18T00:18:25.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6792000000000003e-05 2024-09-18T00:18:25.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.767609+0000", 2024-09-18T00:18:25.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6118999999999998e-05 2024-09-18T00:18:25.242 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.243 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.243 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.243 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:668 3.1a e29/23 3:59ee8e8b:::rbd_data.112ae20746f5.0000000000000031:head v 29'3, mlcod=25'1)", 2024-09-18T00:18:25.243 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.170930+0000", 2024-09-18T00:18:25.243 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.064224457, 2024-09-18T00:18:25.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052043310000000004, 2024-09-18T00:18:25.243 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.243 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.243 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.243 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.170930+0000", 2024-09-18T00:18:25.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.170930+0000", 2024-09-18T00:18:25.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.170932+0000", 2024-09-18T00:18:25.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4699999999999999e-06 2024-09-18T00:18:25.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171038+0000", 2024-09-18T00:18:25.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106205 2024-09-18T00:18:25.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171061+0000", 2024-09-18T00:18:25.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3329999999999999e-05 2024-09-18T00:18:25.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171072+0000", 2024-09-18T00:18:25.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0806e-05 2024-09-18T00:18:25.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171201+0000", 2024-09-18T00:18:25.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012925799999999999 2024-09-18T00:18:25.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.171239+0000", 2024-09-18T00:18:25.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7866e-05 2024-09-18T00:18:25.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.176016+0000", 2024-09-18T00:18:25.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047769930000000002 2024-09-18T00:18:25.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.176134+0000", 2024-09-18T00:18:25.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000118403 2024-09-18T00:18:25.247 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.247 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.247 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.247 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:692 3.0 3:03fab4ad:::rbd_data.112ae20746f5.0000000000000057:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:25.247 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.238016+0000", 2024-09-18T00:18:25.247 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.997138746999999, 2024-09-18T00:18:25.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0087112969999999998, 2024-09-18T00:18:25.248 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.248 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.248 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.248 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:25.248 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:25.248 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:25.248 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 692 2024-09-18T00:18:25.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.248 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238016+0000", 2024-09-18T00:18:25.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238016+0000", 2024-09-18T00:18:25.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238017+0000", 2024-09-18T00:18:25.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.257e-06 2024-09-18T00:18:25.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238023+0000", 2024-09-18T00:18:25.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5470000000000004e-06 2024-09-18T00:18:25.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238046+0000", 2024-09-18T00:18:25.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3088e-05 2024-09-18T00:18:25.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238054+0000", 2024-09-18T00:18:25.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3569999999999998e-06 2024-09-18T00:18:25.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238135+0000", 2024-09-18T00:18:25.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0814000000000002e-05 2024-09-18T00:18:25.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238331+0000", 2024-09-18T00:18:25.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019594 2024-09-18T00:18:25.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:25.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238606+0000", 2024-09-18T00:18:25.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027473200000000003 2024-09-18T00:18:25.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:25.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.240010+0000", 2024-09-18T00:18:25.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001404329 2024-09-18T00:18:25.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:25.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246563+0000", 2024-09-18T00:18:25.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0065535630000000001 2024-09-18T00:18:25.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246643+0000", 2024-09-18T00:18:25.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9468999999999994e-05 2024-09-18T00:18:25.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246727+0000", 2024-09-18T00:18:25.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4201000000000002e-05 2024-09-18T00:18:25.253 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.253 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.253 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.253 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:746 3.0 3:03157719:::rbd_data.112ae20746f5.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:25.253 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.389598+0000", 2024-09-18T00:18:25.253 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.84555698, 2024-09-18T00:18:25.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006052516, 2024-09-18T00:18:25.254 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.254 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.254 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.254 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:25.254 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:25.254 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:25.254 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 746 2024-09-18T00:18:25.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.254 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389598+0000", 2024-09-18T00:18:25.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389598+0000", 2024-09-18T00:18:25.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389599+0000", 2024-09-18T00:18:25.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.844e-06 2024-09-18T00:18:25.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389609+0000", 2024-09-18T00:18:25.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1039999999999996e-06 2024-09-18T00:18:25.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389623+0000", 2024-09-18T00:18:25.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4384e-05 2024-09-18T00:18:25.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389632+0000", 2024-09-18T00:18:25.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5589999999999994e-06 2024-09-18T00:18:25.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389715+0000", 2024-09-18T00:18:25.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2185e-05 2024-09-18T00:18:25.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.389923+0000", 2024-09-18T00:18:25.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000208214 2024-09-18T00:18:25.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:25.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.390263+0000", 2024-09-18T00:18:25.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033984 2024-09-18T00:18:25.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:25.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392457+0000", 2024-09-18T00:18:25.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021944180000000001 2024-09-18T00:18:25.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:25.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395565+0000", 2024-09-18T00:18:25.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031083450000000002 2024-09-18T00:18:25.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395616+0000", 2024-09-18T00:18:25.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0949000000000003e-05 2024-09-18T00:18:25.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395650+0000", 2024-09-18T00:18:25.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3674000000000003e-05 2024-09-18T00:18:25.259 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.259 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.259 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.259 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:870 3.0 3:05335421:::rbd_data.112ae20746f5.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~43670] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:25.259 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.955002+0000", 2024-09-18T00:18:25.259 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.280152822, 2024-09-18T00:18:25.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0055249590000000003, 2024-09-18T00:18:25.259 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.260 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.260 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:25.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:25.260 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:25.260 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 870 2024-09-18T00:18:25.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.260 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955002+0000", 2024-09-18T00:18:25.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955002+0000", 2024-09-18T00:18:25.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955004+0000", 2024-09-18T00:18:25.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0760000000000001e-06 2024-09-18T00:18:25.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955038+0000", 2024-09-18T00:18:25.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4008999999999999e-05 2024-09-18T00:18:25.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955053+0000", 2024-09-18T00:18:25.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5410999999999999e-05 2024-09-18T00:18:25.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955060+0000", 2024-09-18T00:18:25.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7340000000000002e-06 2024-09-18T00:18:25.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955139+0000", 2024-09-18T00:18:25.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8733999999999995e-05 2024-09-18T00:18:25.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955356+0000", 2024-09-18T00:18:25.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021757600000000001 2024-09-18T00:18:25.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:25.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955658+0000", 2024-09-18T00:18:25.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00030183500000000003 2024-09-18T00:18:25.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:25.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960123+0000", 2024-09-18T00:18:25.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0044650269999999999 2024-09-18T00:18:25.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:25.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960423+0000", 2024-09-18T00:18:25.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000300346 2024-09-18T00:18:25.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960476+0000", 2024-09-18T00:18:25.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.295e-05 2024-09-18T00:18:25.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960527+0000", 2024-09-18T00:18:25.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0260999999999999e-05 2024-09-18T00:18:25.265 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.265 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.265 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.265 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2460 3.0 3:016548b0:::rbd_data.112aa7ba4578.0000000000000018:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:25.265 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.984261+0000", 2024-09-18T00:18:25.266 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2508933129999997, 2024-09-18T00:18:25.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057333330000000002, 2024-09-18T00:18:25.266 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.266 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.266 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.266 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:25.266 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:25.266 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:25.266 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2460 2024-09-18T00:18:25.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.266 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984261+0000", 2024-09-18T00:18:25.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984261+0000", 2024-09-18T00:18:25.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984270+0000", 2024-09-18T00:18:25.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7139999999999999e-06 2024-09-18T00:18:25.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984290+0000", 2024-09-18T00:18:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0040000000000001e-05 2024-09-18T00:18:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984333+0000", 2024-09-18T00:18:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3381999999999999e-05 2024-09-18T00:18:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984382+0000", 2024-09-18T00:18:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9106e-05 2024-09-18T00:18:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984527+0000", 2024-09-18T00:18:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014420599999999999 2024-09-18T00:18:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984742+0000", 2024-09-18T00:18:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021512899999999999 2024-09-18T00:18:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984998+0000", 2024-09-18T00:18:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025634399999999998 2024-09-18T00:18:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.986049+0000", 2024-09-18T00:18:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0010510540000000001 2024-09-18T00:18:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989903+0000", 2024-09-18T00:18:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038535100000000001 2024-09-18T00:18:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989954+0000", 2024-09-18T00:18:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1217000000000001e-05 2024-09-18T00:18:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989995+0000", 2024-09-18T00:18:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0630999999999998e-05 2024-09-18T00:18:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2465 3.4 3:21aaa14b:::rbd_data.112aa7ba4578.000000000000003c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.984468+0000", 2024-09-18T00:18:25.271 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2506870350000003, 2024-09-18T00:18:25.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0058618359999999996, 2024-09-18T00:18:25.272 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.272 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.272 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.272 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:25.272 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:25.272 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:25.272 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2465 2024-09-18T00:18:25.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.272 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984468+0000", 2024-09-18T00:18:25.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984468+0000", 2024-09-18T00:18:25.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984470+0000", 2024-09-18T00:18:25.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1710000000000001e-06 2024-09-18T00:18:25.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984479+0000", 2024-09-18T00:18:25.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1430000000000004e-06 2024-09-18T00:18:25.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984503+0000", 2024-09-18T00:18:25.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3883000000000002e-05 2024-09-18T00:18:25.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984519+0000", 2024-09-18T00:18:25.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5886e-05 2024-09-18T00:18:25.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984700+0000", 2024-09-18T00:18:25.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000181678 2024-09-18T00:18:25.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.984890+0000", 2024-09-18T00:18:25.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018991900000000001 2024-09-18T00:18:25.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:25.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985149+0000", 2024-09-18T00:18:25.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025913800000000001 2024-09-18T00:18:25.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:25.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.986564+0000", 2024-09-18T00:18:25.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0014150569999999999 2024-09-18T00:18:25.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:25.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990108+0000", 2024-09-18T00:18:25.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035438459999999998 2024-09-18T00:18:25.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990231+0000", 2024-09-18T00:18:25.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012243000000000001 2024-09-18T00:18:25.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990329+0000", 2024-09-18T00:18:25.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8684999999999995e-05 2024-09-18T00:18:25.277 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.277 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.277 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.277 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2468 3.8 e37/23 3:125ee6f7:::rbd_data.112aa7ba4578.0000000000000043:head v 37'22, mlcod=37'20)", 2024-09-18T00:18:25.277 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.988238+0000", 2024-09-18T00:18:25.277 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2469162560000004, 2024-09-18T00:18:25.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0053209479999999998, 2024-09-18T00:18:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988238+0000", 2024-09-18T00:18:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988238+0000", 2024-09-18T00:18:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988239+0000", 2024-09-18T00:18:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9999999999999996e-07 2024-09-18T00:18:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988245+0000", 2024-09-18T00:18:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3860000000000004e-06 2024-09-18T00:18:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988260+0000", 2024-09-18T00:18:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4285e-05 2024-09-18T00:18:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988268+0000", 2024-09-18T00:18:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7299999999999994e-06 2024-09-18T00:18:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988358+0000", 2024-09-18T00:18:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9442000000000002e-05 2024-09-18T00:18:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988377+0000", 2024-09-18T00:18:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9173000000000001e-05 2024-09-18T00:18:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.993514+0000", 2024-09-18T00:18:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0051369079999999999 2024-09-18T00:18:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.993559+0000", 2024-09-18T00:18:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5223999999999999e-05 2024-09-18T00:18:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2469 3.0 3:02fdd8b9:::rbd_data.112aa7ba4578.0000000000000044:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.988650+0000", 2024-09-18T00:18:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2465048919999999, 2024-09-18T00:18:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0051769240000000003, 2024-09-18T00:18:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2469 2024-09-18T00:18:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988650+0000", 2024-09-18T00:18:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988650+0000", 2024-09-18T00:18:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988652+0000", 2024-09-18T00:18:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.255e-06 2024-09-18T00:18:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988660+0000", 2024-09-18T00:18:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3000000000000002e-06 2024-09-18T00:18:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988693+0000", 2024-09-18T00:18:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2453999999999998e-05 2024-09-18T00:18:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988710+0000", 2024-09-18T00:18:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7694e-05 2024-09-18T00:18:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988794+0000", 2024-09-18T00:18:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3663000000000002e-05 2024-09-18T00:18:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.988888+0000", 2024-09-18T00:18:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4442000000000001e-05 2024-09-18T00:18:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989018+0000", 2024-09-18T00:18:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012929200000000001 2024-09-18T00:18:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.993567+0000", 2024-09-18T00:18:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045491899999999998 2024-09-18T00:18:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.993738+0000", 2024-09-18T00:18:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000171106 2024-09-18T00:18:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.993789+0000", 2024-09-18T00:18:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1138999999999999e-05 2024-09-18T00:18:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.993827+0000", 2024-09-18T00:18:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7388999999999997e-05 2024-09-18T00:18:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2471 3.4 3:26bbbe4b:::rbd_data.112aa7ba4578.000000000000004e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.989237+0000", 2024-09-18T00:18:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2459178419999999, 2024-09-18T00:18:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0056500750000000001, 2024-09-18T00:18:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2471 2024-09-18T00:18:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989237+0000", 2024-09-18T00:18:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989237+0000", 2024-09-18T00:18:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989239+0000", 2024-09-18T00:18:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999998e-06 2024-09-18T00:18:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989248+0000", 2024-09-18T00:18:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7289999999999995e-06 2024-09-18T00:18:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989282+0000", 2024-09-18T00:18:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3525000000000002e-05 2024-09-18T00:18:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989300+0000", 2024-09-18T00:18:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8326e-05 2024-09-18T00:18:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989409+0000", 2024-09-18T00:18:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108649 2024-09-18T00:18:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989487+0000", 2024-09-18T00:18:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8422000000000002e-05 2024-09-18T00:18:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989596+0000", 2024-09-18T00:18:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108887 2024-09-18T00:18:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994288+0000", 2024-09-18T00:18:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0046922420000000001 2024-09-18T00:18:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994755+0000", 2024-09-18T00:18:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00046690799999999998 2024-09-18T00:18:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994821+0000", 2024-09-18T00:18:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6068000000000005e-05 2024-09-18T00:18:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994887+0000", 2024-09-18T00:18:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5364999999999995e-05 2024-09-18T00:18:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2472 3.0 3:01782307:::rbd_data.112aa7ba4578.000000000000004f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.989347+0000", 2024-09-18T00:18:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2458080169999999, 2024-09-18T00:18:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0054597259999999998, 2024-09-18T00:18:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2472 2024-09-18T00:18:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989347+0000", 2024-09-18T00:18:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989347+0000", 2024-09-18T00:18:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989348+0000", 2024-09-18T00:18:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.283e-06 2024-09-18T00:18:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989354+0000", 2024-09-18T00:18:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5060000000000001e-06 2024-09-18T00:18:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989374+0000", 2024-09-18T00:18:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9239e-05 2024-09-18T00:18:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989390+0000", 2024-09-18T00:18:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6089000000000001e-05 2024-09-18T00:18:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990025+0000", 2024-09-18T00:18:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00063553999999999998 2024-09-18T00:18:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990096+0000", 2024-09-18T00:18:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0437000000000003e-05 2024-09-18T00:18:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990189+0000", 2024-09-18T00:18:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3147999999999997e-05 2024-09-18T00:18:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994323+0000", 2024-09-18T00:18:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0041344249999999997 2024-09-18T00:18:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994753+0000", 2024-09-18T00:18:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00042959699999999999 2024-09-18T00:18:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994784+0000", 2024-09-18T00:18:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0836000000000001e-05 2024-09-18T00:18:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994806+0000", 2024-09-18T00:18:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2626e-05 2024-09-18T00:18:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2497 3.19 3:9b4c9be5:::rbd_data.112aa7ba4578.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 196608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:21.014364+0000", 2024-09-18T00:18:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2207907609999999, 2024-09-18T00:18:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0053683259999999997, 2024-09-18T00:18:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2497 2024-09-18T00:18:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.014364+0000", 2024-09-18T00:18:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.014364+0000", 2024-09-18T00:18:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.014366+0000", 2024-09-18T00:18:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8050000000000001e-06 2024-09-18T00:18:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.014392+0000", 2024-09-18T00:18:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6449e-05 2024-09-18T00:18:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.014417+0000", 2024-09-18T00:18:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4794e-05 2024-09-18T00:18:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.014446+0000", 2024-09-18T00:18:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8994000000000001e-05 2024-09-18T00:18:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.014511+0000", 2024-09-18T00:18:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5612999999999997e-05 2024-09-18T00:18:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.014597+0000", 2024-09-18T00:18:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5617000000000004e-05 2024-09-18T00:18:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-18T00:18:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.014705+0000", 2024-09-18T00:18:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010802399999999999 2024-09-18T00:18:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.016463+0000", 2024-09-18T00:18:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00175774 2024-09-18T00:18:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.019580+0000", 2024-09-18T00:18:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031169919999999999 2024-09-18T00:18:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.019669+0000", 2024-09-18T00:18:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9192999999999998e-05 2024-09-18T00:18:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:21.019732+0000", 2024-09-18T00:18:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3105e-05 2024-09-18T00:18:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2897 3.0 3:03db7e23:::rbd_data.112aa7ba4578.000000000000019e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:18:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.036205+0000", 2024-09-18T00:18:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.1989495090000002, 2024-09-18T00:18:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0071173479999999999, 2024-09-18T00:18:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2897 2024-09-18T00:18:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.036205+0000", 2024-09-18T00:18:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.036205+0000", 2024-09-18T00:18:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.036208+0000", 2024-09-18T00:18:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4959999999999999e-06 2024-09-18T00:18:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.036220+0000", 2024-09-18T00:18:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.272e-05 2024-09-18T00:18:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.036256+0000", 2024-09-18T00:18:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6037000000000001e-05 2024-09-18T00:18:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.036278+0000", 2024-09-18T00:18:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.194e-05 2024-09-18T00:18:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.037091+0000", 2024-09-18T00:18:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00081280700000000003 2024-09-18T00:18:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.037299+0000", 2024-09-18T00:18:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020797799999999999 2024-09-18T00:18:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.037607+0000", 2024-09-18T00:18:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00030767400000000002 2024-09-18T00:18:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.039241+0000", 2024-09-18T00:18:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0016341179999999999 2024-09-18T00:18:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.043096+0000", 2024-09-18T00:18:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00385549 2024-09-18T00:18:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.043208+0000", 2024-09-18T00:18:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011184900000000001 2024-09-18T00:18:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.043322+0000", 2024-09-18T00:18:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000114239 2024-09-18T00:18:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2973 3.4 3:23ae92e0:::rbd_data.112aa7ba4578.00000000000002c2:head [set-alloc-hint object_size 262144 write_size 262144,write 42679~22857] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:18:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.228564+0000", 2024-09-18T00:18:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.0065907470000002, 2024-09-18T00:18:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064905359999999999, 2024-09-18T00:18:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2973 2024-09-18T00:18:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.228564+0000", 2024-09-18T00:18:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.228564+0000", 2024-09-18T00:18:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.228567+0000", 2024-09-18T00:18:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7970000000000001e-06 2024-09-18T00:18:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.228635+0000", 2024-09-18T00:18:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8242000000000002e-05 2024-09-18T00:18:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.228675+0000", 2024-09-18T00:18:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9962999999999997e-05 2024-09-18T00:18:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.228694+0000", 2024-09-18T00:18:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9046999999999999e-05 2024-09-18T00:18:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.228873+0000", 2024-09-18T00:18:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017963199999999999 2024-09-18T00:18:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229123+0000", 2024-09-18T00:18:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000249931 2024-09-18T00:18:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229362+0000", 2024-09-18T00:18:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000238889 2024-09-18T00:18:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.234098+0000", 2024-09-18T00:18:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047359990000000003 2024-09-18T00:18:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.234845+0000", 2024-09-18T00:18:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00074637100000000001 2024-09-18T00:18:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.234933+0000", 2024-09-18T00:18:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8332999999999996e-05 2024-09-18T00:18:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.235054+0000", 2024-09-18T00:18:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000121332 2024-09-18T00:18:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2992 3.0 3:05277217:::rbd_data.112aa7ba4578.000000000000025e:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:18:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.273410+0000", 2024-09-18T00:18:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.9617443350000001, 2024-09-18T00:18:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077343064000000003, 2024-09-18T00:18:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2992 2024-09-18T00:18:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.273410+0000", 2024-09-18T00:18:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.273410+0000", 2024-09-18T00:18:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.273413+0000", 2024-09-18T00:18:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3429999999999998e-06 2024-09-18T00:18:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.273423+0000", 2024-09-18T00:18:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0899e-05 2024-09-18T00:18:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.273447+0000", 2024-09-18T00:18:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3567999999999999e-05 2024-09-18T00:18:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.273457+0000", 2024-09-18T00:18:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.747e-06 2024-09-18T00:18:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.273550+0000", 2024-09-18T00:18:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2941000000000003e-05 2024-09-18T00:18:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.273755+0000", 2024-09-18T00:18:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020479800000000001 2024-09-18T00:18:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.274579+0000", 2024-09-18T00:18:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00082421700000000003 2024-09-18T00:18:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.350553+0000", 2024-09-18T00:18:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075974122000000005 2024-09-18T00:18:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.350627+0000", 2024-09-18T00:18:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3751999999999995e-05 2024-09-18T00:18:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.350694+0000", 2024-09-18T00:18:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7490000000000006e-05 2024-09-18T00:18:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.350753+0000", 2024-09-18T00:18:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9187000000000001e-05 2024-09-18T00:18:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2993 3.0 3:02c916df:::rbd_data.112aa7ba4578.0000000000000262:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:18:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.273568+0000", 2024-09-18T00:18:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.9615868430000001, 2024-09-18T00:18:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.077290453999999995, 2024-09-18T00:18:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2993 2024-09-18T00:18:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.273568+0000", 2024-09-18T00:18:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.273568+0000", 2024-09-18T00:18:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.273569+0000", 2024-09-18T00:18:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9899999999999999e-07 2024-09-18T00:18:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.273588+0000", 2024-09-18T00:18:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9296999999999998e-05 2024-09-18T00:18:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.273598+0000", 2024-09-18T00:18:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.645e-06 2024-09-18T00:18:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.273603+0000", 2024-09-18T00:18:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7930000000000003e-06 2024-09-18T00:18:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.278648+0000", 2024-09-18T00:18:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050450549999999997 2024-09-18T00:18:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.278849+0000", 2024-09-18T00:18:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020084200000000001 2024-09-18T00:18:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.283363+0000", 2024-09-18T00:18:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045134490000000001 2024-09-18T00:18:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.350612+0000", 2024-09-18T00:18:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.067249081000000002 2024-09-18T00:18:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.350764+0000", 2024-09-18T00:18:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015174399999999999 2024-09-18T00:18:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.350819+0000", 2024-09-18T00:18:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5946999999999998e-05 2024-09-18T00:18:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.350858+0000", 2024-09-18T00:18:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8702000000000003e-05 2024-09-18T00:18:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2994 3.4 3:2620a86f:::rbd_data.112aa7ba4578.0000000000000263:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:18:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.275802+0000", 2024-09-18T00:18:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.959352483, 2024-09-18T00:18:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085605590999999995, 2024-09-18T00:18:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2994 2024-09-18T00:18:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.275802+0000", 2024-09-18T00:18:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.275802+0000", 2024-09-18T00:18:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.275803+0000", 2024-09-18T00:18:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2470000000000001e-06 2024-09-18T00:18:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.275811+0000", 2024-09-18T00:18:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3799999999999996e-06 2024-09-18T00:18:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.275829+0000", 2024-09-18T00:18:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7898e-05 2024-09-18T00:18:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.275837+0000", 2024-09-18T00:18:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2290000000000003e-06 2024-09-18T00:18:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.275913+0000", 2024-09-18T00:18:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5971999999999997e-05 2024-09-18T00:18:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.276039+0000", 2024-09-18T00:18:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012580499999999999 2024-09-18T00:18:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.281653+0000", 2024-09-18T00:18:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0056143629999999998 2024-09-18T00:18:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.361311+0000", 2024-09-18T00:18:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079657829999999999 2024-09-18T00:18:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.361324+0000", 2024-09-18T00:18:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3135999999999999e-05 2024-09-18T00:18:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.361367+0000", 2024-09-18T00:18:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2799999999999997e-05 2024-09-18T00:18:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.361408+0000", 2024-09-18T00:18:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0930999999999998e-05 2024-09-18T00:18:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2992 3.0 e38/23 ondisk, result = 0)", 2024-09-18T00:18:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.277593+0000", 2024-09-18T00:18:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.9575611569999998, 2024-09-18T00:18:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.072973866999999998, 2024-09-18T00:18:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.277593+0000", 2024-09-18T00:18:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.277593+0000", 2024-09-18T00:18:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.277595+0000", 2024-09-18T00:18:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.753e-06 2024-09-18T00:18:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.277600+0000", 2024-09-18T00:18:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3340000000000001e-06 2024-09-18T00:18:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.277617+0000", 2024-09-18T00:18:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6064000000000001e-05 2024-09-18T00:18:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.277624+0000", 2024-09-18T00:18:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3869999999999997e-06 2024-09-18T00:18:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.350530+0000", 2024-09-18T00:18:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.072905661999999996 2024-09-18T00:18:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.350543+0000", 2024-09-18T00:18:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3301e-05 2024-09-18T00:18:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.350567+0000", 2024-09-18T00:18:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4366e-05 2024-09-18T00:18:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2994 3.4 e38/23 ondisk, result = 0)", 2024-09-18T00:18:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.294411+0000", 2024-09-18T00:18:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.9407438400000001, 2024-09-18T00:18:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.066905920999999993, 2024-09-18T00:18:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.294411+0000", 2024-09-18T00:18:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.294411+0000", 2024-09-18T00:18:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.294412+0000", 2024-09-18T00:18:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.782e-06 2024-09-18T00:18:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.294418+0000", 2024-09-18T00:18:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4620000000000004e-06 2024-09-18T00:18:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.294436+0000", 2024-09-18T00:18:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8289000000000001e-05 2024-09-18T00:18:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.294446+0000", 2024-09-18T00:18:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4509999999999998e-06 2024-09-18T00:18:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.361296+0000", 2024-09-18T00:18:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.066850721000000002 2024-09-18T00:18:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.361304+0000", 2024-09-18T00:18:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9359999999999999e-06 2024-09-18T00:18:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.361317+0000", 2024-09-18T00:18:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2279999999999999e-05 2024-09-18T00:18:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2993 3.0 e38/23 ondisk, result = 0)", 2024-09-18T00:18:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.295125+0000", 2024-09-18T00:18:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.9400294009999999, 2024-09-18T00:18:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.055735139000000003, 2024-09-18T00:18:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.295125+0000", 2024-09-18T00:18:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.295125+0000", 2024-09-18T00:18:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.295126+0000", 2024-09-18T00:18:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2699999999999999e-07 2024-09-18T00:18:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.295128+0000", 2024-09-18T00:18:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7590000000000002e-06 2024-09-18T00:18:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.295137+0000", 2024-09-18T00:18:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4249999999999998e-06 2024-09-18T00:18:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.295144+0000", 2024-09-18T00:18:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6059999999999997e-06 2024-09-18T00:18:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.350594+0000", 2024-09-18T00:18:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.055449518000000003 2024-09-18T00:18:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.350605+0000", 2024-09-18T00:18:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.084e-05 2024-09-18T00:18:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.350860+0000", 2024-09-18T00:18:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025546400000000001 2024-09-18T00:18:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.349 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.349 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:25.350 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:18:25.351 INFO:teuthology.orchestra.run.smithi097.stdout:1139 read 0x583ab5a thru 0x58415b0 (0x6a57 bytes) 2024-09-18T00:18:25.351 INFO:teuthology.orchestra.run.smithi097.stdout:1141 punch from 0x2a6184e to 0x2a69b0f, (0x82c1 bytes) 2024-09-18T00:18:25.351 INFO:teuthology.orchestra.run.smithi097.stdout:1143 read 0xb394993 thru 0xb397ac2 (0x3130 bytes) 2024-09-18T00:18:25.351 INFO:teuthology.orchestra.run.smithi097.stdout:1144 trunc from 0xe9edbfe to 0x8b65f63 2024-09-18T00:18:25.351 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:25.347+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:25.370 INFO:teuthology.orchestra.run.smithi097.stdout:1146 read 0x532482f thru 0x5327fe5 (0x37b7 bytes) 2024-09-18T00:18:25.372 INFO:teuthology.orchestra.run.smithi097.stdout:1147 trunc from 0x8b65f63 to 0xa4bf465 2024-09-18T00:18:25.380 INFO:teuthology.orchestra.run.smithi097.stdout:1148 write 0x4232b2f thru 0x423cfbf (0xa491 bytes) 2024-09-18T00:18:25.417 INFO:teuthology.orchestra.run.smithi097.stdout:1149 writesame 0xe499014 thru 0xe4a41df data_size 0x6(0xb1cc bytes) 2024-09-18T00:18:25.417 INFO:teuthology.orchestra.run.smithi097.stdout:1150 read 0xd92bf5c thru 0xd937f6a (0xc00f bytes) 2024-09-18T00:18:25.417 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:25.417 INFO:teuthology.orchestra.run.smithi097.stdout:1152 write 0x6b9226c thru 0x6b9f045 (0xcdda bytes) 2024-09-18T00:18:25.420 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:25.420 INFO:teuthology.orchestra.run.smithi097.stdout:1154 write 0x2b32dae thru 0x2b3e37f (0xb5d2 bytes) 2024-09-18T00:18:25.448 INFO:teuthology.orchestra.run.smithi097.stdout:1155 trunc from 0xe4a41e0 to 0x4fbf5cb 2024-09-18T00:18:25.450 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:25.447+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:25.503 INFO:teuthology.orchestra.run.smithi097.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T00:18:25.505 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is unset 2024-09-18T00:18:25.511 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:25.511 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:25.511 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:25.511 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:25.533 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T00:18:25.534 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:18:25.537 INFO:teuthology.orchestra.run.smithi097.stdout:1157 read 0x3f7d542 thru 0x3f8aecf (0xd98e bytes) 2024-09-18T00:18:25.537 INFO:teuthology.orchestra.run.smithi097.stdout:1158 punch from 0x45679cd to 0x4574470, (0xcaa3 bytes) 2024-09-18T00:18:25.537 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:25.540 INFO:teuthology.orchestra.run.smithi097.stdout:1162 trunc from 0x4fbf5cb to 0x76c0091 2024-09-18T00:18:25.545 INFO:teuthology.orchestra.run.smithi097.stdout:1166 read 0xdb8224 thru 0xdc6398 (0xe175 bytes) 2024-09-18T00:18:25.552 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:25.547+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:25.555 INFO:teuthology.orchestra.run.smithi097.stdout:1167 trunc from 0x76c0091 to 0xeae473e 2024-09-18T00:18:25.561 INFO:teuthology.orchestra.run.smithi097.stdout:1168 writesame 0x90c09ff thru 0x90cd990 data_size 0x2(0xcf92 bytes) 2024-09-18T00:18:25.619 INFO:teuthology.orchestra.run.smithi097.stdout:1169 trunc from 0xeae473e to 0x9fccfc2 2024-09-18T00:18:25.634 INFO:teuthology.orchestra.run.smithi097.stdout:1170 writesame 0x5185c35 thru 0x5189ee5 data_size 0x15(0x42b1 bytes) 2024-09-18T00:18:25.637 INFO:teuthology.orchestra.run.smithi097.stdout:1171 write 0x40ef857 thru 0x40fb431 (0xbbdb bytes) 2024-09-18T00:18:25.646 INFO:teuthology.orchestra.run.smithi097.stdout:1172 writesame 0x8ba324e thru 0x8baf2e8 data_size 0x1(0xc09b bytes) 2024-09-18T00:18:25.653 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:25.651+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:25.653 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:25.653 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:25.653 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:25.653 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:25.654 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:25.671 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:18:25.712 INFO:teuthology.orchestra.run.smithi097.stdout:1173 write 0x4abe4d8 thru 0x4ac2620 (0x4149 bytes) 2024-09-18T00:18:25.715 INFO:teuthology.orchestra.run.smithi097.stdout:1174 write 0x4efdc8 thru 0x4f4c89 (0x4ec2 bytes) 2024-09-18T00:18:25.741 INFO:teuthology.orchestra.run.smithi097.stdout:1177 read 0x60072bd thru 0x6008c56 (0x199a bytes) 2024-09-18T00:18:25.741 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:25.742 INFO:teuthology.orchestra.run.smithi097.stdout:1180 read 0x6818fe3 thru 0x681da20 (0x4a3e bytes) 2024-09-18T00:18:25.742 INFO:teuthology.orchestra.run.smithi097.stdout:1181 read 0xbd0950 thru 0xbd531c (0x49cd bytes) 2024-09-18T00:18:25.742 INFO:teuthology.orchestra.run.smithi097.stdout:1182 writesame 0x780dd6a thru 0x781bd27 data_size 0xd(0xdfbe bytes) 2024-09-18T00:18:25.754 INFO:teuthology.orchestra.run.smithi097.stdout:1183 write 0x2f7861 thru 0x2fd888 (0x6028 bytes) 2024-09-18T00:18:25.754 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:25.751+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:25.792 INFO:teuthology.orchestra.run.smithi097.stdout:1184 punch from 0xc1e5f3 to 0xc28a3c, (0xa449 bytes) 2024-09-18T00:18:25.792 INFO:teuthology.orchestra.run.smithi097.stdout:1185 read 0x3d78b1e thru 0x3d878af (0xed92 bytes) 2024-09-18T00:18:25.793 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:25.793 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:25.798 INFO:teuthology.orchestra.run.smithi097.stdout:1189 trunc from 0x9fccfc2 to 0xc705f41 2024-09-18T00:18:25.810 INFO:teuthology.orchestra.run.smithi097.stdout:1190 write 0x1e0dedb thru 0x1e13c94 (0x5dba bytes) 2024-09-18T00:18:25.826 INFO:teuthology.orchestra.run.smithi097.stdout:1191 punch from 0x4e542fc to 0x4e54753, (0x457 bytes) 2024-09-18T00:18:25.826 INFO:teuthology.orchestra.run.smithi097.stdout:1192 write 0x3d4aa7b thru 0x3d53b44 (0x90ca bytes) 2024-09-18T00:18:25.830 INFO:teuthology.orchestra.run.smithi097.stdout:1193 punch from 0xa886cf6 to 0xa88fee3, (0x91ed bytes) 2024-09-18T00:18:25.830 INFO:teuthology.orchestra.run.smithi097.stdout:1194 write 0x7e8aa48 thru 0x7e96166 (0xb71f bytes) 2024-09-18T00:18:25.834 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:25.835 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:25.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:25.835 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:25.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.835 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4263.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-18T00:18:25.835 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:55.481299+0000", 2024-09-18T00:18:25.835 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 30.351462896000001, 2024-09-18T00:18:25.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036304219999999999, 2024-09-18T00:18:25.835 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.836 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.836 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.836 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481299+0000", 2024-09-18T00:18:25.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481299+0000", 2024-09-18T00:18:25.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481300+0000", 2024-09-18T00:18:25.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3e-07 2024-09-18T00:18:25.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481307+0000", 2024-09-18T00:18:25.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4959999999999999e-06 2024-09-18T00:18:25.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481330+0000", 2024-09-18T00:18:25.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3363999999999999e-05 2024-09-18T00:18:25.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481347+0000", 2024-09-18T00:18:25.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6263e-05 2024-09-18T00:18:25.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481428+0000", 2024-09-18T00:18:25.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1612000000000004e-05 2024-09-18T00:18:25.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481495+0000", 2024-09-18T00:18:25.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6439999999999994e-05 2024-09-18T00:18:25.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.484828+0000", 2024-09-18T00:18:25.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033337140000000002 2024-09-18T00:18:25.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.484929+0000", 2024-09-18T00:18:25.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100803 2024-09-18T00:18:25.840 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.840 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.840 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.840 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:285 3.9 e28/23 3:94c9a3ca:::rbd_data.112a45c8e167.000000000000001c:head v 28'1, mlcod=0'0)", 2024-09-18T00:18:25.840 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.892623+0000", 2024-09-18T00:18:25.840 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.940138199, 2024-09-18T00:18:25.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040991170000000002, 2024-09-18T00:18:25.840 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.840 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.840 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.841 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892623+0000", 2024-09-18T00:18:25.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892623+0000", 2024-09-18T00:18:25.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892655+0000", 2024-09-18T00:18:25.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1377000000000001e-05 2024-09-18T00:18:25.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892819+0000", 2024-09-18T00:18:25.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000164275 2024-09-18T00:18:25.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892851+0000", 2024-09-18T00:18:25.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1696000000000003e-05 2024-09-18T00:18:25.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892873+0000", 2024-09-18T00:18:25.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.26e-05 2024-09-18T00:18:25.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892958+0000", 2024-09-18T00:18:25.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4245000000000004e-05 2024-09-18T00:18:25.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892975+0000", 2024-09-18T00:18:25.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7240000000000001e-05 2024-09-18T00:18:25.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.896666+0000", 2024-09-18T00:18:25.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003691083 2024-09-18T00:18:25.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.896723+0000", 2024-09-18T00:18:25.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6601000000000002e-05 2024-09-18T00:18:25.845 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.845 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.845 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.845 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:692 3.0 e29/23 3:03fab4ad:::rbd_data.112ae20746f5.0000000000000057:head v 29'1, mlcod=0'0)", 2024-09-18T00:18:25.845 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:25.845 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.593924711, 2024-09-18T00:18:25.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074841819999999998, 2024-09-18T00:18:25.846 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.846 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.846 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.846 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:25.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:25.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238838+0000", 2024-09-18T00:18:25.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.187e-06 2024-09-18T00:18:25.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238874+0000", 2024-09-18T00:18:25.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5380999999999999e-05 2024-09-18T00:18:25.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238893+0000", 2024-09-18T00:18:25.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999999e-05 2024-09-18T00:18:25.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238906+0000", 2024-09-18T00:18:25.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3011e-05 2024-09-18T00:18:25.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238982+0000", 2024-09-18T00:18:25.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6018000000000003e-05 2024-09-18T00:18:25.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.239006+0000", 2024-09-18T00:18:25.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3683e-05 2024-09-18T00:18:25.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246227+0000", 2024-09-18T00:18:25.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072210989999999999 2024-09-18T00:18:25.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246321+0000", 2024-09-18T00:18:25.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4263000000000002e-05 2024-09-18T00:18:25.849 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.849 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.849 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.850 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:746 3.0 e29/23 3:03157719:::rbd_data.112ae20746f5.0000000000000004:head v 29'2, mlcod=0'0)", 2024-09-18T00:18:25.850 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.392074+0000", 2024-09-18T00:18:25.850 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.440687238000001, 2024-09-18T00:18:25.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033355659999999999, 2024-09-18T00:18:25.850 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.850 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.850 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.850 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392074+0000", 2024-09-18T00:18:25.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392074+0000", 2024-09-18T00:18:25.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392075+0000", 2024-09-18T00:18:25.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:18:25.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392101+0000", 2024-09-18T00:18:25.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6001999999999999e-05 2024-09-18T00:18:25.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392125+0000", 2024-09-18T00:18:25.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3771e-05 2024-09-18T00:18:25.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392138+0000", 2024-09-18T00:18:25.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3675e-05 2024-09-18T00:18:25.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392216+0000", 2024-09-18T00:18:25.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8054999999999997e-05 2024-09-18T00:18:25.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392245+0000", 2024-09-18T00:18:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8501999999999999e-05 2024-09-18T00:18:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395315+0000", 2024-09-18T00:18:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003069975 2024-09-18T00:18:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395410+0000", 2024-09-18T00:18:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5068000000000005e-05 2024-09-18T00:18:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.854 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.855 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:870 3.0 e30/23 3:05335421:::rbd_data.112ae20746f5.0000000000000014:head v 30'5, mlcod=30'3)", 2024-09-18T00:18:25.855 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.955786+0000", 2024-09-18T00:18:25.855 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.876975825000001, 2024-09-18T00:18:25.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0044642019999999996, 2024-09-18T00:18:25.855 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.855 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.855 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.855 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955786+0000", 2024-09-18T00:18:25.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955786+0000", 2024-09-18T00:18:25.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955787+0000", 2024-09-18T00:18:25.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.271e-06 2024-09-18T00:18:25.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955824+0000", 2024-09-18T00:18:25.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6668999999999997e-05 2024-09-18T00:18:25.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955840+0000", 2024-09-18T00:18:25.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5977999999999998e-05 2024-09-18T00:18:25.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955847+0000", 2024-09-18T00:18:25.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1760000000000004e-06 2024-09-18T00:18:25.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955929+0000", 2024-09-18T00:18:25.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1568000000000002e-05 2024-09-18T00:18:25.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955961+0000", 2024-09-18T00:18:25.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2873000000000002e-05 2024-09-18T00:18:25.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960199+0000", 2024-09-18T00:18:25.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042373990000000002 2024-09-18T00:18:25.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960250+0000", 2024-09-18T00:18:25.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1267999999999998e-05 2024-09-18T00:18:25.859 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.859 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.859 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.859 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2344 3.4 e37/23 3:20ffa5cc:::rbd_data.112aa7ba4578.000000000000021d:head v 37'95, mlcod=36'93)", 2024-09-18T00:18:25.859 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.593553+0000", 2024-09-18T00:18:25.859 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.239208219, 2024-09-18T00:18:25.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033906819999999999, 2024-09-18T00:18:25.860 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.860 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.860 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.860 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593553+0000", 2024-09-18T00:18:25.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593553+0000", 2024-09-18T00:18:25.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593555+0000", 2024-09-18T00:18:25.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.643e-06 2024-09-18T00:18:25.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593567+0000", 2024-09-18T00:18:25.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2289999999999999e-05 2024-09-18T00:18:25.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593616+0000", 2024-09-18T00:18:25.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8406999999999999e-05 2024-09-18T00:18:25.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593629+0000", 2024-09-18T00:18:25.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2774e-05 2024-09-18T00:18:25.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593696+0000", 2024-09-18T00:18:25.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7587000000000002e-05 2024-09-18T00:18:25.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593720+0000", 2024-09-18T00:18:25.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3638e-05 2024-09-18T00:18:25.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.596897+0000", 2024-09-18T00:18:25.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003176772 2024-09-18T00:18:25.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.596944+0000", 2024-09-18T00:18:25.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7571000000000002e-05 2024-09-18T00:18:25.864 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.864 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.864 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.864 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2381 3.0 e37/23 3:01782307:::rbd_data.112aa7ba4578.000000000000004f:head v 37'16, mlcod=35'14)", 2024-09-18T00:18:25.864 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.662312+0000", 2024-09-18T00:18:25.864 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.1704495589999997, 2024-09-18T00:18:25.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033464179999999999, 2024-09-18T00:18:25.864 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.864 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.865 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.865 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662312+0000", 2024-09-18T00:18:25.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662312+0000", 2024-09-18T00:18:25.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662313+0000", 2024-09-18T00:18:25.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1900000000000001e-07 2024-09-18T00:18:25.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662319+0000", 2024-09-18T00:18:25.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4670000000000001e-06 2024-09-18T00:18:25.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662330+0000", 2024-09-18T00:18:25.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.023e-05 2024-09-18T00:18:25.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662336+0000", 2024-09-18T00:18:25.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.691e-06 2024-09-18T00:18:25.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662427+0000", 2024-09-18T00:18:25.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0216000000000005e-05 2024-09-18T00:18:25.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662448+0000", 2024-09-18T00:18:25.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1795000000000001e-05 2024-09-18T00:18:25.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.665606+0000", 2024-09-18T00:18:25.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031574799999999998 2024-09-18T00:18:25.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.665659+0000", 2024-09-18T00:18:25.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2719999999999997e-05 2024-09-18T00:18:25.869 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.869 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.869 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.869 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2455 3.9 e37/23 3:93e40d20:::rbd_data.112aa7ba4578.000000000000008a:head v 37'16, mlcod=36'14)", 2024-09-18T00:18:25.869 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.948469+0000", 2024-09-18T00:18:25.869 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8842931460000001, 2024-09-18T00:18:25.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028601339999999999, 2024-09-18T00:18:25.869 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.869 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.869 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.869 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.948469+0000", 2024-09-18T00:18:25.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.948469+0000", 2024-09-18T00:18:25.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.948471+0000", 2024-09-18T00:18:25.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6129999999999998e-06 2024-09-18T00:18:25.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.948541+0000", 2024-09-18T00:18:25.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9832999999999994e-05 2024-09-18T00:18:25.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.948577+0000", 2024-09-18T00:18:25.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5849999999999997e-05 2024-09-18T00:18:25.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.948596+0000", 2024-09-18T00:18:25.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8802999999999999e-05 2024-09-18T00:18:25.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.948741+0000", 2024-09-18T00:18:25.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145618 2024-09-18T00:18:25.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.948773+0000", 2024-09-18T00:18:25.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1885999999999999e-05 2024-09-18T00:18:25.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.951234+0000", 2024-09-18T00:18:25.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002460552 2024-09-18T00:18:25.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.951329+0000", 2024-09-18T00:18:25.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4979e-05 2024-09-18T00:18:25.873 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.873 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.873 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.874 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2460 3.0 e37/23 3:016548b0:::rbd_data.112aa7ba4578.0000000000000018:head v 37'18, mlcod=37'16)", 2024-09-18T00:18:25.874 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.985197+0000", 2024-09-18T00:18:25.874 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8475646699999997, 2024-09-18T00:18:25.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004264894, 2024-09-18T00:18:25.874 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.874 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.874 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.874 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985197+0000", 2024-09-18T00:18:25.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985197+0000", 2024-09-18T00:18:25.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985198+0000", 2024-09-18T00:18:25.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.342e-06 2024-09-18T00:18:25.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985210+0000", 2024-09-18T00:18:25.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1783999999999999e-05 2024-09-18T00:18:25.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985231+0000", 2024-09-18T00:18:25.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0769000000000001e-05 2024-09-18T00:18:25.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985241+0000", 2024-09-18T00:18:25.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0266e-05 2024-09-18T00:18:25.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985398+0000", 2024-09-18T00:18:25.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156701 2024-09-18T00:18:25.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985447+0000", 2024-09-18T00:18:25.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9391000000000002e-05 2024-09-18T00:18:25.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989422+0000", 2024-09-18T00:18:25.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039745470000000001 2024-09-18T00:18:25.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989462+0000", 2024-09-18T00:18:25.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0093999999999999e-05 2024-09-18T00:18:25.878 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.878 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.878 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.878 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2469 3.0 e37/23 3:02fdd8b9:::rbd_data.112aa7ba4578.0000000000000044:head v 37'19, mlcod=37'16)", 2024-09-18T00:18:25.879 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.989106+0000", 2024-09-18T00:18:25.879 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8436554520000001, 2024-09-18T00:18:25.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004408843, 2024-09-18T00:18:25.879 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.879 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.879 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.879 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989106+0000", 2024-09-18T00:18:25.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989106+0000", 2024-09-18T00:18:25.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989107+0000", 2024-09-18T00:18:25.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.088e-06 2024-09-18T00:18:25.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989115+0000", 2024-09-18T00:18:25.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8590000000000005e-06 2024-09-18T00:18:25.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989126+0000", 2024-09-18T00:18:25.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1144e-05 2024-09-18T00:18:25.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989134+0000", 2024-09-18T00:18:25.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4229999999999999e-06 2024-09-18T00:18:25.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989215+0000", 2024-09-18T00:18:25.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1551000000000006e-05 2024-09-18T00:18:25.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989237+0000", 2024-09-18T00:18:25.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1832999999999999e-05 2024-09-18T00:18:25.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.993479+0000", 2024-09-18T00:18:25.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042420119999999999 2024-09-18T00:18:25.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.993515+0000", 2024-09-18T00:18:25.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5933000000000003e-05 2024-09-18T00:18:25.883 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.883 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.883 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.883 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2471 3.4 e37/23 3:26bbbe4b:::rbd_data.112aa7ba4578.000000000000004e:head v 37'98, mlcod=37'95)", 2024-09-18T00:18:25.883 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.989721+0000", 2024-09-18T00:18:25.883 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8430405529999998, 2024-09-18T00:18:25.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043834369999999996, 2024-09-18T00:18:25.883 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.883 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.883 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.884 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989721+0000", 2024-09-18T00:18:25.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989721+0000", 2024-09-18T00:18:25.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989722+0000", 2024-09-18T00:18:25.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3300000000000001e-07 2024-09-18T00:18:25.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989728+0000", 2024-09-18T00:18:25.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:18:25.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989741+0000", 2024-09-18T00:18:25.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3444999999999999e-05 2024-09-18T00:18:25.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989750+0000", 2024-09-18T00:18:25.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7369999999999995e-06 2024-09-18T00:18:25.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989807+0000", 2024-09-18T00:18:25.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6971000000000001e-05 2024-09-18T00:18:25.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989826+0000", 2024-09-18T00:18:25.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8918000000000001e-05 2024-09-18T00:18:25.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994077+0000", 2024-09-18T00:18:25.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042507719999999999 2024-09-18T00:18:25.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994105+0000", 2024-09-18T00:18:25.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7885000000000002e-05 2024-09-18T00:18:25.887 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.888 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.888 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.888 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2472 3.0 e37/23 3:01782307:::rbd_data.112aa7ba4578.000000000000004f:head v 37'20, mlcod=37'17)", 2024-09-18T00:18:25.888 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.990421+0000", 2024-09-18T00:18:25.888 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8423404589999999, 2024-09-18T00:18:25.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00366118, 2024-09-18T00:18:25.888 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.888 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.888 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.889 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990421+0000", 2024-09-18T00:18:25.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990421+0000", 2024-09-18T00:18:25.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990422+0000", 2024-09-18T00:18:25.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8599999999999998e-07 2024-09-18T00:18:25.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990428+0000", 2024-09-18T00:18:25.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.908e-06 2024-09-18T00:18:25.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990438+0000", 2024-09-18T00:18:25.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0244000000000001e-05 2024-09-18T00:18:25.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990445+0000", 2024-09-18T00:18:25.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7170000000000002e-06 2024-09-18T00:18:25.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990492+0000", 2024-09-18T00:18:25.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7644e-05 2024-09-18T00:18:25.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990513+0000", 2024-09-18T00:18:25.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.02e-05 2024-09-18T00:18:25.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994045+0000", 2024-09-18T00:18:25.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035321699999999998 2024-09-18T00:18:25.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994082+0000", 2024-09-18T00:18:25.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7611000000000003e-05 2024-09-18T00:18:25.893 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.893 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.893 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.893 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2474 3.0 e37/23 3:01e056be:::rbd_data.112aa7ba4578.0000000000000051:head v 37'21, mlcod=37'17)", 2024-09-18T00:18:25.893 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.991077+0000", 2024-09-18T00:18:25.893 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8416843180000004, 2024-09-18T00:18:25.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0030287030000000002, 2024-09-18T00:18:25.893 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.893 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.893 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.894 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991077+0000", 2024-09-18T00:18:25.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991077+0000", 2024-09-18T00:18:25.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991078+0000", 2024-09-18T00:18:25.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0699999999999996e-07 2024-09-18T00:18:25.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991085+0000", 2024-09-18T00:18:25.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5339999999999996e-06 2024-09-18T00:18:25.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991098+0000", 2024-09-18T00:18:25.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3471e-05 2024-09-18T00:18:25.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991106+0000", 2024-09-18T00:18:25.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.311e-06 2024-09-18T00:18:25.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991152+0000", 2024-09-18T00:18:25.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6458999999999997e-05 2024-09-18T00:18:25.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.991164+0000", 2024-09-18T00:18:25.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2448e-05 2024-09-18T00:18:25.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994085+0000", 2024-09-18T00:18:25.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0029206150000000001 2024-09-18T00:18:25.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994106+0000", 2024-09-18T00:18:25.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0958000000000002e-05 2024-09-18T00:18:25.897 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.897 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.897 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.898 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2898 3.9 e38/23 3:956131f6:::rbd_data.112aa7ba4578.000000000000019f:head v 38'23, mlcod=38'21)", 2024-09-18T00:18:25.898 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.040063+0000", 2024-09-18T00:18:25.898 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.7926983910000001, 2024-09-18T00:18:25.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031556990000000001, 2024-09-18T00:18:25.898 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.898 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.898 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.898 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040063+0000", 2024-09-18T00:18:25.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040063+0000", 2024-09-18T00:18:25.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040065+0000", 2024-09-18T00:18:25.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.84e-06 2024-09-18T00:18:25.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040076+0000", 2024-09-18T00:18:25.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1267000000000001e-05 2024-09-18T00:18:25.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040104+0000", 2024-09-18T00:18:25.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7406000000000002e-05 2024-09-18T00:18:25.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040116+0000", 2024-09-18T00:18:25.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.241e-05 2024-09-18T00:18:25.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040244+0000", 2024-09-18T00:18:25.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012744199999999999 2024-09-18T00:18:25.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040280+0000", 2024-09-18T00:18:25.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6034000000000001e-05 2024-09-18T00:18:25.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.043136+0000", 2024-09-18T00:18:25.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028566170000000001 2024-09-18T00:18:25.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.043219+0000", 2024-09-18T00:18:25.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2682999999999994e-05 2024-09-18T00:18:25.902 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.902 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.902 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.902 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2921 3.0 e38/23 3:02fdd8b9:::rbd_data.112aa7ba4578.0000000000000044:head v 38'27, mlcod=38'25)", 2024-09-18T00:18:25.902 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.091200+0000", 2024-09-18T00:18:25.903 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.7415617989999999, 2024-09-18T00:18:25.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031551589999999998, 2024-09-18T00:18:25.903 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.903 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.903 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.903 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.091200+0000", 2024-09-18T00:18:25.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.091200+0000", 2024-09-18T00:18:25.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.091202+0000", 2024-09-18T00:18:25.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2800000000000002e-06 2024-09-18T00:18:25.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.091253+0000", 2024-09-18T00:18:25.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0593000000000002e-05 2024-09-18T00:18:25.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.091287+0000", 2024-09-18T00:18:25.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3782999999999998e-05 2024-09-18T00:18:25.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.091301+0000", 2024-09-18T00:18:25.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4946999999999999e-05 2024-09-18T00:18:25.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.091443+0000", 2024-09-18T00:18:25.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014168199999999999 2024-09-18T00:18:25.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.091509+0000", 2024-09-18T00:18:25.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5766999999999996e-05 2024-09-18T00:18:25.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.094270+0000", 2024-09-18T00:18:25.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027610880000000001 2024-09-18T00:18:25.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.094355+0000", 2024-09-18T00:18:25.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5018999999999994e-05 2024-09-18T00:18:25.907 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.907 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.907 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.908 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2973 3.4 e38/23 3:23ae92e0:::rbd_data.112aa7ba4578.00000000000002c2:head v 38'99, mlcod=37'97)", 2024-09-18T00:18:25.908 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.229597+0000", 2024-09-18T00:18:25.908 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.6031651230000001, 2024-09-18T00:18:25.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0048943249999999997, 2024-09-18T00:18:25.908 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.908 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.908 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.908 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229597+0000", 2024-09-18T00:18:25.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229597+0000", 2024-09-18T00:18:25.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229599+0000", 2024-09-18T00:18:25.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.396e-06 2024-09-18T00:18:25.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229634+0000", 2024-09-18T00:18:25.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5237000000000002e-05 2024-09-18T00:18:25.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229666+0000", 2024-09-18T00:18:25.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1888999999999999e-05 2024-09-18T00:18:25.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229681+0000", 2024-09-18T00:18:25.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4783000000000001e-05 2024-09-18T00:18:25.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229837+0000", 2024-09-18T00:18:25.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156193 2024-09-18T00:18:25.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229899+0000", 2024-09-18T00:18:25.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1458999999999996e-05 2024-09-18T00:18:25.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.234405+0000", 2024-09-18T00:18:25.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045061220000000004 2024-09-18T00:18:25.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.234491+0000", 2024-09-18T00:18:25.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6246000000000001e-05 2024-09-18T00:18:25.912 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.912 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.912 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.913 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2983 3.9 e38/23 3:93d2c681:::rbd_data.112aa7ba4578.000000000000003d:head v 38'24, mlcod=38'22)", 2024-09-18T00:18:25.913 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.258087+0000", 2024-09-18T00:18:25.913 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.5746748770000001, 2024-09-18T00:18:25.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0034586859999999999, 2024-09-18T00:18:25.913 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.913 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.913 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.913 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258087+0000", 2024-09-18T00:18:25.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258087+0000", 2024-09-18T00:18:25.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258089+0000", 2024-09-18T00:18:25.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5639999999999999e-06 2024-09-18T00:18:25.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258196+0000", 2024-09-18T00:18:25.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106185 2024-09-18T00:18:25.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258235+0000", 2024-09-18T00:18:25.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9867000000000003e-05 2024-09-18T00:18:25.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258257+0000", 2024-09-18T00:18:25.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1166e-05 2024-09-18T00:18:25.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258360+0000", 2024-09-18T00:18:25.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103396 2024-09-18T00:18:25.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258388+0000", 2024-09-18T00:18:25.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7773e-05 2024-09-18T00:18:25.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.261500+0000", 2024-09-18T00:18:25.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00311233 2024-09-18T00:18:25.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.261545+0000", 2024-09-18T00:18:25.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5405000000000003e-05 2024-09-18T00:18:25.917 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.917 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.917 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.918 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:3013 3.4 e38/23 3:23ae92e0:::rbd_data.112aa7ba4578.00000000000002c2:head v 38'101, mlcod=38'99)", 2024-09-18T00:18:25.918 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.411123+0000", 2024-09-18T00:18:25.918 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.4216382900000002, 2024-09-18T00:18:25.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003259236, 2024-09-18T00:18:25.918 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.918 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.918 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.918 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411123+0000", 2024-09-18T00:18:25.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411123+0000", 2024-09-18T00:18:25.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411124+0000", 2024-09-18T00:18:25.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:18:25.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411129+0000", 2024-09-18T00:18:25.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0529999999999997e-06 2024-09-18T00:18:25.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411138+0000", 2024-09-18T00:18:25.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8370000000000003e-06 2024-09-18T00:18:25.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411144+0000", 2024-09-18T00:18:25.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.852e-06 2024-09-18T00:18:25.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411201+0000", 2024-09-18T00:18:25.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6963999999999999e-05 2024-09-18T00:18:25.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411219+0000", 2024-09-18T00:18:25.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8780999999999999e-05 2024-09-18T00:18:25.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.414342+0000", 2024-09-18T00:18:25.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031231929999999998 2024-09-18T00:18:25.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.414383+0000", 2024-09-18T00:18:25.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0139000000000003e-05 2024-09-18T00:18:25.922 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.922 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.922 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.923 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:5722 3.4 e41/23 3:20381d86:::rbd_data.112a3a39fc45.000000000000001d:head v 41'107, mlcod=40'105)", 2024-09-18T00:18:25.923 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:18:25.923 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.49726604099999999, 2024-09-18T00:18:25.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01185172, 2024-09-18T00:18:25.923 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.923 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.923 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.923 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:18:25.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:18:25.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335497+0000", 2024-09-18T00:18:25.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.505e-06 2024-09-18T00:18:25.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335945+0000", 2024-09-18T00:18:25.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000447687 2024-09-18T00:18:25.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335961+0000", 2024-09-18T00:18:25.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6430999999999999e-05 2024-09-18T00:18:25.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335969+0000", 2024-09-18T00:18:25.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:18:25.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336030+0000", 2024-09-18T00:18:25.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1255999999999998e-05 2024-09-18T00:18:25.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336061+0000", 2024-09-18T00:18:25.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0579e-05 2024-09-18T00:18:25.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347296+0000", 2024-09-18T00:18:25.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011235663 2024-09-18T00:18:25.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347347+0000", 2024-09-18T00:18:25.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0914000000000001e-05 2024-09-18T00:18:25.927 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.927 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.927 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:5812 3.4 e41/23 3:20381d86:::rbd_data.112a3a39fc45.000000000000001d:head v 41'108, mlcod=40'106)", 2024-09-18T00:18:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.418666+0000", 2024-09-18T00:18:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.41409591200000001, 2024-09-18T00:18:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0030070420000000001, 2024-09-18T00:18:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.418666+0000", 2024-09-18T00:18:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:25.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.418666+0000", 2024-09-18T00:18:25.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:25.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:25.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.418668+0000", 2024-09-18T00:18:25.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8610000000000001e-06 2024-09-18T00:18:25.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:25.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.418702+0000", 2024-09-18T00:18:25.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3909999999999999e-05 2024-09-18T00:18:25.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:25.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.418720+0000", 2024-09-18T00:18:25.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8406e-05 2024-09-18T00:18:25.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:25.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.418731+0000", 2024-09-18T00:18:25.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0784e-05 2024-09-18T00:18:25.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:25.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.418796+0000", 2024-09-18T00:18:25.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4978e-05 2024-09-18T00:18:25.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:25.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.418811+0000", 2024-09-18T00:18:25.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4874e-05 2024-09-18T00:18:25.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:25.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.421633+0000", 2024-09-18T00:18:25.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002822078 2024-09-18T00:18:25.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:25.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:25.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:25.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.421673+0000", 2024-09-18T00:18:25.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0151000000000002e-05 2024-09-18T00:18:25.933 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.933 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.933 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.933 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:25.933 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:25.933 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:25.934 INFO:teuthology.orchestra.run.smithi097.stdout:1197 read 0x84569d7 thru 0x8457d52 (0x137c bytes) 2024-09-18T00:18:25.934 INFO:teuthology.orchestra.run.smithi097.stdout:1198 read 0xa8a17e4 thru 0xa8afc52 (0xe46f bytes) 2024-09-18T00:18:25.934 INFO:teuthology.orchestra.run.smithi097.stdout:1199 writesame 0x467ffe3 thru 0x468bfc9 data_size 0x1(0xbfe7 bytes) 2024-09-18T00:18:25.935 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:18:25.935 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:25.931+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:25.963 INFO:teuthology.orchestra.run.smithi097.stdout:1200 trunc from 0xc705f41 to 0x456ccdf 2024-09-18T00:18:25.992 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:25.992 INFO:teuthology.orchestra.run.smithi097.stdout:1202 read 0x172dad thru 0x17c078 (0x92cc bytes) 2024-09-18T00:18:25.992 INFO:teuthology.orchestra.run.smithi097.stdout:1204 punch from 0x17b9f23 to 0x17c11be, (0x729b bytes) 2024-09-18T00:18:26.017 INFO:teuthology.orchestra.run.smithi097.stdout:1205 write 0xe9c6d74 thru 0xe9d153f (0xa7cc bytes) 2024-09-18T00:18:26.026 INFO:teuthology.orchestra.run.smithi097.stdout:1206 punch from 0xe51dfa7 to 0xe52bea2, (0xdefb bytes) 2024-09-18T00:18:26.026 INFO:teuthology.orchestra.run.smithi097.stdout:1207 write 0x4eebe2f thru 0x4ef99a9 (0xdb7b bytes) 2024-09-18T00:18:26.033 INFO:teuthology.orchestra.run.smithi097.stdout:1209 punch from 0x57b5147 to 0x57b5432, (0x2eb bytes) 2024-09-18T00:18:26.033 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:26.033 INFO:teuthology.orchestra.run.smithi097.stdout:1212 punch from 0x1c13378 to 0x1c156db, (0x2363 bytes) 2024-09-18T00:18:26.033 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:26.033 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:26.033 INFO:teuthology.orchestra.run.smithi097.stdout:1215 writesame 0x8dda07d thru 0x8dddd79 data_size 0x1(0x3cfd bytes) 2024-09-18T00:18:26.034 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:26.031+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:26.085 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:26.085 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:26.085 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:26.085 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:26.089 INFO:teuthology.orchestra.run.smithi097.stdout:1216 clone 5 order 19 su 65536 sc 2 2024-09-18T00:18:26.098 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:18:26.136 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:26.131+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:26.237 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:26.235+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:26.265 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:26.265 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:26.265 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:26.280 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:18:26.337 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:26.331+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:26.438 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:26.435+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:26.448 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:26.448 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:599 3.15 3:ac22c048:::rbd_data.112ae20746f5.0000000000000110:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.980706+0000", 2024-09-18T00:18:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.465243955, 2024-09-18T00:18:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017399257000000001, 2024-09-18T00:18:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.449 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 599 2024-09-18T00:18:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980706+0000", 2024-09-18T00:18:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980706+0000", 2024-09-18T00:18:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980708+0000", 2024-09-18T00:18:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5919999999999999e-06 2024-09-18T00:18:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980722+0000", 2024-09-18T00:18:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.364e-05 2024-09-18T00:18:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980752+0000", 2024-09-18T00:18:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9794999999999998e-05 2024-09-18T00:18:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980778+0000", 2024-09-18T00:18:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6494000000000001e-05 2024-09-18T00:18:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980885+0000", 2024-09-18T00:18:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107104 2024-09-18T00:18:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.981079+0000", 2024-09-18T00:18:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019375699999999999 2024-09-18T00:18:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.981866+0000", 2024-09-18T00:18:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00078716099999999998 2024-09-18T00:18:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.997852+0000", 2024-09-18T00:18:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015986305999999999 2024-09-18T00:18:26.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.997957+0000", 2024-09-18T00:18:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104085 2024-09-18T00:18:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.998037+0000", 2024-09-18T00:18:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0098000000000005e-05 2024-09-18T00:18:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.998105+0000", 2024-09-18T00:18:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8225000000000005e-05 2024-09-18T00:18:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:640 3.7 3:e0810bc8:::rbd_data.112ae20746f5.0000000000000019:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.104726+0000", 2024-09-18T00:18:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.341223126999999, 2024-09-18T00:18:26.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017067268, 2024-09-18T00:18:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 640 2024-09-18T00:18:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104726+0000", 2024-09-18T00:18:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104726+0000", 2024-09-18T00:18:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104728+0000", 2024-09-18T00:18:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.336e-06 2024-09-18T00:18:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104735+0000", 2024-09-18T00:18:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6149999999999999e-06 2024-09-18T00:18:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104753+0000", 2024-09-18T00:18:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7362000000000001e-05 2024-09-18T00:18:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104761+0000", 2024-09-18T00:18:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5609999999999999e-06 2024-09-18T00:18:26.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104842+0000", 2024-09-18T00:18:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0526999999999997e-05 2024-09-18T00:18:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104916+0000", 2024-09-18T00:18:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3724000000000001e-05 2024-09-18T00:18:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.106201+0000", 2024-09-18T00:18:26.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001285046 2024-09-18T00:18:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121677+0000", 2024-09-18T00:18:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01547665 2024-09-18T00:18:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121696+0000", 2024-09-18T00:18:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8304999999999998e-05 2024-09-18T00:18:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:26.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121744+0000", 2024-09-18T00:18:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8415000000000003e-05 2024-09-18T00:18:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121794+0000", 2024-09-18T00:18:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9727e-05 2024-09-18T00:18:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:26.461 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.645782884999999, 2024-09-18T00:18:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:18:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:18:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:26.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:18:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:18:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:18:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:18:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:18:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:18:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:18:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:18:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:18:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:18:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:18:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:18:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:18:26.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:18:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:18:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:18:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:18:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:18:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:18:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:18:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:18:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:18:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:18:26.467 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.629575012, 2024-09-18T00:18:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:18:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:18:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:18:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:18:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:18:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:18:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:18:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:18:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:18:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:18:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:18:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:18:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:18:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:18:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:18:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:5724 3.1e 3:78324145:::rbd_data.112a3a39fc45.000000000000001e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.341891+0000", 2024-09-18T00:18:26.471 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.1040585489999999, 2024-09-18T00:18:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014843063, 2024-09-18T00:18:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 5724 2024-09-18T00:18:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.341891+0000", 2024-09-18T00:18:26.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.341891+0000", 2024-09-18T00:18:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.341893+0000", 2024-09-18T00:18:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1129999999999999e-06 2024-09-18T00:18:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343233+0000", 2024-09-18T00:18:26.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001339614 2024-09-18T00:18:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343251+0000", 2024-09-18T00:18:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8479e-05 2024-09-18T00:18:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343260+0000", 2024-09-18T00:18:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1470000000000007e-06 2024-09-18T00:18:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343333+0000", 2024-09-18T00:18:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2302e-05 2024-09-18T00:18:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343494+0000", 2024-09-18T00:18:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000161593 2024-09-18T00:18:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 6", 2024-09-18T00:18:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343778+0000", 2024-09-18T00:18:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028366099999999999 2024-09-18T00:18:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:26.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.351688+0000", 2024-09-18T00:18:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0079100739999999996 2024-09-18T00:18:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.356673+0000", 2024-09-18T00:18:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049848210000000004 2024-09-18T00:18:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.356706+0000", 2024-09-18T00:18:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3531000000000001e-05 2024-09-18T00:18:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.356734+0000", 2024-09-18T00:18:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7727999999999999e-05 2024-09-18T00:18:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6175 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 8585216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.473970+0000", 2024-09-18T00:18:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.97197917599999994, 2024-09-18T00:18:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013962384, 2024-09-18T00:18:26.477 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6175 2024-09-18T00:18:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473970+0000", 2024-09-18T00:18:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473970+0000", 2024-09-18T00:18:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473972+0000", 2024-09-18T00:18:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1310000000000001e-06 2024-09-18T00:18:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473977+0000", 2024-09-18T00:18:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1769999999999997e-06 2024-09-18T00:18:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473993+0000", 2024-09-18T00:18:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5988999999999999e-05 2024-09-18T00:18:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474005+0000", 2024-09-18T00:18:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2622999999999999e-05 2024-09-18T00:18:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.485712+0000", 2024-09-18T00:18:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011706701 2024-09-18T00:18:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487289+0000", 2024-09-18T00:18:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0015770280000000001 2024-09-18T00:18:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487933+0000", 2024-09-18T00:18:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00064373499999999999 2024-09-18T00:18:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.481 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6176 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 9371648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474022+0000", 2024-09-18T00:18:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.97192791099999998, 2024-09-18T00:18:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014183915, 2024-09-18T00:18:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.482 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6176 2024-09-18T00:18:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474022+0000", 2024-09-18T00:18:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474022+0000", 2024-09-18T00:18:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474023+0000", 2024-09-18T00:18:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.02e-06 2024-09-18T00:18:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474028+0000", 2024-09-18T00:18:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9599999999999999e-06 2024-09-18T00:18:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474044+0000", 2024-09-18T00:18:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6113999999999999e-05 2024-09-18T00:18:26.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474056+0000", 2024-09-18T00:18:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2349e-05 2024-09-18T00:18:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487994+0000", 2024-09-18T00:18:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013937748 2024-09-18T00:18:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488088+0000", 2024-09-18T00:18:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3888999999999995e-05 2024-09-18T00:18:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488206+0000", 2024-09-18T00:18:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117835 2024-09-18T00:18:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6177 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474074+0000", 2024-09-18T00:18:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.97187524300000006, 2024-09-18T00:18:26.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01434039, 2024-09-18T00:18:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6177 2024-09-18T00:18:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474074+0000", 2024-09-18T00:18:26.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474074+0000", 2024-09-18T00:18:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474075+0000", 2024-09-18T00:18:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-18T00:18:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474080+0000", 2024-09-18T00:18:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9459999999999997e-06 2024-09-18T00:18:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474097+0000", 2024-09-18T00:18:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6705999999999999e-05 2024-09-18T00:18:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474110+0000", 2024-09-18T00:18:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2582000000000001e-05 2024-09-18T00:18:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488252+0000", 2024-09-18T00:18:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014142937 2024-09-18T00:18:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488327+0000", 2024-09-18T00:18:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4184000000000007e-05 2024-09-18T00:18:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488415+0000", 2024-09-18T00:18:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8075e-05 2024-09-18T00:18:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6178 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474128+0000", 2024-09-18T00:18:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.97182108599999995, 2024-09-18T00:18:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014460381, 2024-09-18T00:18:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.491 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6178 2024-09-18T00:18:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474128+0000", 2024-09-18T00:18:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474128+0000", 2024-09-18T00:18:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474130+0000", 2024-09-18T00:18:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1489999999999999e-06 2024-09-18T00:18:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474135+0000", 2024-09-18T00:18:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0830000000000003e-06 2024-09-18T00:18:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474151+0000", 2024-09-18T00:18:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6677e-05 2024-09-18T00:18:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474166+0000", 2024-09-18T00:18:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4871e-05 2024-09-18T00:18:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488458+0000", 2024-09-18T00:18:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014292206999999999 2024-09-18T00:18:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488516+0000", 2024-09-18T00:18:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7627999999999998e-05 2024-09-18T00:18:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488589+0000", 2024-09-18T00:18:26.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2766000000000001e-05 2024-09-18T00:18:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6179 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 11730944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474183+0000", 2024-09-18T00:18:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.97176633199999995, 2024-09-18T00:18:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014618816, 2024-09-18T00:18:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.496 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6179 2024-09-18T00:18:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474183+0000", 2024-09-18T00:18:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474183+0000", 2024-09-18T00:18:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474184+0000", 2024-09-18T00:18:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.083e-06 2024-09-18T00:18:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474190+0000", 2024-09-18T00:18:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6570000000000002e-06 2024-09-18T00:18:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474206+0000", 2024-09-18T00:18:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5914e-05 2024-09-18T00:18:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474225+0000", 2024-09-18T00:18:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9369e-05 2024-09-18T00:18:26.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488639+0000", 2024-09-18T00:18:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014413769 2024-09-18T00:18:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488709+0000", 2024-09-18T00:18:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9655999999999999e-05 2024-09-18T00:18:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488802+0000", 2024-09-18T00:18:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3368000000000005e-05 2024-09-18T00:18:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6180 3.8 3:16b24e91:::rbd_data.112aa7ba4578.0000000000000161:5 [read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474244+0000", 2024-09-18T00:18:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.97170603499999997, 2024-09-18T00:18:26.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014495918999999999, 2024-09-18T00:18:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6180 2024-09-18T00:18:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474244+0000", 2024-09-18T00:18:26.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474244+0000", 2024-09-18T00:18:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474245+0000", 2024-09-18T00:18:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.105e-06 2024-09-18T00:18:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474318+0000", 2024-09-18T00:18:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3182999999999994e-05 2024-09-18T00:18:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474345+0000", 2024-09-18T00:18:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6730999999999999e-05 2024-09-18T00:18:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474359+0000", 2024-09-18T00:18:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4433999999999999e-05 2024-09-18T00:18:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488626+0000", 2024-09-18T00:18:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014266993 2024-09-18T00:18:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488680+0000", 2024-09-18T00:18:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4453999999999997e-05 2024-09-18T00:18:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488739+0000", 2024-09-18T00:18:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9018999999999999e-05 2024-09-18T00:18:26.505 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6181 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 12517376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474384+0000", 2024-09-18T00:18:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.97156509899999999, 2024-09-18T00:18:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014686072999999999, 2024-09-18T00:18:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.506 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6181 2024-09-18T00:18:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474384+0000", 2024-09-18T00:18:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474384+0000", 2024-09-18T00:18:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474386+0000", 2024-09-18T00:18:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.688e-06 2024-09-18T00:18:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474392+0000", 2024-09-18T00:18:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.575e-06 2024-09-18T00:18:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474409+0000", 2024-09-18T00:18:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7638999999999998e-05 2024-09-18T00:18:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474423+0000", 2024-09-18T00:18:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3237e-05 2024-09-18T00:18:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488844+0000", 2024-09-18T00:18:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014421079 2024-09-18T00:18:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488943+0000", 2024-09-18T00:18:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9764000000000002e-05 2024-09-18T00:18:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.489071+0000", 2024-09-18T00:18:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012709099999999999 2024-09-18T00:18:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.511 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6232 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.478338+0000", 2024-09-18T00:18:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.96761202099999999, 2024-09-18T00:18:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012208848, 2024-09-18T00:18:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.512 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6232 2024-09-18T00:18:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478338+0000", 2024-09-18T00:18:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478338+0000", 2024-09-18T00:18:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478339+0000", 2024-09-18T00:18:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7399999999999991e-07 2024-09-18T00:18:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478343+0000", 2024-09-18T00:18:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5090000000000004e-06 2024-09-18T00:18:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478359+0000", 2024-09-18T00:18:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5488000000000001e-05 2024-09-18T00:18:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478371+0000", 2024-09-18T00:18:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.206e-05 2024-09-18T00:18:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.490422+0000", 2024-09-18T00:18:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012051294000000001 2024-09-18T00:18:26.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.490476+0000", 2024-09-18T00:18:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3761999999999998e-05 2024-09-18T00:18:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.490546+0000", 2024-09-18T00:18:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0760999999999996e-05 2024-09-18T00:18:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6278 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 9502720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.478787+0000", 2024-09-18T00:18:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.96716217599999998, 2024-09-18T00:18:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012210743, 2024-09-18T00:18:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.518 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6278 2024-09-18T00:18:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478787+0000", 2024-09-18T00:18:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478787+0000", 2024-09-18T00:18:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478788+0000", 2024-09-18T00:18:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0860000000000001e-06 2024-09-18T00:18:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478794+0000", 2024-09-18T00:18:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0340000000000004e-06 2024-09-18T00:18:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478810+0000", 2024-09-18T00:18:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6030999999999999e-05 2024-09-18T00:18:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478826+0000", 2024-09-18T00:18:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6782999999999998e-05 2024-09-18T00:18:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.490839+0000", 2024-09-18T00:18:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012012492 2024-09-18T00:18:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.490905+0000", 2024-09-18T00:18:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6013999999999995e-05 2024-09-18T00:18:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.490998+0000", 2024-09-18T00:18:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3302999999999998e-05 2024-09-18T00:18:26.522 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6279 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.478842+0000", 2024-09-18T00:18:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.96710769699999999, 2024-09-18T00:18:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012385353999999999, 2024-09-18T00:18:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.523 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6279 2024-09-18T00:18:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478842+0000", 2024-09-18T00:18:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478842+0000", 2024-09-18T00:18:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478843+0000", 2024-09-18T00:18:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.046e-06 2024-09-18T00:18:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478847+0000", 2024-09-18T00:18:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5369999999999999e-06 2024-09-18T00:18:26.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478865+0000", 2024-09-18T00:18:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7129000000000002e-05 2024-09-18T00:18:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478877+0000", 2024-09-18T00:18:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2476999999999999e-05 2024-09-18T00:18:26.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491027+0000", 2024-09-18T00:18:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01214989 2024-09-18T00:18:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491090+0000", 2024-09-18T00:18:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3078999999999997e-05 2024-09-18T00:18:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491227+0000", 2024-09-18T00:18:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013719599999999999 2024-09-18T00:18:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6280 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 11075584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.478893+0000", 2024-09-18T00:18:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.96705684800000002, 2024-09-18T00:18:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012517118000000001, 2024-09-18T00:18:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.528 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.529 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.529 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6280 2024-09-18T00:18:26.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.529 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478893+0000", 2024-09-18T00:18:26.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478893+0000", 2024-09-18T00:18:26.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478894+0000", 2024-09-18T00:18:26.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-18T00:18:26.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478905+0000", 2024-09-18T00:18:26.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1045e-05 2024-09-18T00:18:26.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478921+0000", 2024-09-18T00:18:26.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6469000000000001e-05 2024-09-18T00:18:26.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478934+0000", 2024-09-18T00:18:26.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2525e-05 2024-09-18T00:18:26.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491258+0000", 2024-09-18T00:18:26.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012324422999999999 2024-09-18T00:18:26.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491327+0000", 2024-09-18T00:18:26.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8923000000000004e-05 2024-09-18T00:18:26.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491410+0000", 2024-09-18T00:18:26.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2730000000000002e-05 2024-09-18T00:18:26.532 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.532 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.532 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.532 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6281 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 11862016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:26.533 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.478952+0000", 2024-09-18T00:18:26.533 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.96699744899999995, 2024-09-18T00:18:26.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013203609, 2024-09-18T00:18:26.533 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.533 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.533 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:26.533 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.533 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.534 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.534 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6281 2024-09-18T00:18:26.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.534 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478952+0000", 2024-09-18T00:18:26.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478952+0000", 2024-09-18T00:18:26.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478953+0000", 2024-09-18T00:18:26.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0720000000000001e-06 2024-09-18T00:18:26.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478958+0000", 2024-09-18T00:18:26.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7369999999999997e-06 2024-09-18T00:18:26.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.482143+0000", 2024-09-18T00:18:26.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031850870000000001 2024-09-18T00:18:26.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.482179+0000", 2024-09-18T00:18:26.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6381000000000003e-05 2024-09-18T00:18:26.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.492006+0000", 2024-09-18T00:18:26.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0098267319999999995 2024-09-18T00:18:26.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.492069+0000", 2024-09-18T00:18:26.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2564000000000006e-05 2024-09-18T00:18:26.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.492156+0000", 2024-09-18T00:18:26.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7035999999999998e-05 2024-09-18T00:18:26.537 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.537 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.537 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.537 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6572 3.7 3:e54d9f3a:::rbd_data.112a3a39fc45.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 458184~568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:26.537 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.725733+0000", 2024-09-18T00:18:26.537 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.72021628800000004, 2024-09-18T00:18:26.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016678148, 2024-09-18T00:18:26.538 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.538 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.538 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:26.538 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.538 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.538 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.538 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6572 2024-09-18T00:18:26.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.538 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.725733+0000", 2024-09-18T00:18:26.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.725733+0000", 2024-09-18T00:18:26.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.725736+0000", 2024-09-18T00:18:26.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2919999999999998e-06 2024-09-18T00:18:26.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726273+0000", 2024-09-18T00:18:26.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00053760199999999998 2024-09-18T00:18:26.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726295+0000", 2024-09-18T00:18:26.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1855999999999999e-05 2024-09-18T00:18:26.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726305+0000", 2024-09-18T00:18:26.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0098000000000001e-05 2024-09-18T00:18:26.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726384+0000", 2024-09-18T00:18:26.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8731000000000002e-05 2024-09-18T00:18:26.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726582+0000", 2024-09-18T00:18:26.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019781399999999999 2024-09-18T00:18:26.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:26.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726857+0000", 2024-09-18T00:18:26.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027557299999999999 2024-09-18T00:18:26.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:26.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.735232+0000", 2024-09-18T00:18:26.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083744660000000005 2024-09-18T00:18:26.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:26.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.742317+0000", 2024-09-18T00:18:26.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0070851789999999996 2024-09-18T00:18:26.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:26.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.742357+0000", 2024-09-18T00:18:26.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9975000000000003e-05 2024-09-18T00:18:26.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.742411+0000", 2024-09-18T00:18:26.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4561999999999997e-05 2024-09-18T00:18:26.543 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.543 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.543 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.543 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:26.543 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:26.543 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.60253369199999995, 2024-09-18T00:18:26.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:18:26.544 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.544 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.544 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:26.544 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:26.544 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:26.544 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:26.544 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:18:26.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.544 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:26.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:26.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:18:26.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:18:26.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:18:26.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:18:26.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:18:26.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:18:26.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:18:26.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:18:26.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:18:26.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:18:26.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:18:26.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:18:26.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:26.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:18:26.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:18:26.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:26.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:18:26.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:18:26.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:26.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:18:26.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:18:26.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:26.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:18:26.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:18:26.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:18:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:18:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:18:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.58125713999999995, 2024-09-18T00:18:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:18:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:26.550 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:26.551 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:26.551 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:26.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:26.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:26.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:26.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:26.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:26.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:26.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:18:26.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:18:26.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:26.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:18:26.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:18:26.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:26.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:18:26.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:18:26.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:26.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:18:26.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:18:26.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:26.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:18:26.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:18:26.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:26.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:18:26.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:18:26.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:26.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:26.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:26.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:18:26.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:18:26.554 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.554 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.554 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.554 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:26.554 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:26.554 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:26.556 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:18:26.557 INFO:teuthology.orchestra.run.smithi097.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T00:18:26.557 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T00:18:26.558 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:26.551+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:26.643 INFO:teuthology.orchestra.run.smithi097.stdout:truncating image image_client.0-clone4 from 0xe9d1540 (overlap 0x456ccdf) to 0x3e7b862 2024-09-18T00:18:26.657 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:26.655+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:26.698 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:26.699 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:26.699 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:18:26.699 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:26.713 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:18:26.758 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:26.755+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:26.781 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-18T00:18:26.849 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:26.849 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:26.849 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:26.849 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:26.849 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:26.853 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:18:26.894 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:26.891+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:26.925 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:18:26.972 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:26.971+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:27.026 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:27.026 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:27.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:27.026 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:27.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.026 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2418 3.1a 3:59a8b40d:::rbd_data.112aa7ba4578.00000000000001b0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:27.026 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:18:27.026 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.2859341679999998, 2024-09-18T00:18:27.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10488562999999999, 2024-09-18T00:18:27.027 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.027 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.027 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:27.027 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.027 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.027 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.027 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2418 2024-09-18T00:18:27.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.027 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:18:27.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:18:27.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735592+0000", 2024-09-18T00:18:27.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5229999999999998e-06 2024-09-18T00:18:27.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735604+0000", 2024-09-18T00:18:27.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2143e-05 2024-09-18T00:18:27.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735629+0000", 2024-09-18T00:18:27.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.497e-05 2024-09-18T00:18:27.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735648+0000", 2024-09-18T00:18:27.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8896999999999999e-05 2024-09-18T00:18:27.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735775+0000", 2024-09-18T00:18:27.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126935 2024-09-18T00:18:27.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.736118+0000", 2024-09-18T00:18:27.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034330800000000001 2024-09-18T00:18:27.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-18T00:18:27.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.741633+0000", 2024-09-18T00:18:27.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055144139999999996 2024-09-18T00:18:27.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:27.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840134+0000", 2024-09-18T00:18:27.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.098501618999999999 2024-09-18T00:18:27.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:27.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840184+0000", 2024-09-18T00:18:27.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9953000000000001e-05 2024-09-18T00:18:27.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:27.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840336+0000", 2024-09-18T00:18:27.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151476 2024-09-18T00:18:27.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840475+0000", 2024-09-18T00:18:27.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013939200000000001 2024-09-18T00:18:27.033 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.033 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.033 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.033 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2418 3.1a e37/23 ondisk, result = 0)", 2024-09-18T00:18:27.033 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:18:27.033 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.2673443249999998, 2024-09-18T00:18:27.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.085979978999999998, 2024-09-18T00:18:27.033 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.033 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.033 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.034 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:18:27.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:18:27.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754181+0000", 2024-09-18T00:18:27.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8530000000000001e-06 2024-09-18T00:18:27.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754194+0000", 2024-09-18T00:18:27.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.292e-05 2024-09-18T00:18:27.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754230+0000", 2024-09-18T00:18:27.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5602999999999998e-05 2024-09-18T00:18:27.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754249+0000", 2024-09-18T00:18:27.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8850000000000001e-05 2024-09-18T00:18:27.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840070+0000", 2024-09-18T00:18:27.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.085821123999999999 2024-09-18T00:18:27.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840098+0000", 2024-09-18T00:18:27.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8546000000000001e-05 2024-09-18T00:18:27.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840159+0000", 2024-09-18T00:18:27.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1082999999999998e-05 2024-09-18T00:18:27.037 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.038 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.038 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.038 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6285 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 7143424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.038 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.472064+0000", 2024-09-18T00:18:27.038 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.5494594799999999, 2024-09-18T00:18:27.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035057682, 2024-09-18T00:18:27.038 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.038 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.038 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.039 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.039 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.039 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.039 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6285 2024-09-18T00:18:27.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.039 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472064+0000", 2024-09-18T00:18:27.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472064+0000", 2024-09-18T00:18:27.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472065+0000", 2024-09-18T00:18:27.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7599999999999996e-07 2024-09-18T00:18:27.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472069+0000", 2024-09-18T00:18:27.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6339999999999999e-06 2024-09-18T00:18:27.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472079+0000", 2024-09-18T00:18:27.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0189e-05 2024-09-18T00:18:27.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472099+0000", 2024-09-18T00:18:27.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0103e-05 2024-09-18T00:18:27.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.505302+0000", 2024-09-18T00:18:27.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.03320327 2024-09-18T00:18:27.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.505393+0000", 2024-09-18T00:18:27.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1186000000000004e-05 2024-09-18T00:18:27.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.507122+0000", 2024-09-18T00:18:27.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0017285239999999999 2024-09-18T00:18:27.042 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.043 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.043 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.043 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6288 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 9240576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.043 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.472331+0000", 2024-09-18T00:18:27.043 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.5491922730000001, 2024-09-18T00:18:27.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034099633999999997, 2024-09-18T00:18:27.043 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.043 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.043 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.043 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.043 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.043 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.044 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6288 2024-09-18T00:18:27.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.044 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472331+0000", 2024-09-18T00:18:27.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472331+0000", 2024-09-18T00:18:27.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472333+0000", 2024-09-18T00:18:27.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3999999999999999e-06 2024-09-18T00:18:27.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472339+0000", 2024-09-18T00:18:27.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7100000000000001e-06 2024-09-18T00:18:27.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472359+0000", 2024-09-18T00:18:27.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9896999999999999e-05 2024-09-18T00:18:27.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472389+0000", 2024-09-18T00:18:27.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9833e-05 2024-09-18T00:18:27.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.506259+0000", 2024-09-18T00:18:27.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033869585000000001 2024-09-18T00:18:27.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.506323+0000", 2024-09-18T00:18:27.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3913999999999998e-05 2024-09-18T00:18:27.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.506431+0000", 2024-09-18T00:18:27.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010829500000000001 2024-09-18T00:18:27.047 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.047 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.047 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.048 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6289 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 10027008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.048 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.472411+0000", 2024-09-18T00:18:27.048 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.549112993, 2024-09-18T00:18:27.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034219747000000002, 2024-09-18T00:18:27.048 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.048 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.048 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.048 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.049 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.049 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.049 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6289 2024-09-18T00:18:27.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.049 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472411+0000", 2024-09-18T00:18:27.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472411+0000", 2024-09-18T00:18:27.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472412+0000", 2024-09-18T00:18:27.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4529999999999999e-06 2024-09-18T00:18:27.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472420+0000", 2024-09-18T00:18:27.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3599999999999996e-06 2024-09-18T00:18:27.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472440+0000", 2024-09-18T00:18:27.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9539999999999999e-05 2024-09-18T00:18:27.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472456+0000", 2024-09-18T00:18:27.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5693e-05 2024-09-18T00:18:27.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.506473+0000", 2024-09-18T00:18:27.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034017210999999999 2024-09-18T00:18:27.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.506546+0000", 2024-09-18T00:18:27.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3208999999999997e-05 2024-09-18T00:18:27.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.506630+0000", 2024-09-18T00:18:27.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4281000000000002e-05 2024-09-18T00:18:27.056 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.056 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.056 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.056 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6290 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 10813440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.056 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.472475+0000", 2024-09-18T00:18:27.057 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.5490483639999999, 2024-09-18T00:18:27.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034338691999999997, 2024-09-18T00:18:27.057 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.057 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.057 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.057 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.058 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.058 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.058 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6290 2024-09-18T00:18:27.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.058 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472475+0000", 2024-09-18T00:18:27.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472475+0000", 2024-09-18T00:18:27.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472477+0000", 2024-09-18T00:18:27.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3599999999999999e-06 2024-09-18T00:18:27.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472487+0000", 2024-09-18T00:18:27.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0033e-05 2024-09-18T00:18:27.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472528+0000", 2024-09-18T00:18:27.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1270000000000003e-05 2024-09-18T00:18:27.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472545+0000", 2024-09-18T00:18:27.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7053000000000001e-05 2024-09-18T00:18:27.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.506671+0000", 2024-09-18T00:18:27.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034126093000000003 2024-09-18T00:18:27.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.506732+0000", 2024-09-18T00:18:27.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0637000000000002e-05 2024-09-18T00:18:27.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.506814+0000", 2024-09-18T00:18:27.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2245999999999998e-05 2024-09-18T00:18:27.065 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.065 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.065 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.066 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6291 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 11599872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.066 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.472566+0000", 2024-09-18T00:18:27.066 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.548957208, 2024-09-18T00:18:27.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034472661000000002, 2024-09-18T00:18:27.066 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.066 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.067 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.067 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6291 2024-09-18T00:18:27.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.067 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472566+0000", 2024-09-18T00:18:27.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472566+0000", 2024-09-18T00:18:27.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472568+0000", 2024-09-18T00:18:27.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.48e-06 2024-09-18T00:18:27.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472575+0000", 2024-09-18T00:18:27.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.754e-06 2024-09-18T00:18:27.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472594+0000", 2024-09-18T00:18:27.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9752999999999999e-05 2024-09-18T00:18:27.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472609+0000", 2024-09-18T00:18:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5023e-05 2024-09-18T00:18:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.506852+0000", 2024-09-18T00:18:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034242284999999997 2024-09-18T00:18:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.506917+0000", 2024-09-18T00:18:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5346999999999996e-05 2024-09-18T00:18:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.507039+0000", 2024-09-18T00:18:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000122019 2024-09-18T00:18:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6292 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 12386304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.472629+0000", 2024-09-18T00:18:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.5488948950000001, 2024-09-18T00:18:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034690047000000002, 2024-09-18T00:18:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6292 2024-09-18T00:18:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472629+0000", 2024-09-18T00:18:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472629+0000", 2024-09-18T00:18:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472630+0000", 2024-09-18T00:18:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.553e-06 2024-09-18T00:18:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472636+0000", 2024-09-18T00:18:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8329999999999999e-06 2024-09-18T00:18:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472659+0000", 2024-09-18T00:18:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3070000000000001e-05 2024-09-18T00:18:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.472674+0000", 2024-09-18T00:18:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.526e-05 2024-09-18T00:18:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.507081+0000", 2024-09-18T00:18:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034406401000000003 2024-09-18T00:18:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.507174+0000", 2024-09-18T00:18:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2837999999999994e-05 2024-09-18T00:18:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.507319+0000", 2024-09-18T00:18:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014509200000000001 2024-09-18T00:18:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6342 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.475394+0000", 2024-09-18T00:18:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.5461296309999999, 2024-09-18T00:18:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034143398999999998, 2024-09-18T00:18:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6342 2024-09-18T00:18:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475394+0000", 2024-09-18T00:18:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475394+0000", 2024-09-18T00:18:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475395+0000", 2024-09-18T00:18:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:18:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475400+0000", 2024-09-18T00:18:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0170000000000004e-06 2024-09-18T00:18:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475417+0000", 2024-09-18T00:18:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6830999999999999e-05 2024-09-18T00:18:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475430+0000", 2024-09-18T00:18:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2622999999999999e-05 2024-09-18T00:18:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.509291+0000", 2024-09-18T00:18:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033861731999999999 2024-09-18T00:18:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.509396+0000", 2024-09-18T00:18:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104889 2024-09-18T00:18:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.509537+0000", 2024-09-18T00:18:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014120699999999999 2024-09-18T00:18:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6343 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.475450+0000", 2024-09-18T00:18:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.5460740149999999, 2024-09-18T00:18:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034405660999999997, 2024-09-18T00:18:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6343 2024-09-18T00:18:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475450+0000", 2024-09-18T00:18:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475450+0000", 2024-09-18T00:18:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475451+0000", 2024-09-18T00:18:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.043e-06 2024-09-18T00:18:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475458+0000", 2024-09-18T00:18:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.849e-06 2024-09-18T00:18:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475474+0000", 2024-09-18T00:18:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6322999999999999e-05 2024-09-18T00:18:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475487+0000", 2024-09-18T00:18:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3182e-05 2024-09-18T00:18:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.509583+0000", 2024-09-18T00:18:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034096173 2024-09-18T00:18:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.509698+0000", 2024-09-18T00:18:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000114465 2024-09-18T00:18:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.509855+0000", 2024-09-18T00:18:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015762600000000001 2024-09-18T00:18:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6344 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 11665408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.475504+0000", 2024-09-18T00:18:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.546019501, 2024-09-18T00:18:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034676089, 2024-09-18T00:18:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6344 2024-09-18T00:18:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475504+0000", 2024-09-18T00:18:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475504+0000", 2024-09-18T00:18:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475505+0000", 2024-09-18T00:18:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0950000000000001e-06 2024-09-18T00:18:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475510+0000", 2024-09-18T00:18:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1000000000000003e-06 2024-09-18T00:18:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475526+0000", 2024-09-18T00:18:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6121999999999999e-05 2024-09-18T00:18:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475539+0000", 2024-09-18T00:18:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2472e-05 2024-09-18T00:18:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.509899+0000", 2024-09-18T00:18:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034359558999999998 2024-09-18T00:18:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.510012+0000", 2024-09-18T00:18:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000113964 2024-09-18T00:18:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.510180+0000", 2024-09-18T00:18:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000167777 2024-09-18T00:18:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6345 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 12451840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.475555+0000", 2024-09-18T00:18:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.5459682180000001, 2024-09-18T00:18:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034961043999999997, 2024-09-18T00:18:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6345 2024-09-18T00:18:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475555+0000", 2024-09-18T00:18:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475555+0000", 2024-09-18T00:18:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475557+0000", 2024-09-18T00:18:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2139999999999999e-06 2024-09-18T00:18:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475562+0000", 2024-09-18T00:18:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2660000000000001e-06 2024-09-18T00:18:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475578+0000", 2024-09-18T00:18:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5753999999999999e-05 2024-09-18T00:18:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.475590+0000", 2024-09-18T00:18:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2714000000000001e-05 2024-09-18T00:18:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.510234+0000", 2024-09-18T00:18:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034643548000000003 2024-09-18T00:18:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.510362+0000", 2024-09-18T00:18:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000128455 2024-09-18T00:18:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.510516+0000", 2024-09-18T00:18:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000154093 2024-09-18T00:18:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6386 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.477251+0000", 2024-09-18T00:18:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.5442722600000001, 2024-09-18T00:18:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034027248000000003, 2024-09-18T00:18:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6386 2024-09-18T00:18:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477251+0000", 2024-09-18T00:18:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477251+0000", 2024-09-18T00:18:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477253+0000", 2024-09-18T00:18:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.623e-06 2024-09-18T00:18:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477261+0000", 2024-09-18T00:18:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0269999999999995e-06 2024-09-18T00:18:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477282+0000", 2024-09-18T00:18:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0933000000000001e-05 2024-09-18T00:18:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477300+0000", 2024-09-18T00:18:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8366e-05 2024-09-18T00:18:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.511100+0000", 2024-09-18T00:18:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033799639999999999 2024-09-18T00:18:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.511184+0000", 2024-09-18T00:18:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4012999999999997e-05 2024-09-18T00:18:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.511279+0000", 2024-09-18T00:18:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4646000000000002e-05 2024-09-18T00:18:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6387 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.477348+0000", 2024-09-18T00:18:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.5441755779999999, 2024-09-18T00:18:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034155459999999999, 2024-09-18T00:18:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6387 2024-09-18T00:18:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477348+0000", 2024-09-18T00:18:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477348+0000", 2024-09-18T00:18:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477350+0000", 2024-09-18T00:18:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.77e-06 2024-09-18T00:18:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477358+0000", 2024-09-18T00:18:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9030000000000004e-06 2024-09-18T00:18:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477379+0000", 2024-09-18T00:18:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0993999999999999e-05 2024-09-18T00:18:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477398+0000", 2024-09-18T00:18:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.931e-05 2024-09-18T00:18:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.511313+0000", 2024-09-18T00:18:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033914788000000001 2024-09-18T00:18:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.511394+0000", 2024-09-18T00:18:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1485999999999999e-05 2024-09-18T00:18:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.511504+0000", 2024-09-18T00:18:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109209 2024-09-18T00:18:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6389 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 8585216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.477942+0000", 2024-09-18T00:18:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.5435820090000001, 2024-09-18T00:18:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034359023000000002, 2024-09-18T00:18:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6389 2024-09-18T00:18:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477942+0000", 2024-09-18T00:18:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477942+0000", 2024-09-18T00:18:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477943+0000", 2024-09-18T00:18:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5e-06 2024-09-18T00:18:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477950+0000", 2024-09-18T00:18:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9700000000000002e-06 2024-09-18T00:18:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477971+0000", 2024-09-18T00:18:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.071e-05 2024-09-18T00:18:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.477987+0000", 2024-09-18T00:18:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6480000000000001e-05 2024-09-18T00:18:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.512059+0000", 2024-09-18T00:18:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034072005000000002 2024-09-18T00:18:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.512163+0000", 2024-09-18T00:18:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103634 2024-09-18T00:18:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.512301+0000", 2024-09-18T00:18:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013772400000000001 2024-09-18T00:18:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6390 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 9371648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.478007+0000", 2024-09-18T00:18:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.5435166229999999, 2024-09-18T00:18:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034853740000000001, 2024-09-18T00:18:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6390 2024-09-18T00:18:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478007+0000", 2024-09-18T00:18:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478007+0000", 2024-09-18T00:18:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478008+0000", 2024-09-18T00:18:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.347e-06 2024-09-18T00:18:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478014+0000", 2024-09-18T00:18:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7200000000000003e-06 2024-09-18T00:18:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478038+0000", 2024-09-18T00:18:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4219999999999999e-05 2024-09-18T00:18:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478054+0000", 2024-09-18T00:18:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5823000000000001e-05 2024-09-18T00:18:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.512565+0000", 2024-09-18T00:18:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034510961 2024-09-18T00:18:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.512683+0000", 2024-09-18T00:18:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000118281 2024-09-18T00:18:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.512861+0000", 2024-09-18T00:18:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000177388 2024-09-18T00:18:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6391 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.478074+0000", 2024-09-18T00:18:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.5434497060000001, 2024-09-18T00:18:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035040340000000003, 2024-09-18T00:18:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6391 2024-09-18T00:18:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478074+0000", 2024-09-18T00:18:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478074+0000", 2024-09-18T00:18:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478075+0000", 2024-09-18T00:18:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3060000000000001e-06 2024-09-18T00:18:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478081+0000", 2024-09-18T00:18:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1800000000000001e-06 2024-09-18T00:18:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478100+0000", 2024-09-18T00:18:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8943000000000001e-05 2024-09-18T00:18:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478115+0000", 2024-09-18T00:18:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5027e-05 2024-09-18T00:18:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.512915+0000", 2024-09-18T00:18:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034799783000000001 2024-09-18T00:18:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.513005+0000", 2024-09-18T00:18:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0053e-05 2024-09-18T00:18:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.513114+0000", 2024-09-18T00:18:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109048 2024-09-18T00:18:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6392 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.478161+0000", 2024-09-18T00:18:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.5433627539999999, 2024-09-18T00:18:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035291134000000002, 2024-09-18T00:18:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6392 2024-09-18T00:18:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478161+0000", 2024-09-18T00:18:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478161+0000", 2024-09-18T00:18:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478170+0000", 2024-09-18T00:18:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8899999999999996e-06 2024-09-18T00:18:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478178+0000", 2024-09-18T00:18:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2069999999999995e-06 2024-09-18T00:18:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478204+0000", 2024-09-18T00:18:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6282999999999999e-05 2024-09-18T00:18:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478221+0000", 2024-09-18T00:18:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6572999999999999e-05 2024-09-18T00:18:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.513173+0000", 2024-09-18T00:18:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034952023999999998 2024-09-18T00:18:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.513295+0000", 2024-09-18T00:18:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121731 2024-09-18T00:18:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.513452+0000", 2024-09-18T00:18:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015742600000000001 2024-09-18T00:18:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6393 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 11730944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.478244+0000", 2024-09-18T00:18:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.5432795850000001, 2024-09-18T00:18:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035427317999999999, 2024-09-18T00:18:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6393 2024-09-18T00:18:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478244+0000", 2024-09-18T00:18:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478244+0000", 2024-09-18T00:18:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478245+0000", 2024-09-18T00:18:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.42e-06 2024-09-18T00:18:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478252+0000", 2024-09-18T00:18:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5069999999999997e-06 2024-09-18T00:18:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478274+0000", 2024-09-18T00:18:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1733e-05 2024-09-18T00:18:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478290+0000", 2024-09-18T00:18:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6160000000000001e-05 2024-09-18T00:18:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.513501+0000", 2024-09-18T00:18:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035211564000000001 2024-09-18T00:18:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.513577+0000", 2024-09-18T00:18:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5749999999999998e-05 2024-09-18T00:18:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.513671+0000", 2024-09-18T00:18:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4184000000000005e-05 2024-09-18T00:18:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6394 3.1a 3:5b9389ce:::rbd_data.112a960351d8.0000000000000001:6 [sparse-read 12517376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.478311+0000", 2024-09-18T00:18:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.5432127420000001, 2024-09-18T00:18:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035692582, 2024-09-18T00:18:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6394 2024-09-18T00:18:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478311+0000", 2024-09-18T00:18:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478311+0000", 2024-09-18T00:18:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478312+0000", 2024-09-18T00:18:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3170000000000001e-06 2024-09-18T00:18:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478319+0000", 2024-09-18T00:18:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3069999999999999e-06 2024-09-18T00:18:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478343+0000", 2024-09-18T00:18:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4136000000000001e-05 2024-09-18T00:18:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.478358+0000", 2024-09-18T00:18:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5607000000000002e-05 2024-09-18T00:18:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.513726+0000", 2024-09-18T00:18:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035367523999999997 2024-09-18T00:18:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.513847+0000", 2024-09-18T00:18:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012095399999999999 2024-09-18T00:18:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.514004+0000", 2024-09-18T00:18:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000156737 2024-09-18T00:18:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:27.155 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:27.155 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:27.156 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:27.156 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:18:27.157 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:18:27.157 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:18:27.158 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:27.158 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:27.158 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:27.159 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:27.159 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:27.159 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:27.160 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:27.151+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:27.163 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:18:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:18:27.182 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:27.183 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:27.200 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:27.200 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:27.251 INFO:teuthology.orchestra.run.smithi097.stdout:1217 trunc from 0xe9d1540 to 0x4c856c0 2024-09-18T00:18:27.256 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:27.255+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:27.276 INFO:teuthology.orchestra.run.smithi097.stdout:1218 punch from 0x615300 to 0x624c2f, (0xf92f bytes) 2024-09-18T00:18:27.276 INFO:teuthology.orchestra.run.smithi097.stdout:1219 read 0x14a7ff1 thru 0x14b6353 (0xe363 bytes) 2024-09-18T00:18:27.292 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:27.292 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:27.300 INFO:teuthology.orchestra.run.smithi097.stdout:1220 write 0xc4a2271 thru 0xc4a8ed2 (0x6c62 bytes) 2024-09-18T00:18:27.307 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:18:27.308 INFO:teuthology.orchestra.run.smithi097.stdout:1223 trunc from 0xc4a8ed3 to 0xa58694f 2024-09-18T00:18:27.328 INFO:teuthology.orchestra.run.smithi097.stdout:1224 writesame 0xaad77f3 thru 0xaae7703 data_size 0x17(0xff11 bytes) 2024-09-18T00:18:27.328 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:27.328 INFO:teuthology.orchestra.run.smithi097.stdout:1226 read 0x2ddb2db thru 0x2ddbe6a (0xb90 bytes) 2024-09-18T00:18:27.329 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:27.329 INFO:teuthology.orchestra.run.smithi097.stdout:1230 write 0x37306e1 thru 0x373f92d (0xf24d bytes) 2024-09-18T00:18:27.336 INFO:teuthology.orchestra.run.smithi097.stdout:1231 read 0x811d703 thru 0x8122077 (0x4975 bytes) 2024-09-18T00:18:27.337 INFO:teuthology.orchestra.run.smithi097.stdout:1232 read 0x9445882 thru 0x9448ed0 (0x364f bytes) 2024-09-18T00:18:27.337 INFO:teuthology.orchestra.run.smithi097.stdout:1234 punch from 0x4ccdba2 to 0x4cd8b83, (0xafe1 bytes) 2024-09-18T00:18:27.337 INFO:teuthology.orchestra.run.smithi097.stdout:1235 punch from 0x8e7d2d1 to 0x8e83d83, (0x6ab2 bytes) 2024-09-18T00:18:27.348 INFO:teuthology.orchestra.run.smithi097.stdout:1236 trunc from 0xaae7704 to 0x25eac90 2024-09-18T00:18:27.357 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:27.355+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:27.405 INFO:teuthology.orchestra.run.smithi097.stdout:1237 write 0xd5c01fc thru 0xd5cfd5d (0xfb62 bytes) 2024-09-18T00:18:27.412 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:27.412 INFO:teuthology.orchestra.run.smithi097.stdout:1241 writesame 0x91590b5 thru 0x9168389 data_size 0x1(0xf2d5 bytes) 2024-09-18T00:18:27.457 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:27.457 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:27.458 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:27.455+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:27.485 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:18:27.518 INFO:teuthology.orchestra.run.smithi097.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T00:18:27.537 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:18:27.546 INFO:teuthology.orchestra.run.smithi097.stdout:1242 trunc from 0xd5cfd5e to 0xe33b33 2024-09-18T00:18:27.556 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T00:18:27.558 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:27.555+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:27.569 INFO:teuthology.orchestra.run.smithi097.stdout:1243 read 0x2af8a thru 0x34792 (0x9809 bytes) 2024-09-18T00:18:27.570 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:27.577 INFO:teuthology.orchestra.run.smithi097.stdout:1245 write 0x113de30 thru 0x114acac (0xce7d bytes) 2024-09-18T00:18:27.587 INFO:teuthology.orchestra.run.smithi097.stdout:1246 punch from 0x65e76 to 0x75538, (0xf6c2 bytes) 2024-09-18T00:18:27.588 INFO:teuthology.orchestra.run.smithi097.stdout:1247 punch from 0xc79207 to 0xc87c87, (0xea80 bytes) 2024-09-18T00:18:27.606 INFO:teuthology.orchestra.run.smithi097.stdout:1248 write 0x7c387a9 thru 0x7c3ca8a (0x42e2 bytes) 2024-09-18T00:18:27.611 INFO:teuthology.orchestra.run.smithi097.stdout:1249 write 0x1806e1b thru 0x180cab6 (0x5c9c bytes) 2024-09-18T00:18:27.612 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:27.612 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:27.616 INFO:teuthology.orchestra.run.smithi097.stdout:1250 write 0x38b184e thru 0x38bbea9 (0xa65c bytes) 2024-09-18T00:18:27.625 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:27.625 INFO:teuthology.orchestra.run.smithi097.stdout:1252 write 0x6c19525 thru 0x6c1f783 (0x625f bytes) 2024-09-18T00:18:27.627 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:18:27.641 INFO:teuthology.orchestra.run.smithi097.stdout:1255 trunc from 0x7c3ca8b to 0xb0d414 2024-09-18T00:18:27.659 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:27.655+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:27.664 INFO:teuthology.orchestra.run.smithi097.stdout:1256 trunc from 0xb0d414 to 0x1b5da01 2024-09-18T00:18:27.672 INFO:teuthology.orchestra.run.smithi097.stdout:1260 read 0x18b464e thru 0x18bd57e (0x8f31 bytes) 2024-09-18T00:18:27.679 INFO:teuthology.orchestra.run.smithi097.stdout:1261 write 0x1dfb6ed thru 0x1dfd10f (0x1a23 bytes) 2024-09-18T00:18:27.683 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:27.691 INFO:teuthology.orchestra.run.smithi097.stdout:1263 writesame 0x2d118b2 thru 0x2d1305e data_size 0x13(0x17ad bytes) 2024-09-18T00:18:27.691 INFO:teuthology.orchestra.run.smithi097.stdout:1264 punch from 0x535791 to 0x53a11f, (0x498e bytes) 2024-09-18T00:18:27.691 INFO:teuthology.orchestra.run.smithi097.stdout:1265 punch from 0x258da40 to 0x25923da, (0x499a bytes) 2024-09-18T00:18:27.691 INFO:teuthology.orchestra.run.smithi097.stdout:1266 read 0xae4b03 thru 0xae6c34 (0x2132 bytes) 2024-09-18T00:18:27.697 INFO:teuthology.orchestra.run.smithi097.stdout:1267 write 0x311cd20 thru 0x312340c (0x66ed bytes) 2024-09-18T00:18:27.702 INFO:teuthology.orchestra.run.smithi097.stdout:1269 read 0x1332507 thru 0x133623a (0x3d34 bytes) 2024-09-18T00:18:27.702 INFO:teuthology.orchestra.run.smithi097.stdout:1270 read 0x1b62cfb thru 0x1b6778e (0x4a94 bytes) 2024-09-18T00:18:27.702 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:27.702 INFO:teuthology.orchestra.run.smithi097.stdout:1272 read 0x14f27af thru 0x14ff1f5 (0xca47 bytes) 2024-09-18T00:18:27.702 INFO:teuthology.orchestra.run.smithi097.stdout:1273 read 0x29a43ea thru 0x29ab9d1 (0x75e8 bytes) 2024-09-18T00:18:27.711 INFO:teuthology.orchestra.run.smithi097.stdout:1275 trunc from 0x312340d to 0x970f676 2024-09-18T00:18:27.717 INFO:teuthology.orchestra.run.smithi097.stdout:1276 writesame 0x21d1ff6 thru 0x21d53a9 data_size 0x6(0x33b4 bytes) 2024-09-18T00:18:27.722 INFO:teuthology.orchestra.run.smithi097.stdout:1277 punch from 0x647c33e to 0x648b5b3, (0xf275 bytes) 2024-09-18T00:18:27.723 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:27.723 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:27.723 INFO:teuthology.orchestra.run.smithi097.stdout:1280 read 0x38b6759 thru 0x38c37d4 (0xd07c bytes) 2024-09-18T00:18:27.734 INFO:teuthology.orchestra.run.smithi097.stdout:1281 write 0xd8a1c0c thru 0xd8afd49 (0xe13e bytes) 2024-09-18T00:18:27.745 INFO:teuthology.orchestra.run.smithi097.stdout:1283 trunc from 0xd8afd4a to 0x8b4e270 2024-09-18T00:18:27.760 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:27.755+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:27.769 INFO:teuthology.orchestra.run.smithi097.stdout:1285 writesame 0xb50536f thru 0xb50bea3 data_size 0xb(0x6b35 bytes) 2024-09-18T00:18:27.769 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:27.769 INFO:teuthology.orchestra.run.smithi097.stdout:1287 writesame 0x5e3f2de thru 0x5e413da data_size 0x5(0x20fd bytes) 2024-09-18T00:18:27.774 INFO:teuthology.orchestra.run.smithi097.stdout:1288 read 0x4c6ad8f thru 0x4c77511 (0xc783 bytes) 2024-09-18T00:18:27.774 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:27.774 INFO:teuthology.orchestra.run.smithi097.stdout:1290 write 0x6b9f94d thru 0x6bab68d (0xbd41 bytes) 2024-09-18T00:18:27.781 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:27.785 INFO:teuthology.orchestra.run.smithi097.stdout:1293 trunc from 0xb50bea4 to 0x8144b8a 2024-09-18T00:18:27.792 INFO:teuthology.orchestra.run.smithi097.stdout:1295 write 0x3bbe670 thru 0x3bc38d7 (0x5268 bytes) 2024-09-18T00:18:27.795 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:27.795 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:27.799 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:27.799 INFO:teuthology.orchestra.run.smithi097.stdout:1300 read 0x7ea8668 thru 0x7eaa4a1 (0x1e3a bytes) 2024-09-18T00:18:27.799 INFO:teuthology.orchestra.run.smithi097.stdout:1301 punch from 0x4b37b30 to 0x4b38f60, (0x1430 bytes) 2024-09-18T00:18:27.799 INFO:teuthology.orchestra.run.smithi097.stdout:1302 read 0x16408e thru 0x1654c9 (0x143c bytes) 2024-09-18T00:18:27.799 INFO:teuthology.orchestra.run.smithi097.stdout:1303 write 0x7b57158 thru 0x7b5a072 (0x2f1b bytes) 2024-09-18T00:18:27.812 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:18:27.817 INFO:teuthology.orchestra.run.smithi097.stdout:1305 write 0xb05dbbc thru 0xb06b78a (0xdbcf bytes) 2024-09-18T00:18:27.836 INFO:teuthology.orchestra.run.smithi097.stdout:1306 write 0xe0bc916 thru 0xe0c4008 (0x76f3 bytes) 2024-09-18T00:18:27.841 INFO:teuthology.orchestra.run.smithi097.stdout:1308 read 0x3471a0c thru 0x3475b38 (0x412d bytes) 2024-09-18T00:18:27.842 INFO:teuthology.orchestra.run.smithi097.stdout:1309 write 0x84e8bb0 thru 0x84eda55 (0x4ea6 bytes) 2024-09-18T00:18:27.846 INFO:teuthology.orchestra.run.smithi097.stdout:1310 read 0x8efc442 thru 0x8f00dd7 (0x4996 bytes) 2024-09-18T00:18:27.846 INFO:teuthology.orchestra.run.smithi097.stdout:1311 punch from 0xae16310 to 0xae1cd01, (0x69f1 bytes) 2024-09-18T00:18:27.846 INFO:teuthology.orchestra.run.smithi097.stdout:1312 write 0x22f385d thru 0x22fe5a8 (0xad4c bytes) 2024-09-18T00:18:27.854 INFO:teuthology.orchestra.run.smithi097.stdout:1313 punch from 0x89f4718 to 0x89fd220, (0x8b08 bytes) 2024-09-18T00:18:27.854 INFO:teuthology.orchestra.run.smithi097.stdout:1314 writesame 0xb7aaf02 thru 0xb7b258c data_size 0x1(0x768b bytes) 2024-09-18T00:18:27.861 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:27.859+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:27.902 INFO:teuthology.orchestra.run.smithi097.stdout:1317 trunc from 0xe0c4009 to 0xe998a70 2024-09-18T00:18:27.910 INFO:teuthology.orchestra.run.smithi097.stdout:1318 punch from 0x3cfc1e2 to 0x3d0030e, (0x412c bytes) 2024-09-18T00:18:27.910 INFO:teuthology.orchestra.run.smithi097.stdout:1320 write 0x3043202 thru 0x304b6fc (0x84fb bytes) 2024-09-18T00:18:27.918 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:27.918 INFO:teuthology.orchestra.run.smithi097.stdout:1322 write 0xd1d29fb thru 0xd1e0a7a (0xe080 bytes) 2024-09-18T00:18:27.932 INFO:teuthology.orchestra.run.smithi097.stdout:1323 writesame 0xa1c2bc thru 0xa2ad31 data_size 0x1(0xea76 bytes) 2024-09-18T00:18:27.959 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:27.959 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:27.962 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:27.959+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:27.982 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:18:28.045 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:28.046 INFO:teuthology.orchestra.run.smithi097.stdout:1325 writesame 0x1628735 thru 0x162d3c1 data_size 0x1(0x4c8d bytes) 2024-09-18T00:18:28.063 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:28.059+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:28.075 INFO:teuthology.orchestra.run.smithi097.stdout:1326 writesame 0x637ca20 thru 0x6384d0d data_size 0x1(0x82ee bytes) 2024-09-18T00:18:28.108 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:28.108 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:28.125 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:18:28.142 INFO:teuthology.orchestra.run.smithi097.stdout:1327 read 0xdcc8932 thru 0xdcd00ee (0x77bd bytes) 2024-09-18T00:18:28.142 INFO:teuthology.orchestra.run.smithi097.stdout:1328 punch from 0x755bbe8 to 0x755f158, (0x3570 bytes) 2024-09-18T00:18:28.143 INFO:teuthology.orchestra.run.smithi097.stdout:1329 writesame 0x82cbea7 thru 0x82d79ff data_size 0x49(0xbb59 bytes) 2024-09-18T00:18:28.151 INFO:teuthology.orchestra.run.smithi097.stdout:1331 write 0x4fbabdb thru 0x4fc847c (0xd8a2 bytes) 2024-09-18T00:18:28.162 INFO:teuthology.orchestra.run.smithi097.stdout:1332 read 0x3ef8e6d thru 0x3f01602 (0x8796 bytes) 2024-09-18T00:18:28.162 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:28.163 INFO:teuthology.orchestra.run.smithi097.stdout:1335 read 0xd4eb7f5 thru 0xd4f3542 (0x7d4e bytes) 2024-09-18T00:18:28.163 INFO:teuthology.orchestra.run.smithi097.stdout:1336 read 0xbe053d8 thru 0xbe0ec4e (0x9877 bytes) 2024-09-18T00:18:28.163 INFO:teuthology.orchestra.run.smithi097.stdout:1337 writesame 0x64779c6 thru 0x6480316 data_size 0x1(0x8951 bytes) 2024-09-18T00:18:28.164 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:28.159+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:28.265 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:28.263+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:28.266 INFO:teuthology.orchestra.run.smithi097.stdout:1338 write 0x3563b67 thru 0x35655a2 (0x1a3c bytes) 2024-09-18T00:18:28.295 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:28.295 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:28.298 INFO:teuthology.orchestra.run.smithi097.stdout:1339 trunc from 0xe998a70 to 0x4848f43 2024-09-18T00:18:28.313 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:18:28.351 INFO:teuthology.orchestra.run.smithi097.stdout:1340 write 0xce9274e thru 0xcea2557 (0xfe0a bytes) 2024-09-18T00:18:28.361 INFO:teuthology.orchestra.run.smithi097.stdout:1342 punch from 0x795ebc1 to 0x7968924, (0x9d63 bytes) 2024-09-18T00:18:28.361 INFO:teuthology.orchestra.run.smithi097.stdout:1344 punch from 0xb02b689 to 0xb03a9c9, (0xf340 bytes) 2024-09-18T00:18:28.361 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:28.361 INFO:teuthology.orchestra.run.smithi097.stdout:1346 read 0xc7a5628 thru 0xc7ab5bc (0x5f95 bytes) 2024-09-18T00:18:28.361 INFO:teuthology.orchestra.run.smithi097.stdout:1347 read 0x2abea55 thru 0x2ac2a4f (0x3ffb bytes) 2024-09-18T00:18:28.361 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:28.361 INFO:teuthology.orchestra.run.smithi097.stdout:1349 write 0x6a9f8a8 thru 0x6aacfdf (0xd738 bytes) 2024-09-18T00:18:28.365 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:28.359+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:28.374 INFO:teuthology.orchestra.run.smithi097.stdout:1351 write 0x751e26f thru 0x752d5cb (0xf35d bytes) 2024-09-18T00:18:28.388 INFO:teuthology.orchestra.run.smithi097.stdout:1352 punch from 0x508a17d to 0x50961dd, (0xc060 bytes) 2024-09-18T00:18:28.388 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:28.389 INFO:teuthology.orchestra.run.smithi097.stdout:1357 read 0x5def1f9 thru 0x5df522d (0x6035 bytes) 2024-09-18T00:18:28.403 INFO:teuthology.orchestra.run.smithi097.stdout:1358 write 0xe347f63 thru 0xe34d5cb (0x5669 bytes) 2024-09-18T00:18:28.410 INFO:teuthology.orchestra.run.smithi097.stdout:1359 read 0xb8bc1c9 thru 0xb8cad88 (0xebc0 bytes) 2024-09-18T00:18:28.420 INFO:teuthology.orchestra.run.smithi097.stdout:1360 trunc from 0xe34d5cc to 0xab53058 2024-09-18T00:18:28.439 INFO:teuthology.orchestra.run.smithi097.stdout:1361 punch from 0x831945b to 0x832072d, (0x72d2 bytes) 2024-09-18T00:18:28.439 INFO:teuthology.orchestra.run.smithi097.stdout:1362 writesame 0x5a550cd thru 0x5a601d3 data_size 0x1(0xb107 bytes) 2024-09-18T00:18:28.440 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:28.440 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:28.455 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:18:28.467 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:28.463+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:28.551 INFO:teuthology.orchestra.run.smithi097.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T00:18:28.551 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is set 2024-09-18T00:18:28.555 INFO:teuthology.orchestra.run.smithi097.stdout:1363 trunc from 0xab53058 to 0x1eee4d3 2024-09-18T00:18:28.568 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:28.563+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:28.580 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T00:18:28.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:18:28.584 INFO:teuthology.orchestra.run.smithi097.stdout:1364 trunc from 0x1eee4d3 to 0x7a6dfbc 2024-09-18T00:18:28.591 INFO:teuthology.orchestra.run.smithi097.stdout:1365 write 0x4fdf8a7 thru 0x4fee501 (0xec5b bytes) 2024-09-18T00:18:28.600 INFO:teuthology.orchestra.run.smithi097.stdout:1366 write 0x7168bfe thru 0x7168cb3 (0xb6 bytes) 2024-09-18T00:18:28.609 INFO:teuthology.orchestra.run.smithi097.stdout:1367 writesame 0x7ebe71b thru 0x7ec16fc data_size 0x1b(0x2fe2 bytes) 2024-09-18T00:18:28.609 INFO:teuthology.orchestra.run.smithi097.stdout:1368 write 0x54aa8e0 thru 0x54b7a60 (0xd181 bytes) 2024-09-18T00:18:28.615 INFO:teuthology.orchestra.run.smithi097.stdout:1370 punch from 0x7c82e23 to 0x7c83179, (0x356 bytes) 2024-09-18T00:18:28.616 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:28.616 INFO:teuthology.orchestra.run.smithi097.stdout:1372 write 0x50da21c thru 0x50dafb9 (0xd9e bytes) 2024-09-18T00:18:28.618 INFO:teuthology.orchestra.run.smithi097.stdout:1373 read 0x69477df thru 0x6955708 (0xdf2a bytes) 2024-09-18T00:18:28.627 INFO:teuthology.orchestra.run.smithi097.stdout:1374 write 0x9000be8 thru 0x900582c (0x4c45 bytes) 2024-09-18T00:18:28.631 INFO:teuthology.orchestra.run.smithi097.stdout:1375 read 0x58a8f93 thru 0x58b1ae3 (0x8b51 bytes) 2024-09-18T00:18:28.631 INFO:teuthology.orchestra.run.smithi097.stdout:1376 write 0x1e9fe1e thru 0x1ea45c7 (0x47aa bytes) 2024-09-18T00:18:28.635 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:28.635 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:28.636 INFO:teuthology.orchestra.run.smithi097.stdout:1379 read 0x8f83416 thru 0x8f83d89 (0x974 bytes) 2024-09-18T00:18:28.649 INFO:teuthology.orchestra.run.smithi097.stdout:1380 writesame 0xb35c626 thru 0xb3657c4 data_size 0x1(0x919f bytes) 2024-09-18T00:18:28.649 INFO:teuthology.orchestra.run.smithi097.stdout:1381 write 0x53c55e3 thru 0x53d19f2 (0xc410 bytes) 2024-09-18T00:18:28.650 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:18:28.657 INFO:teuthology.orchestra.run.smithi097.stdout:1383 read 0x4fd7b68 thru 0x4fe0751 (0x8bea bytes) 2024-09-18T00:18:28.658 INFO:teuthology.orchestra.run.smithi097.stdout:1385 read 0x5ea5dda thru 0x5ea8f75 (0x319c bytes) 2024-09-18T00:18:28.658 INFO:teuthology.orchestra.run.smithi097.stdout:1386 read 0xb139387 thru 0xb14035d (0x6fd7 bytes) 2024-09-18T00:18:28.661 INFO:teuthology.orchestra.run.smithi097.stdout:1387 trunc from 0xb3657c5 to 0xd66c22d 2024-09-18T00:18:28.668 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:28.667+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:28.669 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:28.670 INFO:teuthology.orchestra.run.smithi097.stdout:1391 write 0x48132f4 thru 0x481e478 (0xb185 bytes) 2024-09-18T00:18:28.679 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:28.679 INFO:teuthology.orchestra.run.smithi097.stdout:1393 read 0xc583fd9 thru 0xc58c678 (0x86a0 bytes) 2024-09-18T00:18:28.680 INFO:teuthology.orchestra.run.smithi097.stdout:1394 write 0xb696ef7 thru 0xb6a3044 (0xc14e bytes) 2024-09-18T00:18:28.698 INFO:teuthology.orchestra.run.smithi097.stdout:1397 trunc from 0xd66c22d to 0x73ebda3 2024-09-18T00:18:28.721 INFO:teuthology.orchestra.run.smithi097.stdout:1398 trunc from 0x73ebda3 to 0x2155b33 2024-09-18T00:18:28.738 INFO:teuthology.orchestra.run.smithi097.stdout:1399 read 0x14b4e84 thru 0x14c1d7f (0xcefc bytes) 2024-09-18T00:18:28.756 INFO:teuthology.orchestra.run.smithi097.stdout:1402 write 0x57452ad thru 0x5745335 (0x89 bytes) 2024-09-18T00:18:28.763 INFO:teuthology.orchestra.run.smithi097.stdout:1403 trunc from 0x5745336 to 0x39664b9 2024-09-18T00:18:28.769 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:18:28.767+0000 7f20dceb7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26437) UID: 0 2024-09-18T00:18:28.784 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:28.785 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:28.785 INFO:teuthology.orchestra.run.smithi097.stdout:1404 trunc from 0x39664b9 to 0x45aa838 2024-09-18T00:18:28.800 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:18:28.822 INFO:teuthology.orchestra.run.smithi097.stdout:1405 write 0xbf7481f thru 0xbf7e029 (0x980b bytes) 2024-09-18T00:18:28.827 INFO:teuthology.orchestra.run.smithi097.stdout:1407 punch from 0xaa33ad5 to 0xaa4250f, (0xea3a bytes) 2024-09-18T00:18:28.827 INFO:teuthology.orchestra.run.smithi097.stdout:1408 writesame 0x33d50b thru 0x340ebf data_size 0x11(0x39b5 bytes) 2024-09-18T00:18:28.837 INFO:teuthology.orchestra.run.smithi097.stdout:1409 read 0x171f860 thru 0x17203b8 (0xb59 bytes) 2024-09-18T00:18:28.840 INFO:teuthology.orchestra.run.smithi097.stdout:1412 trunc from 0xbf7e02a to 0x9b41611 2024-09-18T00:18:28.859 INFO:teuthology.orchestra.run.smithi097.stdout:1413 write 0x33e28cb thru 0x33f28b3 (0xffe9 bytes) 2024-09-18T00:18:28.870 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:28.867+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:28.872 INFO:teuthology.orchestra.run.smithi097.stdout:1414 writesame 0x44da319 thru 0x44e08b9 data_size 0x1(0x65a1 bytes) 2024-09-18T00:18:28.906 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:18:28.906 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-18T00:18:28.922 INFO:teuthology.orchestra.run.smithi097.stdout:1416 write 0x7449c9f thru 0x74535c0 (0x9922 bytes) 2024-09-18T00:18:28.926 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-09-18T00:18:28.547409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":121607588,"num_objects":111,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":0,"num_read":9272,"num_read_kb":10266,"num_write":2697,"num_write_kb":14449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1502581,"num_keys_recovered":17,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2055,"ondisk_log_size":2055,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256680,"kb_used_data":7148,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718040,"statfs":{"total":773094113280,"available":772831272960,"internally_reserved":0,"allocated":7319552,"data_stored":7057907,"data_compressed":1815151,"data_compressed_allocated":2215936,"data_compressed_original":4765008,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":64987666,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":3918,"num_read_kb":4619,"num_write":941,"num_write_kb":3396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.054241"},"pg_stats":[{"pgid":"3.1f","version":"39'24","reported_seq":93,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.291599+0000","last_change":"2024-09-18T00:18:07.410382+0000","last_active":"2024-09-18T00:18:24.291599+0000","last_peered":"2024-09-18T00:18:24.291599+0000","last_clean":"2024-09-18T00:18:24.291599+0000","last_became_active":"2024-09-18T00:18:05.411865+0000","last_became_peered":"2024-09-18T00:18:05.411865+0000","last_unstale":"2024-09-18T00:18:24.291599+0000","last_undegraded":"2024-09-18T00:18:24.291599+0000","last_fullsized":"2024-09-18T00:18:24.291599+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:06:32.938057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001658056,"stat_sum":{"num_bytes":283268,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":32,"num_write":24,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"39'24","reported_seq":75,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:23.488610+0000","last_change":"2024-09-18T00:18:07.502464+0000","last_active":"2024-09-18T00:18:23.488610+0000","last_peered":"2024-09-18T00:18:23.488610+0000","last_clean":"2024-09-18T00:18:23.488610+0000","last_became_active":"2024-09-18T00:18:05.403587+0000","last_became_peered":"2024-09-18T00:18:05.403587+0000","last_unstale":"2024-09-18T00:18:23.488610+0000","last_undegraded":"2024-09-18T00:18:23.488610+0000","last_fullsized":"2024-09-18T00:18:23.488610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:57.767230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019357549999999999,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":20,"num_write":24,"num_write_kb":204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"40'532","reported_seq":598,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:25.005922+0000","last_change":"2024-09-18T00:18:07.390275+0000","last_active":"2024-09-18T00:18:25.005922+0000","last_peered":"2024-09-18T00:18:25.005922+0000","last_clean":"2024-09-18T00:18:25.005922+0000","last_became_active":"2024-09-18T00:18:05.415210+0000","last_became_peered":"2024-09-18T00:18:05.415210+0000","last_unstale":"2024-09-18T00:18:25.005922+0000","last_undegraded":"2024-09-18T00:18:25.005922+0000","last_fullsized":"2024-09-18T00:18:25.005922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:57:00.709996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117322,"stat_sum":{"num_bytes":851985,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2670,"num_read_kb":2308,"num_write":920,"num_write_kb":1251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"41'33","reported_seq":102,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.847792+0000","last_change":"2024-09-18T00:18:07.389495+0000","last_active":"2024-09-18T00:18:24.847792+0000","last_peered":"2024-09-18T00:18:24.847792+0000","last_clean":"2024-09-18T00:18:24.847792+0000","last_became_active":"2024-09-18T00:18:05.407036+0000","last_became_peered":"2024-09-18T00:18:05.407036+0000","last_unstale":"2024-09-18T00:18:24.847792+0000","last_undegraded":"2024-09-18T00:18:24.847792+0000","last_fullsized":"2024-09-18T00:18:24.847792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:10:15.997577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055723800000000005,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":38,"num_write":33,"num_write_kb":180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"39'23","reported_seq":74,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:23.540374+0000","last_change":"2024-09-18T00:18:07.409072+0000","last_active":"2024-09-18T00:18:23.540374+0000","last_peered":"2024-09-18T00:18:23.540374+0000","last_clean":"2024-09-18T00:18:23.540374+0000","last_became_active":"2024-09-18T00:18:05.415810+0000","last_became_peered":"2024-09-18T00:18:05.415810+0000","last_unstale":"2024-09-18T00:18:23.540374+0000","last_undegraded":"2024-09-18T00:18:23.540374+0000","last_fullsized":"2024-09-18T00:18:23.540374+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:56:57.406500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059152399999999996,"stat_sum":{"num_bytes":98304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":16,"num_write":23,"num_write_kb":186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"40'25","reported_seq":367,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:25.323538+0000","last_change":"2024-09-18T00:18:07.399486+0000","last_active":"2024-09-18T00:18:25.323538+0000","last_peered":"2024-09-18T00:18:25.323538+0000","last_clean":"2024-09-18T00:18:25.323538+0000","last_became_active":"2024-09-18T00:18:05.395573+0000","last_became_peered":"2024-09-18T00:18:05.395573+0000","last_unstale":"2024-09-18T00:18:25.323538+0000","last_undegraded":"2024-09-18T00:18:25.323538+0000","last_fullsized":"2024-09-18T00:18:25.323538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:01:11.873931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020525439999999999,"stat_sum":{"num_bytes":4893811,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":292,"num_read_kb":254,"num_write":26,"num_write_kb":332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"39'27","reported_seq":88,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.985186+0000","last_change":"2024-09-18T00:18:07.393316+0000","last_active":"2024-09-18T00:18:24.985186+0000","last_peered":"2024-09-18T00:18:24.985186+0000","last_clean":"2024-09-18T00:18:24.985186+0000","last_became_active":"2024-09-18T00:18:05.403595+0000","last_became_peered":"2024-09-18T00:18:05.403595+0000","last_unstale":"2024-09-18T00:18:24.985186+0000","last_undegraded":"2024-09-18T00:18:24.985186+0000","last_fullsized":"2024-09-18T00:18:24.985186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:51:41.679476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077375,"stat_sum":{"num_bytes":1000188,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":130,"num_write":29,"num_write_kb":329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"40'16","reported_seq":67,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.183792+0000","last_change":"2024-09-18T00:18:07.500927+0000","last_active":"2024-09-18T00:18:24.183792+0000","last_peered":"2024-09-18T00:18:24.183792+0000","last_clean":"2024-09-18T00:18:24.183792+0000","last_became_active":"2024-09-18T00:18:05.406821+0000","last_became_peered":"2024-09-18T00:18:05.406821+0000","last_unstale":"2024-09-18T00:18:24.183792+0000","last_undegraded":"2024-09-18T00:18:24.183792+0000","last_fullsized":"2024-09-18T00:18:24.183792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:49:53.291639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074352700000000001,"stat_sum":{"num_bytes":16723727,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":17,"num_write":20,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"39'41","reported_seq":92,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:23.540567+0000","last_change":"2024-09-18T00:18:07.410454+0000","last_active":"2024-09-18T00:18:23.540567+0000","last_peered":"2024-09-18T00:18:23.540567+0000","last_clean":"2024-09-18T00:18:23.540567+0000","last_became_active":"2024-09-18T00:18:05.411650+0000","last_became_peered":"2024-09-18T00:18:05.411650+0000","last_unstale":"2024-09-18T00:18:23.540567+0000","last_undegraded":"2024-09-18T00:18:23.540567+0000","last_fullsized":"2024-09-18T00:18:23.540567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:10:42.627134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012546969999999999,"stat_sum":{"num_bytes":172280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":12,"num_write":41,"num_write_kb":405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"39'43","reported_seq":318,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.501211+0000","last_change":"2024-09-18T00:18:16.552071+0000","last_active":"2024-09-18T00:18:24.501211+0000","last_peered":"2024-09-18T00:18:24.501211+0000","last_clean":"2024-09-18T00:18:24.501211+0000","last_became_active":"2024-09-18T00:18:16.444631+0000","last_became_peered":"2024-09-18T00:18:16.444631+0000","last_unstale":"2024-09-18T00:18:24.501211+0000","last_undegraded":"2024-09-18T00:18:24.501211+0000","last_fullsized":"2024-09-18T00:18:24.501211+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:05.394823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015027420000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":599,"num_read_kb":523,"num_write":37,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":17,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"39'38","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.558482+0000","last_change":"2024-09-18T00:18:16.491563+0000","last_active":"2024-09-18T00:18:24.558482+0000","last_peered":"2024-09-18T00:18:24.558482+0000","last_clean":"2024-09-18T00:18:24.558482+0000","last_became_active":"2024-09-18T00:18:16.438164+0000","last_became_peered":"2024-09-18T00:18:16.438164+0000","last_unstale":"2024-09-18T00:18:24.558482+0000","last_undegraded":"2024-09-18T00:18:24.558482+0000","last_fullsized":"2024-09-18T00:18:24.558482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:54:51.185189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":28,"num_write":40,"num_write_kb":684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"39'25","reported_seq":130,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.512696+0000","last_change":"2024-09-18T00:18:16.544097+0000","last_active":"2024-09-18T00:18:24.512696+0000","last_peered":"2024-09-18T00:18:24.512696+0000","last_clean":"2024-09-18T00:18:24.512696+0000","last_became_active":"2024-09-18T00:18:16.441758+0000","last_became_peered":"2024-09-18T00:18:16.441758+0000","last_unstale":"2024-09-18T00:18:24.512696+0000","last_undegraded":"2024-09-18T00:18:24.512696+0000","last_fullsized":"2024-09-18T00:18:24.512696+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:12:34.554135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000991729,"stat_sum":{"num_bytes":272361,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":173,"num_write":25,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":103027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.496023+0000","last_change":"2024-09-18T00:18:16.445661+0000","last_active":"2024-09-18T00:18:24.496023+0000","last_peered":"2024-09-18T00:18:24.496023+0000","last_clean":"2024-09-18T00:18:24.496023+0000","last_became_active":"2024-09-18T00:18:16.444732+0000","last_became_peered":"2024-09-18T00:18:16.444732+0000","last_unstale":"2024-09-18T00:18:24.496023+0000","last_undegraded":"2024-09-18T00:18:24.496023+0000","last_fullsized":"2024-09-18T00:18:24.496023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:29.764590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089554000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"40'43","reported_seq":94,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.295646+0000","last_change":"2024-09-18T00:18:07.409625+0000","last_active":"2024-09-18T00:18:24.295646+0000","last_peered":"2024-09-18T00:18:24.295646+0000","last_clean":"2024-09-18T00:18:24.295646+0000","last_became_active":"2024-09-18T00:18:05.403736+0000","last_became_peered":"2024-09-18T00:18:05.403736+0000","last_unstale":"2024-09-18T00:18:24.295646+0000","last_undegraded":"2024-09-18T00:18:24.295646+0000","last_fullsized":"2024-09-18T00:18:24.295646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:45:58.930230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010971830000000001,"stat_sum":{"num_bytes":16281135,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":55,"num_read_kb":40,"num_write":50,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.499448+0000","last_change":"2024-09-18T00:17:57.480235+0000","last_active":"2024-09-18T00:18:24.499448+0000","last_peered":"2024-09-18T00:18:24.499448+0000","last_clean":"2024-09-18T00:18:24.499448+0000","last_became_active":"2024-09-18T00:17:55.478616+0000","last_became_peered":"2024-09-18T00:17:55.478616+0000","last_unstale":"2024-09-18T00:18:24.499448+0000","last_undegraded":"2024-09-18T00:18:24.499448+0000","last_fullsized":"2024-09-18T00:18:24.499448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:12:32.092294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097420299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"40'35","reported_seq":112,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.289884+0000","last_change":"2024-09-18T00:18:16.504734+0000","last_active":"2024-09-18T00:18:24.289884+0000","last_peered":"2024-09-18T00:18:24.289884+0000","last_clean":"2024-09-18T00:18:24.289884+0000","last_became_active":"2024-09-18T00:18:16.440536+0000","last_became_peered":"2024-09-18T00:18:16.440536+0000","last_unstale":"2024-09-18T00:18:24.289884+0000","last_undegraded":"2024-09-18T00:18:24.289884+0000","last_fullsized":"2024-09-18T00:18:24.289884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:09:49.277785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066608800000000001,"stat_sum":{"num_bytes":7429434,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":151,"num_write":37,"num_write_kb":529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":73512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.557951+0000","last_change":"2024-09-18T00:18:16.439964+0000","last_active":"2024-09-18T00:18:24.557951+0000","last_peered":"2024-09-18T00:18:24.557951+0000","last_clean":"2024-09-18T00:18:24.557951+0000","last_became_active":"2024-09-18T00:18:16.437150+0000","last_became_peered":"2024-09-18T00:18:16.437150+0000","last_unstale":"2024-09-18T00:18:24.557951+0000","last_undegraded":"2024-09-18T00:18:24.557951+0000","last_fullsized":"2024-09-18T00:18:24.557951+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:25.772294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"40'106","reported_seq":732,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:25.299027+0000","last_change":"2024-09-18T00:18:07.394381+0000","last_active":"2024-09-18T00:18:25.299027+0000","last_peered":"2024-09-18T00:18:25.299027+0000","last_clean":"2024-09-18T00:18:25.299027+0000","last_became_active":"2024-09-18T00:18:05.400804+0000","last_became_peered":"2024-09-18T00:18:05.400804+0000","last_unstale":"2024-09-18T00:18:25.299027+0000","last_undegraded":"2024-09-18T00:18:25.299027+0000","last_fullsized":"2024-09-18T00:18:25.299027+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:37:28.448764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015579680000000001,"stat_sum":{"num_bytes":9028308,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1503,"num_read_kb":1681,"num_write":105,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.499520+0000","last_change":"2024-09-18T00:17:57.479917+0000","last_active":"2024-09-18T00:18:24.499520+0000","last_peered":"2024-09-18T00:18:24.499520+0000","last_clean":"2024-09-18T00:18:24.499520+0000","last_became_active":"2024-09-18T00:17:55.549824+0000","last_became_peered":"2024-09-18T00:17:55.549824+0000","last_unstale":"2024-09-18T00:18:24.499520+0000","last_undegraded":"2024-09-18T00:18:24.499520+0000","last_fullsized":"2024-09-18T00:18:24.499520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:48.269029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"40'24","reported_seq":82,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.501674+0000","last_change":"2024-09-18T00:18:07.390158+0000","last_active":"2024-09-18T00:18:24.501674+0000","last_peered":"2024-09-18T00:18:24.501674+0000","last_clean":"2024-09-18T00:18:24.501674+0000","last_became_active":"2024-09-18T00:18:05.406861+0000","last_became_peered":"2024-09-18T00:18:05.406861+0000","last_unstale":"2024-09-18T00:18:24.501674+0000","last_undegraded":"2024-09-18T00:18:24.501674+0000","last_fullsized":"2024-09-18T00:18:24.501674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:20.864229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011486619999999999,"stat_sum":{"num_bytes":163856,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":16,"num_write":24,"num_write_kb":209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"41'33","reported_seq":109,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:25.277086+0000","last_change":"2024-09-18T00:18:07.398743+0000","last_active":"2024-09-18T00:18:25.277086+0000","last_peered":"2024-09-18T00:18:25.277086+0000","last_clean":"2024-09-18T00:18:25.277086+0000","last_became_active":"2024-09-18T00:18:05.407889+0000","last_became_peered":"2024-09-18T00:18:05.407889+0000","last_unstale":"2024-09-18T00:18:25.277086+0000","last_undegraded":"2024-09-18T00:18:25.277086+0000","last_fullsized":"2024-09-18T00:18:25.277086+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:41.675682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097981000000000001,"stat_sum":{"num_bytes":4063232,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":97,"num_write":31,"num_write_kb":345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":59,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.500926+0000","last_change":"2024-09-18T00:17:57.478618+0000","last_active":"2024-09-18T00:18:24.500926+0000","last_peered":"2024-09-18T00:18:24.500926+0000","last_clean":"2024-09-18T00:18:24.500926+0000","last_became_active":"2024-09-18T00:17:55.477945+0000","last_became_peered":"2024-09-18T00:17:55.477945+0000","last_unstale":"2024-09-18T00:18:24.500926+0000","last_undegraded":"2024-09-18T00:18:24.500926+0000","last_fullsized":"2024-09-18T00:18:24.500926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:27.096993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088189300000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"39'40","reported_seq":113,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.988159+0000","last_change":"2024-09-18T00:18:07.393363+0000","last_active":"2024-09-18T00:18:24.988159+0000","last_peered":"2024-09-18T00:18:24.988159+0000","last_clean":"2024-09-18T00:18:24.988159+0000","last_became_active":"2024-09-18T00:18:05.402765+0000","last_became_peered":"2024-09-18T00:18:05.402765+0000","last_unstale":"2024-09-18T00:18:24.988159+0000","last_undegraded":"2024-09-18T00:18:24.988159+0000","last_fullsized":"2024-09-18T00:18:24.988159+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:42:28.522267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101741,"stat_sum":{"num_bytes":1797359,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":32,"num_read_kb":256,"num_write":40,"num_write_kb":370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.498804+0000","last_change":"2024-09-18T00:17:57.477633+0000","last_active":"2024-09-18T00:18:24.498804+0000","last_peered":"2024-09-18T00:18:24.498804+0000","last_clean":"2024-09-18T00:18:24.498804+0000","last_became_active":"2024-09-18T00:17:55.631277+0000","last_became_peered":"2024-09-18T00:17:55.631277+0000","last_unstale":"2024-09-18T00:18:24.498804+0000","last_undegraded":"2024-09-18T00:18:24.498804+0000","last_fullsized":"2024-09-18T00:18:24.498804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:55.421684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001141204,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"39'30","reported_seq":102,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:23.540144+0000","last_change":"2024-09-18T00:18:07.408962+0000","last_active":"2024-09-18T00:18:23.540144+0000","last_peered":"2024-09-18T00:18:23.540144+0000","last_clean":"2024-09-18T00:18:23.540144+0000","last_became_active":"2024-09-18T00:18:05.410952+0000","last_became_peered":"2024-09-18T00:18:05.410952+0000","last_unstale":"2024-09-18T00:18:23.540144+0000","last_undegraded":"2024-09-18T00:18:23.540144+0000","last_fullsized":"2024-09-18T00:18:23.540144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:09:57.102610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057326299999999996,"stat_sum":{"num_bytes":2525172,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":107,"num_write":30,"num_write_kb":332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.557945+0000","last_change":"2024-09-18T00:18:16.438044+0000","last_active":"2024-09-18T00:18:24.557945+0000","last_peered":"2024-09-18T00:18:24.557945+0000","last_clean":"2024-09-18T00:18:24.557945+0000","last_became_active":"2024-09-18T00:18:16.437092+0000","last_became_peered":"2024-09-18T00:18:16.437092+0000","last_unstale":"2024-09-18T00:18:24.557945+0000","last_undegraded":"2024-09-18T00:18:24.557945+0000","last_fullsized":"2024-09-18T00:18:24.557945+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:15:13.366138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"39'32","reported_seq":95,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:23.743384+0000","last_change":"2024-09-18T00:18:07.501360+0000","last_active":"2024-09-18T00:18:23.743384+0000","last_peered":"2024-09-18T00:18:23.743384+0000","last_clean":"2024-09-18T00:18:23.743384+0000","last_became_active":"2024-09-18T00:18:05.408271+0000","last_became_peered":"2024-09-18T00:18:05.408271+0000","last_unstale":"2024-09-18T00:18:23.743384+0000","last_undegraded":"2024-09-18T00:18:23.743384+0000","last_fullsized":"2024-09-18T00:18:23.743384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:06.146301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010779679999999999,"stat_sum":{"num_bytes":176364,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":75,"num_write":32,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":58,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.500927+0000","last_change":"2024-09-18T00:17:57.478704+0000","last_active":"2024-09-18T00:18:24.500927+0000","last_peered":"2024-09-18T00:18:24.500927+0000","last_clean":"2024-09-18T00:18:24.500927+0000","last_became_active":"2024-09-18T00:17:55.477618+0000","last_became_peered":"2024-09-18T00:17:55.477618+0000","last_unstale":"2024-09-18T00:18:24.500927+0000","last_undegraded":"2024-09-18T00:18:24.500927+0000","last_fullsized":"2024-09-18T00:18:24.500927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:11.961859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089022700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":207,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.557748+0000","last_change":"2024-09-18T00:18:19.466612+0000","last_active":"2024-09-18T00:18:24.557748+0000","last_peered":"2024-09-18T00:18:24.557748+0000","last_clean":"2024-09-18T00:18:24.557748+0000","last_became_active":"2024-09-18T00:18:19.465557+0000","last_became_peered":"2024-09-18T00:18:19.465557+0000","last_unstale":"2024-09-18T00:18:24.557748+0000","last_undegraded":"2024-09-18T00:18:24.557748+0000","last_fullsized":"2024-09-18T00:18:24.557748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_clean_scrub_stamp":"2024-09-18T00:17:50.414591+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:08:40.389972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"39'47","reported_seq":98,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:23.488193+0000","last_change":"2024-09-18T00:18:07.501829+0000","last_active":"2024-09-18T00:18:23.488193+0000","last_peered":"2024-09-18T00:18:23.488193+0000","last_clean":"2024-09-18T00:18:23.488193+0000","last_became_active":"2024-09-18T00:18:05.404098+0000","last_became_peered":"2024-09-18T00:18:05.404098+0000","last_unstale":"2024-09-18T00:18:23.488193+0000","last_undegraded":"2024-09-18T00:18:23.488193+0000","last_fullsized":"2024-09-18T00:18:23.488193+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:50.449216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001352535,"stat_sum":{"num_bytes":714793,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":48,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"39'32","reported_seq":96,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.512781+0000","last_change":"2024-09-18T00:18:07.398132+0000","last_active":"2024-09-18T00:18:24.512781+0000","last_peered":"2024-09-18T00:18:24.512781+0000","last_clean":"2024-09-18T00:18:24.512781+0000","last_became_active":"2024-09-18T00:18:05.401097+0000","last_became_peered":"2024-09-18T00:18:05.401097+0000","last_unstale":"2024-09-18T00:18:24.512781+0000","last_undegraded":"2024-09-18T00:18:24.512781+0000","last_fullsized":"2024-09-18T00:18:24.512781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:44:04.647087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061791899999999998,"stat_sum":{"num_bytes":315251,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":153,"num_write":32,"num_write_kb":394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"41'27","reported_seq":90,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.863960+0000","last_change":"2024-09-18T00:18:07.501168+0000","last_active":"2024-09-18T00:18:24.863960+0000","last_peered":"2024-09-18T00:18:24.863960+0000","last_clean":"2024-09-18T00:18:24.863960+0000","last_became_active":"2024-09-18T00:18:05.414990+0000","last_became_peered":"2024-09-18T00:18:05.414990+0000","last_unstale":"2024-09-18T00:18:24.863960+0000","last_undegraded":"2024-09-18T00:18:24.863960+0000","last_fullsized":"2024-09-18T00:18:24.863960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:41:36.267808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010106729999999999,"stat_sum":{"num_bytes":333218,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":23,"num_read_kb":92,"num_write":27,"num_write_kb":238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"39'24","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:23.854762+0000","last_change":"2024-09-18T00:18:07.502018+0000","last_active":"2024-09-18T00:18:23.854762+0000","last_peered":"2024-09-18T00:18:23.854762+0000","last_clean":"2024-09-18T00:18:23.854762+0000","last_became_active":"2024-09-18T00:18:05.406977+0000","last_became_peered":"2024-09-18T00:18:05.406977+0000","last_unstale":"2024-09-18T00:18:23.854762+0000","last_undegraded":"2024-09-18T00:18:23.854762+0000","last_fullsized":"2024-09-18T00:18:23.854762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:30.712218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001533539,"stat_sum":{"num_bytes":752508,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":195,"num_write":25,"num_write_kb":266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"41'36","reported_seq":194,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.842867+0000","last_change":"2024-09-18T00:18:07.389395+0000","last_active":"2024-09-18T00:18:24.842867+0000","last_peered":"2024-09-18T00:18:24.842867+0000","last_clean":"2024-09-18T00:18:24.842867+0000","last_became_active":"2024-09-18T00:18:05.408761+0000","last_became_peered":"2024-09-18T00:18:05.408761+0000","last_unstale":"2024-09-18T00:18:24.842867+0000","last_undegraded":"2024-09-18T00:18:24.842867+0000","last_fullsized":"2024-09-18T00:18:24.842867+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:10:30.248434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073075799999999997,"stat_sum":{"num_bytes":2848770,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":284,"num_read_kb":237,"num_write":34,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"39'293","reported_seq":371,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.993909+0000","last_change":"2024-09-18T00:18:19.467234+0000","last_active":"2024-09-18T00:18:24.993909+0000","last_peered":"2024-09-18T00:18:24.993909+0000","last_clean":"2024-09-18T00:18:24.993909+0000","last_became_active":"2024-09-18T00:18:19.465745+0000","last_became_peered":"2024-09-18T00:18:19.465745+0000","last_unstale":"2024-09-18T00:18:24.993909+0000","last_undegraded":"2024-09-18T00:18:24.993909+0000","last_fullsized":"2024-09-18T00:18:24.993909+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:34:24.603043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604930,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1334,"num_read_kb":1223,"num_write":479,"num_write_kb":920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":20783,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"41'37","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:25.331521+0000","last_change":"2024-09-18T00:18:16.512669+0000","last_active":"2024-09-18T00:18:25.331521+0000","last_peered":"2024-09-18T00:18:25.331521+0000","last_clean":"2024-09-18T00:18:25.331521+0000","last_became_active":"2024-09-18T00:18:16.440223+0000","last_became_peered":"2024-09-18T00:18:16.440223+0000","last_unstale":"2024-09-18T00:18:25.331521+0000","last_undegraded":"2024-09-18T00:18:25.331521+0000","last_fullsized":"2024-09-18T00:18:25.331521+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:34.911124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10112664,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":58,"num_read_kb":49,"num_write":44,"num_write_kb":301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":7258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"41'124","reported_seq":871,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:25.335039+0000","last_change":"2024-09-18T00:18:16.528331+0000","last_active":"2024-09-18T00:18:25.335039+0000","last_peered":"2024-09-18T00:18:25.335039+0000","last_clean":"2024-09-18T00:18:25.335039+0000","last_became_active":"2024-09-18T00:18:16.441020+0000","last_became_peered":"2024-09-18T00:18:16.441020+0000","last_unstale":"2024-09-18T00:18:25.335039+0000","last_undegraded":"2024-09-18T00:18:25.335039+0000","last_fullsized":"2024-09-18T00:18:25.335039+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:22:02.012547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12354658,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1856,"num_read_kb":1719,"num_write":128,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"39'17","reported_seq":77,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.496088+0000","last_change":"2024-09-18T00:18:07.501022+0000","last_active":"2024-09-18T00:18:24.496088+0000","last_peered":"2024-09-18T00:18:24.496088+0000","last_clean":"2024-09-18T00:18:24.496088+0000","last_became_active":"2024-09-18T00:18:05.406199+0000","last_became_peered":"2024-09-18T00:18:05.406199+0000","last_unstale":"2024-09-18T00:18:24.496088+0000","last_undegraded":"2024-09-18T00:18:24.496088+0000","last_fullsized":"2024-09-18T00:18:24.496088+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:22:38.909479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091276599999999997,"stat_sum":{"num_bytes":109265,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":73,"num_write":17,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"40'30","reported_seq":125,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:25.315912+0000","last_change":"2024-09-18T00:18:16.458821+0000","last_active":"2024-09-18T00:18:25.315912+0000","last_peered":"2024-09-18T00:18:25.315912+0000","last_clean":"2024-09-18T00:18:25.315912+0000","last_became_active":"2024-09-18T00:18:16.438286+0000","last_became_peered":"2024-09-18T00:18:16.438286+0000","last_unstale":"2024-09-18T00:18:25.315912+0000","last_undegraded":"2024-09-18T00:18:25.315912+0000","last_fullsized":"2024-09-18T00:18:25.315912+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:01.203633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063423600000000004,"stat_sum":{"num_bytes":8494610,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":19,"num_write":34,"num_write_kb":525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":70145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"39'35","reported_seq":128,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:23.858748+0000","last_change":"2024-09-18T00:18:16.482158+0000","last_active":"2024-09-18T00:18:23.858748+0000","last_peered":"2024-09-18T00:18:23.858748+0000","last_clean":"2024-09-18T00:18:23.858748+0000","last_became_active":"2024-09-18T00:18:16.438556+0000","last_became_peered":"2024-09-18T00:18:16.438556+0000","last_unstale":"2024-09-18T00:18:23.858748+0000","last_undegraded":"2024-09-18T00:18:23.858748+0000","last_fullsized":"2024-09-18T00:18:23.858748+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:38:45.345202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010146669999999999,"stat_sum":{"num_bytes":1353501,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":240,"num_write":35,"num_write_kb":487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":47120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"40'39","reported_seq":139,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:25.321207+0000","last_change":"2024-09-18T00:18:16.524316+0000","last_active":"2024-09-18T00:18:25.321207+0000","last_peered":"2024-09-18T00:18:25.321207+0000","last_clean":"2024-09-18T00:18:25.321207+0000","last_became_active":"2024-09-18T00:18:16.440910+0000","last_became_peered":"2024-09-18T00:18:16.440910+0000","last_unstale":"2024-09-18T00:18:25.321207+0000","last_undegraded":"2024-09-18T00:18:25.321207+0000","last_fullsized":"2024-09-18T00:18:25.321207+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:27.850383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16672425,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":77,"num_read_kb":192,"num_write":42,"num_write_kb":513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":121017201,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":9156,"num_read_kb":10166,"num_write":2536,"num_write_kb":12031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":321845,"num_keys_recovered":17,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4145152,"data_stored":7974292,"data_compressed":1954376,"data_compressed_allocated":2510848,"data_compressed_original":7099728,"omap_allocated":0,"internal_metadata":0},"log_size":1935,"ondisk_log_size":1935,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20247,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1327104,"data_stored":1829615,"data_compressed":434123,"data_compressed_allocated":552960,"data_compressed_original":1465000,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":532480,"data_stored":299638,"data_compressed":27257,"data_compressed_allocated":40960,"data_compressed_original":94208,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":737280,"data_stored":462756,"data_compressed":102802,"data_compressed_allocated":122880,"data_compressed_original":188416,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509451,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1327104,"data_stored":1301158,"data_compressed":427054,"data_compressed_allocated":516096,"data_compressed_original":922945,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509452,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":745472,"data_stored":522731,"data_compressed":162767,"data_compressed_allocated":180224,"data_compressed_original":282624,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509452,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":806262,"data_compressed":281533,"data_compressed_allocated":319488,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542154,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":823296,"data_stored":613210,"data_compressed":183756,"data_compressed_allocated":221184,"data_compressed_original":333159,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542155,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":847872,"data_stored":1222537,"data_compressed":195859,"data_compressed_allocated":262144,"data_compressed_original":966656,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":624443,"data_compressed":244560,"data_compressed_allocated":286720,"data_compressed_original":536576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":617485,"data_compressed":281533,"data_compressed_allocated":319488,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290816,"data_stored":333973,"data_compressed":162767,"data_compressed_allocated":180224,"data_compressed_original":282624,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":639281,"data_compressed":249558,"data_compressed_allocated":303104,"data_compressed_original":501095,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":3258898,"data_compressed":457294,"data_compressed_allocated":675840,"data_compressed_original":3061057,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":273979,"data_compressed":102802,"data_compressed_allocated":122880,"data_compressed_original":188416,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1169576,"data_compressed":28777,"data_compressed_allocated":106496,"data_compressed_original":1142784,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":847872,"data_stored":1056657,"data_compressed":427085,"data_compressed_allocated":516096,"data_compressed_original":875176,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:18:28.929 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:18:28.945 INFO:teuthology.orchestra.run.smithi097.stdout:1417 write 0xa943aeb thru 0xa951ad6 (0xdfec bytes) 2024-09-18T00:18:28.947 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:28.947 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:28.960 INFO:teuthology.orchestra.run.smithi097.stdout:1418 trunc from 0xa951ad7 to 0x4009ace 2024-09-18T00:18:28.968 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:18:28.971 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:28.967+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:29.003 INFO:teuthology.orchestra.run.smithi097.stdout:1419 write 0xdeeb3da thru 0xdeedad9 (0x2700 bytes) 2024-09-18T00:18:29.008 INFO:teuthology.orchestra.run.smithi097.stdout:1420 write 0xabffcce thru 0xac0f079 (0xf3ac bytes) 2024-09-18T00:18:29.024 INFO:teuthology.orchestra.run.smithi097.stdout:1421 read 0x52a4280 thru 0x52ab9c5 (0x7746 bytes) 2024-09-18T00:18:29.024 INFO:teuthology.orchestra.run.smithi097.stdout:1422 write 0xd582a04 thru 0xd58471e (0x1d1b bytes) 2024-09-18T00:18:29.028 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:29.029 INFO:teuthology.orchestra.run.smithi097.stdout:1424 writesame 0x3fb516d thru 0x3fb7282 data_size 0x16(0x2116 bytes) 2024-09-18T00:18:29.034 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:29.034 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:29.034 INFO:teuthology.orchestra.run.smithi097.stdout:1430 punch from 0x54d4f77 to 0x54e314c, (0xe1d5 bytes) 2024-09-18T00:18:29.034 INFO:teuthology.orchestra.run.smithi097.stdout:1434 punch from 0xc7d7ec5 to 0xc7e0b92, (0x8ccd bytes) 2024-09-18T00:18:29.046 INFO:teuthology.orchestra.run.smithi097.stdout:1436 writesame 0xed95e46 thru 0xeda56ce data_size 0x19(0xf889 bytes) 2024-09-18T00:18:29.046 INFO:teuthology.orchestra.run.smithi097.stdout:1437 writesame 0x54fadea thru 0x55036b2 data_size 0x13(0x88c9 bytes) 2024-09-18T00:18:29.056 INFO:teuthology.orchestra.run.smithi097.stdout:1438 write 0xc9b264c thru 0xc9b9d34 (0x76e9 bytes) 2024-09-18T00:18:29.067 INFO:teuthology.orchestra.run.smithi097.stdout:1439 trunc from 0xeda56cf to 0xc55fa49 2024-09-18T00:18:29.073 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:29.067+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:29.086 INFO:teuthology.orchestra.run.smithi097.stdout:1440 punch from 0xb901668 to 0xb907166, (0x5afe bytes) 2024-09-18T00:18:29.086 INFO:teuthology.orchestra.run.smithi097.stdout:1441 write 0x7855dc9 thru 0x78598df (0x3b17 bytes) 2024-09-18T00:18:29.092 INFO:teuthology.orchestra.run.smithi097.stdout:1442 read 0xb4fb31c thru 0xb501778 (0x645d bytes) 2024-09-18T00:18:29.093 INFO:teuthology.orchestra.run.smithi097.stdout:1443 read 0x4d3829e thru 0x4d3e1f0 (0x5f53 bytes) 2024-09-18T00:18:29.105 INFO:teuthology.orchestra.run.smithi097.stdout:1444 trunc from 0xc55fa49 to 0x45e9746 2024-09-18T00:18:29.124 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:29.124 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:29.126 INFO:teuthology.orchestra.run.smithi097.stdout:1445 punch from 0x3ccadf4 to 0x3cd8aa6, (0xdcb2 bytes) 2024-09-18T00:18:29.126 INFO:teuthology.orchestra.run.smithi097.stdout:1446 punch from 0xdaa65a to 0xdab5ff, (0xfa5 bytes) 2024-09-18T00:18:29.138 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:18:29.141 INFO:teuthology.orchestra.run.smithi097.stdout:1447 writesame 0x5079d40 thru 0x5084b74 data_size 0x1(0xae35 bytes) 2024-09-18T00:18:29.164 INFO:teuthology.orchestra.run.smithi097.stdout:1448 writesame 0xcd61f42 thru 0xcd62fde data_size 0x1(0x109d bytes) 2024-09-18T00:18:29.164 INFO:teuthology.orchestra.run.smithi097.stdout:1449 write 0xa40f8f thru 0xa4e268 (0xd2da bytes) 2024-09-18T00:18:29.173 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:29.167+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:29.175 INFO:teuthology.orchestra.run.smithi097.stdout:1450 write 0x63ca1b6 thru 0x63cf585 (0x53d0 bytes) 2024-09-18T00:18:29.181 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:29.193 INFO:teuthology.orchestra.run.smithi097.stdout:1455 write 0xd1a403e thru 0xd1b0d1d (0xcce0 bytes) 2024-09-18T00:18:29.199 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:29.199 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:29.199 INFO:teuthology.orchestra.run.smithi097.stdout:1459 writesame 0x5945a7e thru 0x594e031 data_size 0x56(0x85b4 bytes) 2024-09-18T00:18:29.206 INFO:teuthology.orchestra.run.smithi097.stdout:1460 write 0x30d5684 thru 0x30dd423 (0x7da0 bytes) 2024-09-18T00:18:29.216 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:18:29.230 INFO:teuthology.orchestra.run.smithi097.stdout:1461 trunc from 0xd1b0d1e to 0x13657de 2024-09-18T00:18:29.258 INFO:teuthology.orchestra.run.smithi097.stdout:1464 trunc from 0x13657de to 0x4227e62 2024-09-18T00:18:29.261 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:29.261 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:29.269 INFO:teuthology.orchestra.run.smithi097.stdout:1465 read 0x2fecb16 thru 0x2ff575e (0x8c49 bytes) 2024-09-18T00:18:29.272 INFO:teuthology.orchestra.run.smithi097.stdout:1467 trunc from 0x4227e62 to 0x1c20864 2024-09-18T00:18:29.274 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:29.271+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:29.277 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:18:29.288 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:18:29.289 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-18T00:18:29.311 INFO:teuthology.orchestra.run.smithi097.stdout:1468 write 0xd05466d thru 0xd05f13a (0xaace bytes) 2024-09-18T00:18:29.315 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-09-18T00:18:28.547409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":121607588,"num_objects":111,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":0,"num_read":9272,"num_read_kb":10266,"num_write":2697,"num_write_kb":14449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1502581,"num_keys_recovered":17,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2055,"ondisk_log_size":2055,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256680,"kb_used_data":7148,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718040,"statfs":{"total":773094113280,"available":772831272960,"internally_reserved":0,"allocated":7319552,"data_stored":7057907,"data_compressed":1815151,"data_compressed_allocated":2215936,"data_compressed_original":4765008,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":64987666,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":3918,"num_read_kb":4619,"num_write":941,"num_write_kb":3396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.054241"},"pg_stats":[{"pgid":"3.1f","version":"39'24","reported_seq":93,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.291599+0000","last_change":"2024-09-18T00:18:07.410382+0000","last_active":"2024-09-18T00:18:24.291599+0000","last_peered":"2024-09-18T00:18:24.291599+0000","last_clean":"2024-09-18T00:18:24.291599+0000","last_became_active":"2024-09-18T00:18:05.411865+0000","last_became_peered":"2024-09-18T00:18:05.411865+0000","last_unstale":"2024-09-18T00:18:24.291599+0000","last_undegraded":"2024-09-18T00:18:24.291599+0000","last_fullsized":"2024-09-18T00:18:24.291599+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:06:32.938057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001658056,"stat_sum":{"num_bytes":283268,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":32,"num_write":24,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"39'24","reported_seq":75,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:23.488610+0000","last_change":"2024-09-18T00:18:07.502464+0000","last_active":"2024-09-18T00:18:23.488610+0000","last_peered":"2024-09-18T00:18:23.488610+0000","last_clean":"2024-09-18T00:18:23.488610+0000","last_became_active":"2024-09-18T00:18:05.403587+0000","last_became_peered":"2024-09-18T00:18:05.403587+0000","last_unstale":"2024-09-18T00:18:23.488610+0000","last_undegraded":"2024-09-18T00:18:23.488610+0000","last_fullsized":"2024-09-18T00:18:23.488610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:57.767230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019357549999999999,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":20,"num_write":24,"num_write_kb":204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"40'532","reported_seq":598,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:25.005922+0000","last_change":"2024-09-18T00:18:07.390275+0000","last_active":"2024-09-18T00:18:25.005922+0000","last_peered":"2024-09-18T00:18:25.005922+0000","last_clean":"2024-09-18T00:18:25.005922+0000","last_became_active":"2024-09-18T00:18:05.415210+0000","last_became_peered":"2024-09-18T00:18:05.415210+0000","last_unstale":"2024-09-18T00:18:25.005922+0000","last_undegraded":"2024-09-18T00:18:25.005922+0000","last_fullsized":"2024-09-18T00:18:25.005922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:57:00.709996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117322,"stat_sum":{"num_bytes":851985,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2670,"num_read_kb":2308,"num_write":920,"num_write_kb":1251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"41'33","reported_seq":102,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.847792+0000","last_change":"2024-09-18T00:18:07.389495+0000","last_active":"2024-09-18T00:18:24.847792+0000","last_peered":"2024-09-18T00:18:24.847792+0000","last_clean":"2024-09-18T00:18:24.847792+0000","last_became_active":"2024-09-18T00:18:05.407036+0000","last_became_peered":"2024-09-18T00:18:05.407036+0000","last_unstale":"2024-09-18T00:18:24.847792+0000","last_undegraded":"2024-09-18T00:18:24.847792+0000","last_fullsized":"2024-09-18T00:18:24.847792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:10:15.997577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055723800000000005,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":38,"num_write":33,"num_write_kb":180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"39'23","reported_seq":74,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:23.540374+0000","last_change":"2024-09-18T00:18:07.409072+0000","last_active":"2024-09-18T00:18:23.540374+0000","last_peered":"2024-09-18T00:18:23.540374+0000","last_clean":"2024-09-18T00:18:23.540374+0000","last_became_active":"2024-09-18T00:18:05.415810+0000","last_became_peered":"2024-09-18T00:18:05.415810+0000","last_unstale":"2024-09-18T00:18:23.540374+0000","last_undegraded":"2024-09-18T00:18:23.540374+0000","last_fullsized":"2024-09-18T00:18:23.540374+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:56:57.406500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059152399999999996,"stat_sum":{"num_bytes":98304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":16,"num_write":23,"num_write_kb":186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"40'25","reported_seq":367,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:25.323538+0000","last_change":"2024-09-18T00:18:07.399486+0000","last_active":"2024-09-18T00:18:25.323538+0000","last_peered":"2024-09-18T00:18:25.323538+0000","last_clean":"2024-09-18T00:18:25.323538+0000","last_became_active":"2024-09-18T00:18:05.395573+0000","last_became_peered":"2024-09-18T00:18:05.395573+0000","last_unstale":"2024-09-18T00:18:25.323538+0000","last_undegraded":"2024-09-18T00:18:25.323538+0000","last_fullsized":"2024-09-18T00:18:25.323538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:01:11.873931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020525439999999999,"stat_sum":{"num_bytes":4893811,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":292,"num_read_kb":254,"num_write":26,"num_write_kb":332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"39'27","reported_seq":88,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.985186+0000","last_change":"2024-09-18T00:18:07.393316+0000","last_active":"2024-09-18T00:18:24.985186+0000","last_peered":"2024-09-18T00:18:24.985186+0000","last_clean":"2024-09-18T00:18:24.985186+0000","last_became_active":"2024-09-18T00:18:05.403595+0000","last_became_peered":"2024-09-18T00:18:05.403595+0000","last_unstale":"2024-09-18T00:18:24.985186+0000","last_undegraded":"2024-09-18T00:18:24.985186+0000","last_fullsized":"2024-09-18T00:18:24.985186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:51:41.679476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077375,"stat_sum":{"num_bytes":1000188,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":130,"num_write":29,"num_write_kb":329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"40'16","reported_seq":67,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.183792+0000","last_change":"2024-09-18T00:18:07.500927+0000","last_active":"2024-09-18T00:18:24.183792+0000","last_peered":"2024-09-18T00:18:24.183792+0000","last_clean":"2024-09-18T00:18:24.183792+0000","last_became_active":"2024-09-18T00:18:05.406821+0000","last_became_peered":"2024-09-18T00:18:05.406821+0000","last_unstale":"2024-09-18T00:18:24.183792+0000","last_undegraded":"2024-09-18T00:18:24.183792+0000","last_fullsized":"2024-09-18T00:18:24.183792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:49:53.291639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074352700000000001,"stat_sum":{"num_bytes":16723727,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":17,"num_write":20,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"39'41","reported_seq":92,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:23.540567+0000","last_change":"2024-09-18T00:18:07.410454+0000","last_active":"2024-09-18T00:18:23.540567+0000","last_peered":"2024-09-18T00:18:23.540567+0000","last_clean":"2024-09-18T00:18:23.540567+0000","last_became_active":"2024-09-18T00:18:05.411650+0000","last_became_peered":"2024-09-18T00:18:05.411650+0000","last_unstale":"2024-09-18T00:18:23.540567+0000","last_undegraded":"2024-09-18T00:18:23.540567+0000","last_fullsized":"2024-09-18T00:18:23.540567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:10:42.627134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012546969999999999,"stat_sum":{"num_bytes":172280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":12,"num_write":41,"num_write_kb":405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"39'43","reported_seq":318,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.501211+0000","last_change":"2024-09-18T00:18:16.552071+0000","last_active":"2024-09-18T00:18:24.501211+0000","last_peered":"2024-09-18T00:18:24.501211+0000","last_clean":"2024-09-18T00:18:24.501211+0000","last_became_active":"2024-09-18T00:18:16.444631+0000","last_became_peered":"2024-09-18T00:18:16.444631+0000","last_unstale":"2024-09-18T00:18:24.501211+0000","last_undegraded":"2024-09-18T00:18:24.501211+0000","last_fullsized":"2024-09-18T00:18:24.501211+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:05.394823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015027420000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":599,"num_read_kb":523,"num_write":37,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":17,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"39'38","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.558482+0000","last_change":"2024-09-18T00:18:16.491563+0000","last_active":"2024-09-18T00:18:24.558482+0000","last_peered":"2024-09-18T00:18:24.558482+0000","last_clean":"2024-09-18T00:18:24.558482+0000","last_became_active":"2024-09-18T00:18:16.438164+0000","last_became_peered":"2024-09-18T00:18:16.438164+0000","last_unstale":"2024-09-18T00:18:24.558482+0000","last_undegraded":"2024-09-18T00:18:24.558482+0000","last_fullsized":"2024-09-18T00:18:24.558482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:54:51.185189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":28,"num_write":40,"num_write_kb":684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"39'25","reported_seq":130,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.512696+0000","last_change":"2024-09-18T00:18:16.544097+0000","last_active":"2024-09-18T00:18:24.512696+0000","last_peered":"2024-09-18T00:18:24.512696+0000","last_clean":"2024-09-18T00:18:24.512696+0000","last_became_active":"2024-09-18T00:18:16.441758+0000","last_became_peered":"2024-09-18T00:18:16.441758+0000","last_unstale":"2024-09-18T00:18:24.512696+0000","last_undegraded":"2024-09-18T00:18:24.512696+0000","last_fullsized":"2024-09-18T00:18:24.512696+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:12:34.554135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000991729,"stat_sum":{"num_bytes":272361,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":173,"num_write":25,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":103027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.496023+0000","last_change":"2024-09-18T00:18:16.445661+0000","last_active":"2024-09-18T00:18:24.496023+0000","last_peered":"2024-09-18T00:18:24.496023+0000","last_clean":"2024-09-18T00:18:24.496023+0000","last_became_active":"2024-09-18T00:18:16.444732+0000","last_became_peered":"2024-09-18T00:18:16.444732+0000","last_unstale":"2024-09-18T00:18:24.496023+0000","last_undegraded":"2024-09-18T00:18:24.496023+0000","last_fullsized":"2024-09-18T00:18:24.496023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:29.764590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089554000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"40'43","reported_seq":94,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.295646+0000","last_change":"2024-09-18T00:18:07.409625+0000","last_active":"2024-09-18T00:18:24.295646+0000","last_peered":"2024-09-18T00:18:24.295646+0000","last_clean":"2024-09-18T00:18:24.295646+0000","last_became_active":"2024-09-18T00:18:05.403736+0000","last_became_peered":"2024-09-18T00:18:05.403736+0000","last_unstale":"2024-09-18T00:18:24.295646+0000","last_undegraded":"2024-09-18T00:18:24.295646+0000","last_fullsized":"2024-09-18T00:18:24.295646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:45:58.930230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010971830000000001,"stat_sum":{"num_bytes":16281135,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":55,"num_read_kb":40,"num_write":50,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.499448+0000","last_change":"2024-09-18T00:17:57.480235+0000","last_active":"2024-09-18T00:18:24.499448+0000","last_peered":"2024-09-18T00:18:24.499448+0000","last_clean":"2024-09-18T00:18:24.499448+0000","last_became_active":"2024-09-18T00:17:55.478616+0000","last_became_peered":"2024-09-18T00:17:55.478616+0000","last_unstale":"2024-09-18T00:18:24.499448+0000","last_undegraded":"2024-09-18T00:18:24.499448+0000","last_fullsized":"2024-09-18T00:18:24.499448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:12:32.092294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097420299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"40'35","reported_seq":112,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.289884+0000","last_change":"2024-09-18T00:18:16.504734+0000","last_active":"2024-09-18T00:18:24.289884+0000","last_peered":"2024-09-18T00:18:24.289884+0000","last_clean":"2024-09-18T00:18:24.289884+0000","last_became_active":"2024-09-18T00:18:16.440536+0000","last_became_peered":"2024-09-18T00:18:16.440536+0000","last_unstale":"2024-09-18T00:18:24.289884+0000","last_undegraded":"2024-09-18T00:18:24.289884+0000","last_fullsized":"2024-09-18T00:18:24.289884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:09:49.277785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066608800000000001,"stat_sum":{"num_bytes":7429434,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":151,"num_write":37,"num_write_kb":529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":73512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.557951+0000","last_change":"2024-09-18T00:18:16.439964+0000","last_active":"2024-09-18T00:18:24.557951+0000","last_peered":"2024-09-18T00:18:24.557951+0000","last_clean":"2024-09-18T00:18:24.557951+0000","last_became_active":"2024-09-18T00:18:16.437150+0000","last_became_peered":"2024-09-18T00:18:16.437150+0000","last_unstale":"2024-09-18T00:18:24.557951+0000","last_undegraded":"2024-09-18T00:18:24.557951+0000","last_fullsized":"2024-09-18T00:18:24.557951+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:25.772294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"40'106","reported_seq":732,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:25.299027+0000","last_change":"2024-09-18T00:18:07.394381+0000","last_active":"2024-09-18T00:18:25.299027+0000","last_peered":"2024-09-18T00:18:25.299027+0000","last_clean":"2024-09-18T00:18:25.299027+0000","last_became_active":"2024-09-18T00:18:05.400804+0000","last_became_peered":"2024-09-18T00:18:05.400804+0000","last_unstale":"2024-09-18T00:18:25.299027+0000","last_undegraded":"2024-09-18T00:18:25.299027+0000","last_fullsized":"2024-09-18T00:18:25.299027+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:37:28.448764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015579680000000001,"stat_sum":{"num_bytes":9028308,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1503,"num_read_kb":1681,"num_write":105,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.499520+0000","last_change":"2024-09-18T00:17:57.479917+0000","last_active":"2024-09-18T00:18:24.499520+0000","last_peered":"2024-09-18T00:18:24.499520+0000","last_clean":"2024-09-18T00:18:24.499520+0000","last_became_active":"2024-09-18T00:17:55.549824+0000","last_became_peered":"2024-09-18T00:17:55.549824+0000","last_unstale":"2024-09-18T00:18:24.499520+0000","last_undegraded":"2024-09-18T00:18:24.499520+0000","last_fullsized":"2024-09-18T00:18:24.499520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:48.269029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"40'24","reported_seq":82,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.501674+0000","last_change":"2024-09-18T00:18:07.390158+0000","last_active":"2024-09-18T00:18:24.501674+0000","last_peered":"2024-09-18T00:18:24.501674+0000","last_clean":"2024-09-18T00:18:24.501674+0000","last_became_active":"2024-09-18T00:18:05.406861+0000","last_became_peered":"2024-09-18T00:18:05.406861+0000","last_unstale":"2024-09-18T00:18:24.501674+0000","last_undegraded":"2024-09-18T00:18:24.501674+0000","last_fullsized":"2024-09-18T00:18:24.501674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:20.864229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011486619999999999,"stat_sum":{"num_bytes":163856,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":16,"num_write":24,"num_write_kb":209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"41'33","reported_seq":109,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:25.277086+0000","last_change":"2024-09-18T00:18:07.398743+0000","last_active":"2024-09-18T00:18:25.277086+0000","last_peered":"2024-09-18T00:18:25.277086+0000","last_clean":"2024-09-18T00:18:25.277086+0000","last_became_active":"2024-09-18T00:18:05.407889+0000","last_became_peered":"2024-09-18T00:18:05.407889+0000","last_unstale":"2024-09-18T00:18:25.277086+0000","last_undegraded":"2024-09-18T00:18:25.277086+0000","last_fullsized":"2024-09-18T00:18:25.277086+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:41.675682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097981000000000001,"stat_sum":{"num_bytes":4063232,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":97,"num_write":31,"num_write_kb":345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":59,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.500926+0000","last_change":"2024-09-18T00:17:57.478618+0000","last_active":"2024-09-18T00:18:24.500926+0000","last_peered":"2024-09-18T00:18:24.500926+0000","last_clean":"2024-09-18T00:18:24.500926+0000","last_became_active":"2024-09-18T00:17:55.477945+0000","last_became_peered":"2024-09-18T00:17:55.477945+0000","last_unstale":"2024-09-18T00:18:24.500926+0000","last_undegraded":"2024-09-18T00:18:24.500926+0000","last_fullsized":"2024-09-18T00:18:24.500926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:27.096993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088189300000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"39'40","reported_seq":113,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.988159+0000","last_change":"2024-09-18T00:18:07.393363+0000","last_active":"2024-09-18T00:18:24.988159+0000","last_peered":"2024-09-18T00:18:24.988159+0000","last_clean":"2024-09-18T00:18:24.988159+0000","last_became_active":"2024-09-18T00:18:05.402765+0000","last_became_peered":"2024-09-18T00:18:05.402765+0000","last_unstale":"2024-09-18T00:18:24.988159+0000","last_undegraded":"2024-09-18T00:18:24.988159+0000","last_fullsized":"2024-09-18T00:18:24.988159+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:42:28.522267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101741,"stat_sum":{"num_bytes":1797359,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":32,"num_read_kb":256,"num_write":40,"num_write_kb":370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.498804+0000","last_change":"2024-09-18T00:17:57.477633+0000","last_active":"2024-09-18T00:18:24.498804+0000","last_peered":"2024-09-18T00:18:24.498804+0000","last_clean":"2024-09-18T00:18:24.498804+0000","last_became_active":"2024-09-18T00:17:55.631277+0000","last_became_peered":"2024-09-18T00:17:55.631277+0000","last_unstale":"2024-09-18T00:18:24.498804+0000","last_undegraded":"2024-09-18T00:18:24.498804+0000","last_fullsized":"2024-09-18T00:18:24.498804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:55.421684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001141204,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"39'30","reported_seq":102,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:23.540144+0000","last_change":"2024-09-18T00:18:07.408962+0000","last_active":"2024-09-18T00:18:23.540144+0000","last_peered":"2024-09-18T00:18:23.540144+0000","last_clean":"2024-09-18T00:18:23.540144+0000","last_became_active":"2024-09-18T00:18:05.410952+0000","last_became_peered":"2024-09-18T00:18:05.410952+0000","last_unstale":"2024-09-18T00:18:23.540144+0000","last_undegraded":"2024-09-18T00:18:23.540144+0000","last_fullsized":"2024-09-18T00:18:23.540144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:09:57.102610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057326299999999996,"stat_sum":{"num_bytes":2525172,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":107,"num_write":30,"num_write_kb":332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.557945+0000","last_change":"2024-09-18T00:18:16.438044+0000","last_active":"2024-09-18T00:18:24.557945+0000","last_peered":"2024-09-18T00:18:24.557945+0000","last_clean":"2024-09-18T00:18:24.557945+0000","last_became_active":"2024-09-18T00:18:16.437092+0000","last_became_peered":"2024-09-18T00:18:16.437092+0000","last_unstale":"2024-09-18T00:18:24.557945+0000","last_undegraded":"2024-09-18T00:18:24.557945+0000","last_fullsized":"2024-09-18T00:18:24.557945+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:15:13.366138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"39'32","reported_seq":95,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:23.743384+0000","last_change":"2024-09-18T00:18:07.501360+0000","last_active":"2024-09-18T00:18:23.743384+0000","last_peered":"2024-09-18T00:18:23.743384+0000","last_clean":"2024-09-18T00:18:23.743384+0000","last_became_active":"2024-09-18T00:18:05.408271+0000","last_became_peered":"2024-09-18T00:18:05.408271+0000","last_unstale":"2024-09-18T00:18:23.743384+0000","last_undegraded":"2024-09-18T00:18:23.743384+0000","last_fullsized":"2024-09-18T00:18:23.743384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:06.146301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010779679999999999,"stat_sum":{"num_bytes":176364,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":75,"num_write":32,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":58,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.500927+0000","last_change":"2024-09-18T00:17:57.478704+0000","last_active":"2024-09-18T00:18:24.500927+0000","last_peered":"2024-09-18T00:18:24.500927+0000","last_clean":"2024-09-18T00:18:24.500927+0000","last_became_active":"2024-09-18T00:17:55.477618+0000","last_became_peered":"2024-09-18T00:17:55.477618+0000","last_unstale":"2024-09-18T00:18:24.500927+0000","last_undegraded":"2024-09-18T00:18:24.500927+0000","last_fullsized":"2024-09-18T00:18:24.500927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:11.961859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089022700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":207,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.557748+0000","last_change":"2024-09-18T00:18:19.466612+0000","last_active":"2024-09-18T00:18:24.557748+0000","last_peered":"2024-09-18T00:18:24.557748+0000","last_clean":"2024-09-18T00:18:24.557748+0000","last_became_active":"2024-09-18T00:18:19.465557+0000","last_became_peered":"2024-09-18T00:18:19.465557+0000","last_unstale":"2024-09-18T00:18:24.557748+0000","last_undegraded":"2024-09-18T00:18:24.557748+0000","last_fullsized":"2024-09-18T00:18:24.557748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_clean_scrub_stamp":"2024-09-18T00:17:50.414591+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:08:40.389972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"39'47","reported_seq":98,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:23.488193+0000","last_change":"2024-09-18T00:18:07.501829+0000","last_active":"2024-09-18T00:18:23.488193+0000","last_peered":"2024-09-18T00:18:23.488193+0000","last_clean":"2024-09-18T00:18:23.488193+0000","last_became_active":"2024-09-18T00:18:05.404098+0000","last_became_peered":"2024-09-18T00:18:05.404098+0000","last_unstale":"2024-09-18T00:18:23.488193+0000","last_undegraded":"2024-09-18T00:18:23.488193+0000","last_fullsized":"2024-09-18T00:18:23.488193+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:50.449216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001352535,"stat_sum":{"num_bytes":714793,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":48,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"39'32","reported_seq":96,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.512781+0000","last_change":"2024-09-18T00:18:07.398132+0000","last_active":"2024-09-18T00:18:24.512781+0000","last_peered":"2024-09-18T00:18:24.512781+0000","last_clean":"2024-09-18T00:18:24.512781+0000","last_became_active":"2024-09-18T00:18:05.401097+0000","last_became_peered":"2024-09-18T00:18:05.401097+0000","last_unstale":"2024-09-18T00:18:24.512781+0000","last_undegraded":"2024-09-18T00:18:24.512781+0000","last_fullsized":"2024-09-18T00:18:24.512781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:44:04.647087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061791899999999998,"stat_sum":{"num_bytes":315251,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":153,"num_write":32,"num_write_kb":394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"41'27","reported_seq":90,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.863960+0000","last_change":"2024-09-18T00:18:07.501168+0000","last_active":"2024-09-18T00:18:24.863960+0000","last_peered":"2024-09-18T00:18:24.863960+0000","last_clean":"2024-09-18T00:18:24.863960+0000","last_became_active":"2024-09-18T00:18:05.414990+0000","last_became_peered":"2024-09-18T00:18:05.414990+0000","last_unstale":"2024-09-18T00:18:24.863960+0000","last_undegraded":"2024-09-18T00:18:24.863960+0000","last_fullsized":"2024-09-18T00:18:24.863960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:41:36.267808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010106729999999999,"stat_sum":{"num_bytes":333218,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":23,"num_read_kb":92,"num_write":27,"num_write_kb":238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"39'24","reported_seq":82,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:23.854762+0000","last_change":"2024-09-18T00:18:07.502018+0000","last_active":"2024-09-18T00:18:23.854762+0000","last_peered":"2024-09-18T00:18:23.854762+0000","last_clean":"2024-09-18T00:18:23.854762+0000","last_became_active":"2024-09-18T00:18:05.406977+0000","last_became_peered":"2024-09-18T00:18:05.406977+0000","last_unstale":"2024-09-18T00:18:23.854762+0000","last_undegraded":"2024-09-18T00:18:23.854762+0000","last_fullsized":"2024-09-18T00:18:23.854762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:30.712218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001533539,"stat_sum":{"num_bytes":752508,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":195,"num_write":25,"num_write_kb":266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"41'36","reported_seq":194,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.842867+0000","last_change":"2024-09-18T00:18:07.389395+0000","last_active":"2024-09-18T00:18:24.842867+0000","last_peered":"2024-09-18T00:18:24.842867+0000","last_clean":"2024-09-18T00:18:24.842867+0000","last_became_active":"2024-09-18T00:18:05.408761+0000","last_became_peered":"2024-09-18T00:18:05.408761+0000","last_unstale":"2024-09-18T00:18:24.842867+0000","last_undegraded":"2024-09-18T00:18:24.842867+0000","last_fullsized":"2024-09-18T00:18:24.842867+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:10:30.248434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073075799999999997,"stat_sum":{"num_bytes":2848770,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":284,"num_read_kb":237,"num_write":34,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"39'293","reported_seq":371,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.993909+0000","last_change":"2024-09-18T00:18:19.467234+0000","last_active":"2024-09-18T00:18:24.993909+0000","last_peered":"2024-09-18T00:18:24.993909+0000","last_clean":"2024-09-18T00:18:24.993909+0000","last_became_active":"2024-09-18T00:18:19.465745+0000","last_became_peered":"2024-09-18T00:18:19.465745+0000","last_unstale":"2024-09-18T00:18:24.993909+0000","last_undegraded":"2024-09-18T00:18:24.993909+0000","last_fullsized":"2024-09-18T00:18:24.993909+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:34:24.603043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604930,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1334,"num_read_kb":1223,"num_write":479,"num_write_kb":920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":20783,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"41'37","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:25.331521+0000","last_change":"2024-09-18T00:18:16.512669+0000","last_active":"2024-09-18T00:18:25.331521+0000","last_peered":"2024-09-18T00:18:25.331521+0000","last_clean":"2024-09-18T00:18:25.331521+0000","last_became_active":"2024-09-18T00:18:16.440223+0000","last_became_peered":"2024-09-18T00:18:16.440223+0000","last_unstale":"2024-09-18T00:18:25.331521+0000","last_undegraded":"2024-09-18T00:18:25.331521+0000","last_fullsized":"2024-09-18T00:18:25.331521+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:34.911124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10112664,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":58,"num_read_kb":49,"num_write":44,"num_write_kb":301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":7258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"41'124","reported_seq":871,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:25.335039+0000","last_change":"2024-09-18T00:18:16.528331+0000","last_active":"2024-09-18T00:18:25.335039+0000","last_peered":"2024-09-18T00:18:25.335039+0000","last_clean":"2024-09-18T00:18:25.335039+0000","last_became_active":"2024-09-18T00:18:16.441020+0000","last_became_peered":"2024-09-18T00:18:16.441020+0000","last_unstale":"2024-09-18T00:18:25.335039+0000","last_undegraded":"2024-09-18T00:18:25.335039+0000","last_fullsized":"2024-09-18T00:18:25.335039+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:22:02.012547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12354658,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1856,"num_read_kb":1719,"num_write":128,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"39'17","reported_seq":77,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:24.496088+0000","last_change":"2024-09-18T00:18:07.501022+0000","last_active":"2024-09-18T00:18:24.496088+0000","last_peered":"2024-09-18T00:18:24.496088+0000","last_clean":"2024-09-18T00:18:24.496088+0000","last_became_active":"2024-09-18T00:18:05.406199+0000","last_became_peered":"2024-09-18T00:18:05.406199+0000","last_unstale":"2024-09-18T00:18:24.496088+0000","last_undegraded":"2024-09-18T00:18:24.496088+0000","last_fullsized":"2024-09-18T00:18:24.496088+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:22:38.909479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091276599999999997,"stat_sum":{"num_bytes":109265,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":73,"num_write":17,"num_write_kb":168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"40'30","reported_seq":125,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:25.315912+0000","last_change":"2024-09-18T00:18:16.458821+0000","last_active":"2024-09-18T00:18:25.315912+0000","last_peered":"2024-09-18T00:18:25.315912+0000","last_clean":"2024-09-18T00:18:25.315912+0000","last_became_active":"2024-09-18T00:18:16.438286+0000","last_became_peered":"2024-09-18T00:18:16.438286+0000","last_unstale":"2024-09-18T00:18:25.315912+0000","last_undegraded":"2024-09-18T00:18:25.315912+0000","last_fullsized":"2024-09-18T00:18:25.315912+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:01.203633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063423600000000004,"stat_sum":{"num_bytes":8494610,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":19,"num_write":34,"num_write_kb":525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":70145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"39'35","reported_seq":128,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-18T00:18:23.858748+0000","last_change":"2024-09-18T00:18:16.482158+0000","last_active":"2024-09-18T00:18:23.858748+0000","last_peered":"2024-09-18T00:18:23.858748+0000","last_clean":"2024-09-18T00:18:23.858748+0000","last_became_active":"2024-09-18T00:18:16.438556+0000","last_became_peered":"2024-09-18T00:18:16.438556+0000","last_unstale":"2024-09-18T00:18:23.858748+0000","last_undegraded":"2024-09-18T00:18:23.858748+0000","last_fullsized":"2024-09-18T00:18:23.858748+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:38:45.345202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010146669999999999,"stat_sum":{"num_bytes":1353501,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":240,"num_write":35,"num_write_kb":487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":47120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"40'39","reported_seq":139,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-18T00:18:25.321207+0000","last_change":"2024-09-18T00:18:16.524316+0000","last_active":"2024-09-18T00:18:25.321207+0000","last_peered":"2024-09-18T00:18:25.321207+0000","last_clean":"2024-09-18T00:18:25.321207+0000","last_became_active":"2024-09-18T00:18:16.440910+0000","last_became_peered":"2024-09-18T00:18:16.440910+0000","last_unstale":"2024-09-18T00:18:25.321207+0000","last_undegraded":"2024-09-18T00:18:25.321207+0000","last_fullsized":"2024-09-18T00:18:25.321207+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:27.850383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16672425,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":77,"num_read_kb":192,"num_write":42,"num_write_kb":513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":121017201,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":9156,"num_read_kb":10166,"num_write":2536,"num_write_kb":12031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":321845,"num_keys_recovered":17,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4145152,"data_stored":7974292,"data_compressed":1954376,"data_compressed_allocated":2510848,"data_compressed_original":7099728,"omap_allocated":0,"internal_metadata":0},"log_size":1935,"ondisk_log_size":1935,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20247,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1327104,"data_stored":1829615,"data_compressed":434123,"data_compressed_allocated":552960,"data_compressed_original":1465000,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":532480,"data_stored":299638,"data_compressed":27257,"data_compressed_allocated":40960,"data_compressed_original":94208,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":737280,"data_stored":462756,"data_compressed":102802,"data_compressed_allocated":122880,"data_compressed_original":188416,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509451,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1327104,"data_stored":1301158,"data_compressed":427054,"data_compressed_allocated":516096,"data_compressed_original":922945,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509452,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":745472,"data_stored":522731,"data_compressed":162767,"data_compressed_allocated":180224,"data_compressed_original":282624,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509452,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":806262,"data_compressed":281533,"data_compressed_allocated":319488,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542154,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":823296,"data_stored":613210,"data_compressed":183756,"data_compressed_allocated":221184,"data_compressed_original":333159,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542155,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":847872,"data_stored":1222537,"data_compressed":195859,"data_compressed_allocated":262144,"data_compressed_original":966656,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":624443,"data_compressed":244560,"data_compressed_allocated":286720,"data_compressed_original":536576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":617485,"data_compressed":281533,"data_compressed_allocated":319488,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290816,"data_stored":333973,"data_compressed":162767,"data_compressed_allocated":180224,"data_compressed_original":282624,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":639281,"data_compressed":249558,"data_compressed_allocated":303104,"data_compressed_original":501095,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":3258898,"data_compressed":457294,"data_compressed_allocated":675840,"data_compressed_original":3061057,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":273979,"data_compressed":102802,"data_compressed_allocated":122880,"data_compressed_original":188416,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1169576,"data_compressed":28777,"data_compressed_allocated":106496,"data_compressed_original":1142784,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":847872,"data_stored":1056657,"data_compressed":427085,"data_compressed_allocated":516096,"data_compressed_original":875176,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:18:29.317 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T00:18:29.317 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:18:29.317 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:18:29.317 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-18T00:18:29.317 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T00:18:29.318 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:29.322 INFO:teuthology.orchestra.run.smithi097.stdout:1471 trunc from 0xd05f13b to 0xa3c058c 2024-09-18T00:18:29.342 INFO:teuthology.orchestra.run.smithi097.stdout:1472 read 0x3ea7d10 thru 0x3eb797f (0xfc70 bytes) 2024-09-18T00:18:29.342 INFO:teuthology.orchestra.run.smithi097.stdout:1473 write 0x11fca19 thru 0x12093ce (0xc9b6 bytes) 2024-09-18T00:18:29.369 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:29.369 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:29.375 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:29.371+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:29.397 INFO:teuthology.orchestra.run.smithi097.stderr:admin_socket: exception getting command descriptions: exception: [Errno 104] Connection reset by peer 2024-09-18T00:18:29.400 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T00:18:29.401 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:18:29.405 INFO:tasks.ceph.osd.1:Stopped 2024-09-18T00:18:29.405 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.1 2024-09-18T00:18:29.405 INFO:tasks.thrashosds.thrasher:remote for osd 1 is ubuntu@smithi097.front.sepia.ceph.com 2024-09-18T00:18:29.406 DEBUG:teuthology.orchestra.run.smithi097:> type ceph-objectstore-tool 2024-09-18T00:18:29.452 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T00:18:29.452 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op list-pgs 2024-09-18T00:18:29.493 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:29.491+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:29.545 INFO:teuthology.orchestra.run.smithi097.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-18T00:18:29.547 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T00:18:29.548 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:18:29.596 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:29.595+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:29.662 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:29.662 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:29.675 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:18:29.678 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:29.675+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:29.778 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:29.775+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:29.833 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:29.834 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:29.846 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:18:29.879 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:29.875+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:29.963 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:29.963 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:29.976 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:18:29.979 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:29.974+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:30.080 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:30.075+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:30.109 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:30.109 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:30.123 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:18:30.214 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:30.211+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:30.282 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:30.278+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:30.289 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:30.290 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:30.306 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:18:30.390 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:30.387+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:30.472 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:30.472 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:30.483 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:30.479+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:30.487 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:18:30.565 INFO:teuthology.orchestra.run.smithi097.stdout:1474 punch from 0x6c7ba4c to 0x6c8b526, (0xfada bytes) 2024-09-18T00:18:30.565 INFO:teuthology.orchestra.run.smithi097.stdout:1475 punch from 0xa1a545f to 0xa1b1a6e, (0xc60f bytes) 2024-09-18T00:18:30.566 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:30.566 INFO:teuthology.orchestra.run.smithi097.stdout:1477 punch from 0x5cd8f6c to 0x5ce3d48, (0xaddc bytes) 2024-09-18T00:18:30.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:18:30.583 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:30.579+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:30.585 INFO:teuthology.orchestra.run.smithi097.stdout:1479 writesame 0xe9a81bd thru 0xe9ab275 data_size 0x1(0x30b9 bytes) 2024-09-18T00:18:30.586 INFO:teuthology.orchestra.run.smithi097.stdout:1480 writesame 0x31f0c7 thru 0x329a56 data_size 0x8(0xa990 bytes) 2024-09-18T00:18:30.591 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:30.591 INFO:teuthology.orchestra.run.smithi097.stdout:1482 write 0x12ca8fe thru 0x12cf555 (0x4c58 bytes) 2024-09-18T00:18:30.602 INFO:teuthology.orchestra.run.smithi097.stdout:1484 writesame 0xea2cc69 thru 0xea3b2ce data_size 0x4d(0xe666 bytes) 2024-09-18T00:18:30.612 INFO:teuthology.orchestra.run.smithi097.stdout:1486 trunc from 0xea3b2cf to 0xaaa2cbb 2024-09-18T00:18:30.624 INFO:teuthology.orchestra.run.smithi097.stdout:1489 read 0x8e5e958 thru 0x8e63cbf (0x5368 bytes) 2024-09-18T00:18:30.624 INFO:teuthology.orchestra.run.smithi097.stdout:1490 read 0x425beca thru 0x425f297 (0x33ce bytes) 2024-09-18T00:18:30.624 INFO:teuthology.orchestra.run.smithi097.stdout:1492 punch from 0x9ed3d32 to 0x9ede69f, (0xa96d bytes) 2024-09-18T00:18:30.624 INFO:teuthology.orchestra.run.smithi097.stdout:1493 write 0x3753f55 thru 0x3761f7e (0xe02a bytes) 2024-09-18T00:18:30.637 INFO:teuthology.orchestra.run.smithi097.stdout:1494 write 0x1d53d77 thru 0x1d5d6c9 (0x9953 bytes) 2024-09-18T00:18:30.649 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:30.649 INFO:teuthology.orchestra.run.smithi097.stdout:1496 read 0x119ab5a thru 0x119ff0c (0x53b3 bytes) 2024-09-18T00:18:30.650 INFO:teuthology.orchestra.run.smithi097.stdout:1497 punch from 0x971b478 to 0x971f797, (0x431f bytes) 2024-09-18T00:18:30.657 INFO:teuthology.orchestra.run.smithi097.stdout:1498 trunc from 0xaaa2cbb to 0x78f3442 2024-09-18T00:18:30.661 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:30.661 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:30.676 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:18:30.684 INFO:teuthology.orchestra.run.smithi097.stdout:1499 writesame 0x7ec45e0 thru 0x7ec58a1 data_size 0xe(0x12c2 bytes) 2024-09-18T00:18:30.684 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:30.684 INFO:teuthology.orchestra.run.smithi097.stdout:1501 punch from 0x4b31d29 to 0x4b3f219, (0xd4f0 bytes) 2024-09-18T00:18:30.684 INFO:teuthology.orchestra.run.smithi097.stdout:1502 punch from 0x53d663e to 0x53e2986, (0xc348 bytes) 2024-09-18T00:18:30.684 INFO:teuthology.orchestra.run.smithi097.stdout:1503 write 0x25fa32e thru 0x25ff180 (0x4e53 bytes) 2024-09-18T00:18:30.689 INFO:teuthology.orchestra.run.smithi097.stdout:1504 read 0x206a86d thru 0x206dd06 (0x349a bytes) 2024-09-18T00:18:30.690 INFO:teuthology.orchestra.run.smithi097.stdout:1505 punch from 0x4bec773 to 0x4bee257, (0x1ae4 bytes) 2024-09-18T00:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout:1506 writesame 0xdde0ddd thru 0xdde7e05 data_size 0x11(0x7029 bytes) 2024-09-18T00:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout:1509 punch from 0x12320d2 to 0x123be79, (0x9da7 bytes) 2024-09-18T00:18:30.713 INFO:teuthology.orchestra.run.smithi097.stdout:1510 writesame 0x3e632fd thru 0x3e71c53 data_size 0xd(0xe957 bytes) 2024-09-18T00:18:30.718 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:30.715+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout:1511 punch from 0x41622c8 to 0x416a656, (0x838e bytes) 2024-09-18T00:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout:1512 read 0x6c062d0 thru 0x6c0c187 (0x5eb8 bytes) 2024-09-18T00:18:30.719 INFO:teuthology.orchestra.run.smithi097.stdout:1513 write 0x88e445b thru 0x88e50ca (0xc70 bytes) 2024-09-18T00:18:30.722 INFO:teuthology.orchestra.run.smithi097.stdout:1514 write 0x4084460 thru 0x409074b (0xc2ec bytes) 2024-09-18T00:18:30.728 INFO:teuthology.orchestra.run.smithi097.stdout:1516 write 0x99eb0a8 thru 0x99f069a (0x55f3 bytes) 2024-09-18T00:18:30.733 INFO:teuthology.orchestra.run.smithi097.stdout:1517 read 0xae7c971 thru 0xae8c4b4 (0xfb44 bytes) 2024-09-18T00:18:30.733 INFO:teuthology.orchestra.run.smithi097.stdout:1518 read 0x8d80672 thru 0x8d82624 (0x1fb3 bytes) 2024-09-18T00:18:30.733 INFO:teuthology.orchestra.run.smithi097.stdout:1519 writesame 0xdb8957a thru 0xdb8de69 data_size 0x18(0x48f0 bytes) 2024-09-18T00:18:30.738 INFO:teuthology.orchestra.run.smithi097.stdout:1520 punch from 0xd39b7c7 to 0xd3aae4b, (0xf684 bytes) 2024-09-18T00:18:30.738 INFO:teuthology.orchestra.run.smithi097.stdout:1521 write 0x78419cb thru 0x7842452 (0xa88 bytes) 2024-09-18T00:18:30.742 INFO:teuthology.orchestra.run.smithi097.stdout:1522 read 0x915a2fe thru 0x915eb48 (0x484b bytes) 2024-09-18T00:18:30.742 INFO:teuthology.orchestra.run.smithi097.stdout:1523 write 0x1e7b26e thru 0x1e88a57 (0xd7ea bytes) 2024-09-18T00:18:30.748 INFO:teuthology.orchestra.run.smithi097.stdout:1525 write 0x5aac0ec thru 0x5aadb60 (0x1a75 bytes) 2024-09-18T00:18:30.751 INFO:teuthology.orchestra.run.smithi097.stdout:1526 write 0x2a1be50 thru 0x2a2828c (0xc43d bytes) 2024-09-18T00:18:30.758 INFO:teuthology.orchestra.run.smithi097.stdout:1527 read 0x20f8b40 thru 0x20fa325 (0x17e6 bytes) 2024-09-18T00:18:30.763 INFO:teuthology.orchestra.run.smithi097.stdout:1528 trunc from 0xdde7e06 to 0xae081e1 2024-09-18T00:18:30.774 INFO:teuthology.orchestra.run.smithi097.stdout:1530 punch from 0x87dbc10 to 0x87ea467, (0xe857 bytes) 2024-09-18T00:18:30.776 INFO:teuthology.orchestra.run.smithi097.stdout:1531 trunc from 0xae081e1 to 0x9ef00b3 2024-09-18T00:18:30.785 INFO:teuthology.orchestra.run.smithi097.stdout:1532 punch from 0x96b3944 to 0x96bdb98, (0xa254 bytes) 2024-09-18T00:18:30.785 INFO:teuthology.orchestra.run.smithi097.stdout:1533 punch from 0x76ef0f3 to 0x76f1f59, (0x2e66 bytes) 2024-09-18T00:18:30.785 INFO:teuthology.orchestra.run.smithi097.stdout:1534 read 0x89341c6 thru 0x893a007 (0x5e42 bytes) 2024-09-18T00:18:30.785 INFO:teuthology.orchestra.run.smithi097.stdout:1535 write 0x3f603a2 thru 0x3f69298 (0x8ef7 bytes) 2024-09-18T00:18:30.785 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:30.782+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:30.803 INFO:teuthology.orchestra.run.smithi097.stdout:1536 trunc from 0x9ef00b3 to 0x2cc876 2024-09-18T00:18:30.838 INFO:teuthology.orchestra.run.smithi097.stdout:1537 read 0x1c4583 thru 0x1c9702 (0x5180 bytes) 2024-09-18T00:18:30.848 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:30.848 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:30.851 INFO:teuthology.orchestra.run.smithi097.stdout:1538 writesame 0x218b58c thru 0x2195120 data_size 0x1(0x9b95 bytes) 2024-09-18T00:18:30.864 INFO:teuthology.orchestra.run.smithi097.stdout:1539 writesame 0x2c6503c thru 0x2c65902 data_size 0x7(0x8c7 bytes) 2024-09-18T00:18:30.864 INFO:teuthology.orchestra.run.smithi097.stdout:1540 write 0x1b4251d thru 0x1b46efe (0x49e2 bytes) 2024-09-18T00:18:30.865 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:18:30.872 INFO:teuthology.orchestra.run.smithi097.stdout:1541 read 0x14aa713 thru 0x14b7e2b (0xd719 bytes) 2024-09-18T00:18:30.872 INFO:teuthology.orchestra.run.smithi097.stdout:1542 read 0x15ac1a9 thru 0x15b8f9a (0xcdf2 bytes) 2024-09-18T00:18:30.872 INFO:teuthology.orchestra.run.smithi097.stdout:1544 punch from 0xf82116 to 0xf89f3e, (0x7e28 bytes) 2024-09-18T00:18:30.872 INFO:teuthology.orchestra.run.smithi097.stdout:1546 read 0x2bffa5e thru 0x2c09544 (0x9ae7 bytes) 2024-09-18T00:18:30.872 INFO:teuthology.orchestra.run.smithi097.stdout:1547 read 0x4e9267 thru 0x4f8810 (0xf5aa bytes) 2024-09-18T00:18:30.884 INFO:teuthology.orchestra.run.smithi097.stdout:1550 write 0x587f80e thru 0x5889267 (0x9a5a bytes) 2024-09-18T00:18:30.892 INFO:teuthology.orchestra.run.smithi097.stdout:1552 punch from 0x110db63 to 0x1113e6a, (0x6307 bytes) 2024-09-18T00:18:30.892 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:30.892 INFO:teuthology.orchestra.run.smithi097.stdout:1554 writesame 0x1086042 thru 0x108e72b data_size 0x7(0x86ea bytes) 2024-09-18T00:18:30.908 INFO:teuthology.orchestra.run.smithi097.stdout:1555 read 0xa45baf thru 0xa460f2 (0x544 bytes) 2024-09-18T00:18:30.908 INFO:teuthology.orchestra.run.smithi097.stdout:1556 read 0x1b72f7 thru 0x1ba784 (0x348e bytes) 2024-09-18T00:18:30.908 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:30.908 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:30.910 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:30.907+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:30.937 INFO:teuthology.orchestra.run.smithi097.stdout:1559 write 0xd95bec8 thru 0xd96430f (0x8448 bytes) 2024-09-18T00:18:30.942 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:30.942 INFO:teuthology.orchestra.run.smithi097.stdout:1562 writesame 0x8c6a01f thru 0x8c7718c data_size 0xb(0xd16e bytes) 2024-09-18T00:18:30.951 INFO:teuthology.orchestra.run.smithi097.stdout:1566 write 0xa70bb7 thru 0xa7c2eb (0xb735 bytes) 2024-09-18T00:18:30.962 INFO:teuthology.orchestra.run.smithi097.stdout:1567 read 0xc809ee9 thru 0xc80a69a (0x7b2 bytes) 2024-09-18T00:18:30.962 INFO:teuthology.orchestra.run.smithi097.stdout:1568 punch from 0x4b81502 to 0x4b8a926, (0x9424 bytes) 2024-09-18T00:18:30.962 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:30.963 INFO:teuthology.orchestra.run.smithi097.stdout:1570 read 0xd7befc9 thru 0xd7c2767 (0x379f bytes) 2024-09-18T00:18:30.963 INFO:teuthology.orchestra.run.smithi097.stdout:1571 read 0xac80838 thru 0xac81f5d (0x1726 bytes) 2024-09-18T00:18:30.963 INFO:teuthology.orchestra.run.smithi097.stdout:1572 punch from 0x3d465a8 to 0x3d55aa4, (0xf4fc bytes) 2024-09-18T00:18:30.969 INFO:teuthology.orchestra.run.smithi097.stdout:1573 trunc from 0xd964310 to 0xac7f295 2024-09-18T00:18:30.986 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:30.983+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:30.989 INFO:teuthology.orchestra.run.smithi097.stdout:1574 read 0x37ffbe4 thru 0x380dc8c (0xe0a9 bytes) 2024-09-18T00:18:30.989 INFO:teuthology.orchestra.run.smithi097.stdout:1576 writesame 0x7c31da0 thru 0x7c3cbaf data_size 0x28(0xae10 bytes) 2024-09-18T00:18:30.996 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:30.997 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:30.997 INFO:teuthology.orchestra.run.smithi097.stdout:1577 punch from 0x434be1c to 0x435a46c, (0xe650 bytes) 2024-09-18T00:18:30.997 INFO:teuthology.orchestra.run.smithi097.stdout:1578 punch from 0x90a9eee to 0x90b1a09, (0x7b1b bytes) 2024-09-18T00:18:30.997 INFO:teuthology.orchestra.run.smithi097.stdout:1579 writesame 0x861dcd5 thru 0x862c43e data_size 0x13(0xe76a bytes) 2024-09-18T00:18:31.007 INFO:teuthology.orchestra.run.smithi097.stdout:1580 writesame 0x200af0f thru 0x2015703 data_size 0x1f(0xa7f5 bytes) 2024-09-18T00:18:31.012 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:18:31.017 INFO:teuthology.orchestra.run.smithi097.stdout:1581 punch from 0x64a67fa to 0x64afcb7, (0x94bd bytes) 2024-09-18T00:18:31.017 INFO:teuthology.orchestra.run.smithi097.stdout:1582 read 0x220fce5 thru 0x2219b0a (0x9e26 bytes) 2024-09-18T00:18:31.017 INFO:teuthology.orchestra.run.smithi097.stdout:1584 read 0x7829d9d thru 0x782fd27 (0x5f8b bytes) 2024-09-18T00:18:31.017 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.017 INFO:teuthology.orchestra.run.smithi097.stdout:1586 read 0x5202c21 thru 0x5212959 (0xfd39 bytes) 2024-09-18T00:18:31.017 INFO:teuthology.orchestra.run.smithi097.stdout:1587 punch from 0x4d4c7ea to 0x4d5ac5a, (0xe470 bytes) 2024-09-18T00:18:31.018 INFO:teuthology.orchestra.run.smithi097.stdout:1588 write 0x7df1c1f thru 0x7df9631 (0x7a13 bytes) 2024-09-18T00:18:31.024 INFO:teuthology.orchestra.run.smithi097.stdout:1590 write 0x824dffc thru 0x825a9d8 (0xc9dd bytes) 2024-09-18T00:18:31.034 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.034 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.034 INFO:teuthology.orchestra.run.smithi097.stdout:1593 read 0x53c0f0d thru 0x53c2f2d (0x2021 bytes) 2024-09-18T00:18:31.034 INFO:teuthology.orchestra.run.smithi097.stdout:1594 write 0x6e39d6f thru 0x6e491d0 (0xf462 bytes) 2024-09-18T00:18:31.043 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.044 INFO:teuthology.orchestra.run.smithi097.stdout:1596 read 0x83cd6cb thru 0x83cee7a (0x17b0 bytes) 2024-09-18T00:18:31.044 INFO:teuthology.orchestra.run.smithi097.stdout:1597 writesame 0x8754b36 thru 0x875d5e4 data_size 0x1(0x8aaf bytes) 2024-09-18T00:18:31.087 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:31.082+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:31.123 INFO:teuthology.orchestra.run.smithi097.stdout:1598 punch from 0x7881e83 to 0x789021a, (0xe397 bytes) 2024-09-18T00:18:31.123 INFO:teuthology.orchestra.run.smithi097.stdout:1600 read 0x8dbf0cb thru 0x8dc53e9 (0x631f bytes) 2024-09-18T00:18:31.139 INFO:teuthology.orchestra.run.smithi097.stdout:1601 writesame 0xcd181ff thru 0xcd2169e data_size 0x20(0x94a0 bytes) 2024-09-18T00:18:31.140 INFO:teuthology.orchestra.run.smithi097.stdout:1602 trunc from 0xcd2169f to 0xd137cc0 2024-09-18T00:18:31.153 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.153 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.153 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.153 INFO:teuthology.orchestra.run.smithi097.stdout:1606 read 0x7ded6ee thru 0x7dfd2b2 (0xfbc5 bytes) 2024-09-18T00:18:31.155 INFO:teuthology.orchestra.run.smithi097.stdout:1607 punch from 0x38f63c to 0x3946f7, (0x50bb bytes) 2024-09-18T00:18:31.155 INFO:teuthology.orchestra.run.smithi097.stdout:1608 write 0xbce1d5e thru 0xbcef14f (0xd3f2 bytes) 2024-09-18T00:18:31.178 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:31.178 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:31.183 INFO:teuthology.orchestra.run.smithi097.stdout:1610 writesame 0xd59b33d thru 0xd5a2747 data_size 0x1(0x740b bytes) 2024-09-18T00:18:31.184 INFO:teuthology.orchestra.run.smithi097.stdout:1613 write 0x4c89ab7 thru 0x4c8f7a1 (0x5ceb bytes) 2024-09-18T00:18:31.188 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:31.183+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:31.192 INFO:teuthology.orchestra.run.smithi097.stdout:1616 write 0xa6173b7 thru 0xa61f2a1 (0x7eeb bytes) 2024-09-18T00:18:31.193 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:18:31.199 INFO:teuthology.orchestra.run.smithi097.stdout:1617 read 0x3d79a2c thru 0x3d86b8b (0xd160 bytes) 2024-09-18T00:18:31.199 INFO:teuthology.orchestra.run.smithi097.stdout:1618 read 0xa734ee thru 0xa7c695 (0x91a8 bytes) 2024-09-18T00:18:31.201 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.201 INFO:teuthology.orchestra.run.smithi097.stdout:1625 write 0x79e343e thru 0x79f2bcb (0xf78e bytes) 2024-09-18T00:18:31.211 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.211 INFO:teuthology.orchestra.run.smithi097.stdout:1627 write 0x6a5ba92 thru 0x6a614ac (0x5a1b bytes) 2024-09-18T00:18:31.218 INFO:teuthology.orchestra.run.smithi097.stdout:1628 write 0x91b6eaa thru 0x91bb1f8 (0x434f bytes) 2024-09-18T00:18:31.226 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.226 INFO:teuthology.orchestra.run.smithi097.stdout:1630 punch from 0x3b11e23 to 0x3b15e49, (0x4026 bytes) 2024-09-18T00:18:31.226 INFO:teuthology.orchestra.run.smithi097.stdout:1631 punch from 0x58d8703 to 0x58e12e9, (0x8be6 bytes) 2024-09-18T00:18:31.226 INFO:teuthology.orchestra.run.smithi097.stdout:1632 write 0x87103c0 thru 0x8714327 (0x3f68 bytes) 2024-09-18T00:18:31.229 INFO:teuthology.orchestra.run.smithi097.stdout:1635 read 0xd3439f5 thru 0xd348cee (0x52fa bytes) 2024-09-18T00:18:31.242 INFO:teuthology.orchestra.run.smithi097.stdout:1636 trunc from 0xd5a2748 to 0x7154279 2024-09-18T00:18:31.262 INFO:teuthology.orchestra.run.smithi097.stdout:1637 read 0x276b65c thru 0x2771062 (0x5a07 bytes) 2024-09-18T00:18:31.262 INFO:teuthology.orchestra.run.smithi097.stdout:1640 read 0x191d5f7 thru 0x192c83e (0xf248 bytes) 2024-09-18T00:18:31.262 INFO:teuthology.orchestra.run.smithi097.stdout:1641 read 0x334b1e4 thru 0x33547e1 (0x95fe bytes) 2024-09-18T00:18:31.262 INFO:teuthology.orchestra.run.smithi097.stdout:1642 punch from 0x3d2f58e to 0x3d2f5a4, (0x16 bytes) 2024-09-18T00:18:31.283 INFO:teuthology.orchestra.run.smithi097.stdout:1643 writesame 0xbfef034 thru 0xbff7633 data_size 0x43(0x8600 bytes) 2024-09-18T00:18:31.283 INFO:teuthology.orchestra.run.smithi097.stdout:1645 read 0xac1df4e thru 0xac2c086 (0xe139 bytes) 2024-09-18T00:18:31.284 INFO:teuthology.orchestra.run.smithi097.stdout:1646 writesame 0x57635e7 thru 0x57717e8 data_size 0x3(0xe202 bytes) 2024-09-18T00:18:31.289 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:31.283+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:31.305 INFO:teuthology.orchestra.run.smithi097.stdout:1647 write 0xb0a8786 thru 0xb0ac5a0 (0x3e1b bytes) 2024-09-18T00:18:31.330 INFO:teuthology.orchestra.run.smithi097.stdout:1648 write 0xd6b9fdf thru 0xd6bfff1 (0x6013 bytes) 2024-09-18T00:18:31.336 INFO:teuthology.orchestra.run.smithi097.stdout:1649 writesame 0x3a9240e thru 0x3a9d150 data_size 0x5(0xad43 bytes) 2024-09-18T00:18:31.348 INFO:teuthology.orchestra.run.smithi097.stdout:1650 read 0xc6ad56 thru 0xc782e7 (0xd592 bytes) 2024-09-18T00:18:31.348 INFO:teuthology.orchestra.run.smithi097.stdout:1651 read 0x31f86ac thru 0x320608a (0xd9df bytes) 2024-09-18T00:18:31.348 INFO:teuthology.orchestra.run.smithi097.stdout:1652 write 0x1a999bd thru 0x1a9e12b (0x476f bytes) 2024-09-18T00:18:31.355 INFO:teuthology.orchestra.run.smithi097.stdout:1653 write 0x692c88a thru 0x6935570 (0x8ce7 bytes) 2024-09-18T00:18:31.364 INFO:teuthology.orchestra.run.smithi097.stdout:1654 read 0xd54e42e thru 0xd55d43f (0xf012 bytes) 2024-09-18T00:18:31.364 INFO:teuthology.orchestra.run.smithi097.stdout:1656 read 0xb26db2a thru 0xb270cd6 (0x31ad bytes) 2024-09-18T00:18:31.364 INFO:teuthology.orchestra.run.smithi097.stdout:1657 flatten 2024-09-18T00:18:31.374 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:31.374 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:31.391 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:18:31.391 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:31.387+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:31.392 INFO:teuthology.orchestra.run.smithi097.stdout:1658 read 0x6339461 thru 0x634601b (0xcbbb bytes) 2024-09-18T00:18:31.392 INFO:teuthology.orchestra.run.smithi097.stdout:1659 writesame 0x3380627 thru 0x3384823 data_size 0x3(0x41fd bytes) 2024-09-18T00:18:31.411 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.411 INFO:teuthology.orchestra.run.smithi097.stdout:1662 read 0x3bbb1fd thru 0x3bc6019 (0xae1d bytes) 2024-09-18T00:18:31.411 INFO:teuthology.orchestra.run.smithi097.stdout:1663 write 0x794a89b thru 0x794b4e4 (0xc4a bytes) 2024-09-18T00:18:31.414 INFO:teuthology.orchestra.run.smithi097.stdout:3.6 2024-09-18T00:18:31.414 INFO:teuthology.orchestra.run.smithi097.stdout:3.8 2024-09-18T00:18:31.414 INFO:teuthology.orchestra.run.smithi097.stdout:3.4 2024-09-18T00:18:31.415 INFO:teuthology.orchestra.run.smithi097.stdout:3.1a 2024-09-18T00:18:31.415 INFO:teuthology.orchestra.run.smithi097.stdout:3.19 2024-09-18T00:18:31.415 INFO:teuthology.orchestra.run.smithi097.stdout:3.0 2024-09-18T00:18:31.415 INFO:teuthology.orchestra.run.smithi097.stdout:2.6 2024-09-18T00:18:31.415 INFO:teuthology.orchestra.run.smithi097.stdout:2.4 2024-09-18T00:18:31.415 INFO:teuthology.orchestra.run.smithi097.stdout:2.2 2024-09-18T00:18:31.415 INFO:teuthology.orchestra.run.smithi097.stdout:2.0 2024-09-18T00:18:31.417 INFO:teuthology.orchestra.run.smithi097.stdout:1664 read 0x4f7133a thru 0x4f7e5b5 (0xd27c bytes) 2024-09-18T00:18:31.417 INFO:teuthology.orchestra.run.smithi097.stdout:1665 punch from 0xc2580b to 0xc3166f, (0xbe64 bytes) 2024-09-18T00:18:31.417 INFO:teuthology.orchestra.run.smithi097.stdout:1666 read 0x2ba9d02 thru 0x2bb8749 (0xea48 bytes) 2024-09-18T00:18:31.417 INFO:teuthology.orchestra.run.smithi097.stdout:1667 read 0x49da73d thru 0x49e4814 (0xa0d8 bytes) 2024-09-18T00:18:31.417 INFO:teuthology.orchestra.run.smithi097.stdout:1668 writesame 0x7652a47 thru 0x765ad0e data_size 0x7c(0x82c8 bytes) 2024-09-18T00:18:31.423 INFO:teuthology.orchestra.run.smithi097.stdout:1669 write 0xad475ff thru 0xad55428 (0xde2a bytes) 2024-09-18T00:18:31.430 INFO:teuthology.orchestra.run.smithi097.stdout:1671 punch from 0xc1a8c8f to 0xc1ab107, (0x2478 bytes) 2024-09-18T00:18:31.430 INFO:teuthology.orchestra.run.smithi097.stdout:1672 read 0x456439f thru 0x45647f7 (0x459 bytes) 2024-09-18T00:18:31.430 INFO:teuthology.orchestra.run.smithi097.stdout:1673 write 0x4a1e2e9 thru 0x4a26f85 (0x8c9d bytes) 2024-09-18T00:18:31.455 INFO:teuthology.orchestra.run.smithi097.stdout:1674 trunc from 0xd6bfff2 to 0x323e2e0 2024-09-18T00:18:31.491 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:31.487+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:31.496 INFO:teuthology.orchestra.run.smithi097.stdout:1675 writesame 0x93160a1 thru 0x9318580 data_size 0xa(0x24e0 bytes) 2024-09-18T00:18:31.497 INFO:teuthology.orchestra.run.smithi097.stdout:1676 read 0x203f9f7 thru 0x2041aa8 (0x20b2 bytes) 2024-09-18T00:18:31.498 INFO:teuthology.orchestra.run.smithi097.stdout:1678 trunc from 0x9318581 to 0x9fe5649 2024-09-18T00:18:31.516 INFO:teuthology.orchestra.run.smithi097.stdout:1679 write 0xab1d537 thru 0xab25ed7 (0x89a1 bytes) 2024-09-18T00:18:31.523 INFO:teuthology.orchestra.run.smithi097.stdout:1680 write 0x219e231 thru 0x21a2cfb (0x4acb bytes) 2024-09-18T00:18:31.532 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.532 INFO:teuthology.orchestra.run.smithi097.stdout:1682 read 0x741fb02 thru 0x742063a (0xb39 bytes) 2024-09-18T00:18:31.532 INFO:teuthology.orchestra.run.smithi097.stdout:1683 punch from 0x4fe42de to 0x4fe9b2c, (0x584e bytes) 2024-09-18T00:18:31.532 INFO:teuthology.orchestra.run.smithi097.stdout:1684 write 0x7f16a3e thru 0x7f1f317 (0x88da bytes) 2024-09-18T00:18:31.541 INFO:teuthology.orchestra.run.smithi097.stdout:1687 write 0x80184e5 thru 0x801998e (0x14aa bytes) 2024-09-18T00:18:31.546 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.546 INFO:teuthology.orchestra.run.smithi097.stdout:1690 read 0x1aa40e3 thru 0x1ab4011 (0xff2f bytes) 2024-09-18T00:18:31.547 INFO:teuthology.orchestra.run.smithi097.stdout:1692 write 0x30e6551 thru 0x30f20d0 (0xbb80 bytes) 2024-09-18T00:18:31.555 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.564 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is set 2024-09-18T00:18:31.568 INFO:teuthology.orchestra.run.smithi097.stdout:1694 write 0xb305596 thru 0xb30e64b (0x90b6 bytes) 2024-09-18T00:18:31.571 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:31.571 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:31.573 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.573 INFO:teuthology.orchestra.run.smithi097.stdout:1698 writesame 0x65b71ea thru 0x65bc1ac data_size 0x1(0x4fc3 bytes) 2024-09-18T00:18:31.591 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:18:31.619 INFO:teuthology.orchestra.run.smithi097.stdout:1699 read 0x552efc9 thru 0x5533dc0 (0x4df8 bytes) 2024-09-18T00:18:31.619 INFO:teuthology.orchestra.run.smithi097.stdout:1700 write 0x14b322a thru 0x14c07b0 (0xd587 bytes) 2024-09-18T00:18:31.633 INFO:teuthology.orchestra.run.smithi097.stdout:1702 read 0x9fb5e8c thru 0x9fbabf7 (0x4d6c bytes) 2024-09-18T00:18:31.633 INFO:teuthology.orchestra.run.smithi097.stdout:1704 read 0x23e410a thru 0x23eff41 (0xbe38 bytes) 2024-09-18T00:18:31.634 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:31.631+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:31.646 INFO:teuthology.orchestra.run.smithi097.stdout:1708 write 0xdfaa284 thru 0xdfaac51 (0x9ce bytes) 2024-09-18T00:18:31.652 INFO:teuthology.orchestra.run.smithi097.stdout:1709 trunc from 0xdfaac52 to 0xc6f5b0e 2024-09-18T00:18:31.664 INFO:teuthology.orchestra.run.smithi097.stdout:1710 punch from 0x74fc057 to 0x7501659, (0x5602 bytes) 2024-09-18T00:18:31.664 INFO:teuthology.orchestra.run.smithi097.stdout:1711 read 0x8cd5770 thru 0x8cd90da (0x396b bytes) 2024-09-18T00:18:31.687 INFO:teuthology.orchestra.run.smithi097.stdout:1712 trunc from 0xc6f5b0e to 0xb0a72a 2024-09-18T00:18:31.693 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:31.690+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:31.725 INFO:teuthology.orchestra.run.smithi097.stdout:1713 trunc from 0xb0a72a to 0xad6a687 2024-09-18T00:18:31.738 INFO:teuthology.orchestra.run.smithi097.stdout:1714 trunc from 0xad6a687 to 0x8dae797 2024-09-18T00:18:31.760 INFO:teuthology.orchestra.run.smithi097.stdout:1715 trunc from 0x8dae797 to 0xe68da17 2024-09-18T00:18:31.765 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:31.766 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:31.772 INFO:teuthology.orchestra.run.smithi097.stdout:1718 write 0xb3af32a thru 0xb3b5eaf (0x6b86 bytes) 2024-09-18T00:18:31.782 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:18:31.782 INFO:teuthology.orchestra.run.smithi097.stdout:1719 read 0xb5316a9 thru 0xb534514 (0x2e6c bytes) 2024-09-18T00:18:31.783 INFO:teuthology.orchestra.run.smithi097.stdout:1720 writesame 0xd79fda6 thru 0xd7a1f8d data_size 0x28(0x21e8 bytes) 2024-09-18T00:18:31.787 INFO:teuthology.orchestra.run.smithi097.stdout:1721 punch from 0x4e97f9d to 0x4e99d07, (0x1d6a bytes) 2024-09-18T00:18:31.787 INFO:teuthology.orchestra.run.smithi097.stdout:1722 read 0x1fa91b3 thru 0x1fb420b (0xb059 bytes) 2024-09-18T00:18:31.790 INFO:teuthology.orchestra.run.smithi097.stdout:1723 trunc from 0xe68da17 to 0xcfa7de9 2024-09-18T00:18:31.794 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:31.790+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:31.823 INFO:teuthology.orchestra.run.smithi097.stdout:1724 trunc from 0xcfa7de9 to 0x1b87d93 2024-09-18T00:18:31.852 INFO:teuthology.orchestra.run.smithi097.stdout:1725 write 0x24bc8df thru 0x24cc4c4 (0xfbe6 bytes) 2024-09-18T00:18:31.859 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.859 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:31.860 INFO:teuthology.orchestra.run.smithi097.stdout:1728 trunc from 0x24cc4c5 to 0x2b0c551 2024-09-18T00:18:31.874 INFO:teuthology.orchestra.run.smithi097.stdout:1729 read 0x22ffc9f thru 0x230624d (0x65af bytes) 2024-09-18T00:18:31.876 INFO:teuthology.orchestra.run.smithi097.stdout:1730 trunc from 0x2b0c551 to 0x231f821 2024-09-18T00:18:31.895 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:31.890+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:31.901 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:31.901 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:31.915 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:18:31.940 INFO:teuthology.orchestra.run.smithi097.stdout:1731 write 0xcf70f0f thru 0xcf76448 (0x553a bytes) 2024-09-18T00:18:31.946 INFO:teuthology.orchestra.run.smithi097.stdout:1732 read 0x7315c24 thru 0x7318a82 (0x2e5f bytes) 2024-09-18T00:18:31.946 INFO:teuthology.orchestra.run.smithi097.stdout:1733 punch from 0xbbdc798 to 0xbbdfb0d, (0x3375 bytes) 2024-09-18T00:18:31.946 INFO:teuthology.orchestra.run.smithi097.stdout:1734 trunc from 0xcf76449 to 0xcfe5242 2024-09-18T00:18:31.955 INFO:teuthology.orchestra.run.smithi097.stdout:1735 punch from 0x4c350e5 to 0x4c3da3f, (0x895a bytes) 2024-09-18T00:18:31.958 INFO:teuthology.orchestra.run.smithi097.stdout:1736 trunc from 0xcfe5242 to 0xbdd1319 2024-09-18T00:18:31.976 INFO:teuthology.orchestra.run.smithi097.stdout:1737 write 0x377299f thru 0x377cc03 (0xa265 bytes) 2024-09-18T00:18:31.983 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op export --pgid 3.8 --file /var/log/ceph/exp.3.8.1 2024-09-18T00:18:31.984 INFO:teuthology.orchestra.run.smithi097.stdout:1738 writesame 0x1321cd7 thru 0x132a6cd data_size 0x3d(0x89f7 bytes) 2024-09-18T00:18:31.991 INFO:teuthology.orchestra.run.smithi097.stdout:1741 trunc from 0xbdd1319 to 0xa8f8831 2024-09-18T00:18:31.996 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:31.990+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:32.024 INFO:teuthology.orchestra.run.smithi097.stdout:1742 trunc from 0xa8f8831 to 0x2b31a7 2024-09-18T00:18:32.046 INFO:teuthology.orchestra.run.smithi097.stdout:1743 read 0x14c561 thru 0x1591de (0xcc7e bytes) 2024-09-18T00:18:32.046 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:32.047 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:32.060 INFO:teuthology.orchestra.run.smithi097.stdout:1744 trunc from 0x2b31a7 to 0x8143292 2024-09-18T00:18:32.061 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:18:32.069 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:32.070 INFO:teuthology.orchestra.run.smithi097.stdout:1746 write 0x74e0020 thru 0x74ea01e (0x9fff bytes) 2024-09-18T00:18:32.076 INFO:teuthology.orchestra.run.smithi097.stdout:1747 punch from 0x60dfbb5 to 0x60e9b4b, (0x9f96 bytes) 2024-09-18T00:18:32.077 INFO:teuthology.orchestra.run.smithi097.stdout:1748 write 0x64bf293 thru 0x64c4ca7 (0x5a15 bytes) 2024-09-18T00:18:32.083 INFO:teuthology.orchestra.run.smithi097.stdout:1749 write 0xb02ab thru 0xbc4f2 (0xc248 bytes) 2024-09-18T00:18:32.086 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:32.091 INFO:teuthology.orchestra.run.smithi097.stdout:1751 trunc from 0x8143292 to 0x52dfae7 2024-09-18T00:18:32.097 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:32.091+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:32.113 INFO:teuthology.orchestra.run.smithi097.stdout:1752 write 0x53ab950 thru 0x53b7aa7 (0xc158 bytes) 2024-09-18T00:18:32.122 INFO:teuthology.orchestra.run.smithi097.stdout:1753 trunc from 0x53b7aa8 to 0x8136db5 2024-09-18T00:18:32.129 INFO:teuthology.orchestra.run.smithi097.stdout:1756 writesame 0x1dcefd4 thru 0x1dd15f7 data_size 0x2(0x2624 bytes) 2024-09-18T00:18:32.143 INFO:teuthology.orchestra.run.smithi097.stdout:1757 trunc from 0x8136db5 to 0xbf017f8 2024-09-18T00:18:32.194 INFO:teuthology.orchestra.run.smithi097.stdout:1758 punch from 0x3d924b8 to 0x3d96b97, (0x46df bytes) 2024-09-18T00:18:32.194 INFO:teuthology.orchestra.run.smithi097.stdout:1760 read 0x11ae899 thru 0x11bc440 (0xdba8 bytes) 2024-09-18T00:18:32.194 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:32.194 INFO:teuthology.orchestra.run.smithi097.stdout:1762 write 0x87e338e thru 0x87eb603 (0x8276 bytes) 2024-09-18T00:18:32.194 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:32.195 INFO:teuthology.orchestra.run.smithi097.stdout:1764 punch from 0x7bc85be to 0x7bd3a15, (0xb457 bytes) 2024-09-18T00:18:32.195 INFO:teuthology.orchestra.run.smithi097.stdout:1765 write 0xd280acd thru 0xd282a2f (0x1f63 bytes) 2024-09-18T00:18:32.195 INFO:teuthology.orchestra.run.smithi097.stdout:1766 write 0x64ae95b thru 0x64b9169 (0xa80f bytes) 2024-09-18T00:18:32.195 INFO:teuthology.orchestra.run.smithi097.stdout:1767 read 0x1855a86 thru 0x185e9a8 (0x8f23 bytes) 2024-09-18T00:18:32.195 INFO:teuthology.orchestra.run.smithi097.stdout:1769 punch from 0x2e3d331 to 0x2e49620, (0xc2ef bytes) 2024-09-18T00:18:32.198 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:32.198 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:32.198 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:32.198 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:32.198 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:32.195+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:32.217 INFO:teuthology.orchestra.run.smithi097.stdout:1770 trunc from 0xd282a30 to 0x5f0914 2024-09-18T00:18:32.221 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:18:32.276 INFO:teuthology.orchestra.run.smithi097.stdout:1771 writesame 0xe65cefa thru 0xe6633f6 data_size 0x1(0x64fd bytes) 2024-09-18T00:18:32.276 INFO:teuthology.orchestra.run.smithi097.stdout:1772 writesame 0x985ebb2 thru 0x986bd2b data_size 0x1(0xd17a bytes) 2024-09-18T00:18:32.300 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:32.295+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:32.385 INFO:teuthology.orchestra.run.smithi097.stdout:1773 writesame 0x73a44c9 thru 0x73ac2f2 data_size 0x15(0x7e2a bytes) 2024-09-18T00:18:32.393 INFO:teuthology.orchestra.run.smithi097.stdout:1774 punch from 0x9028639 to 0x9035b91, (0xd558 bytes) 2024-09-18T00:18:32.393 INFO:teuthology.orchestra.run.smithi097.stdout:1776 read 0x1efe9a0 thru 0x1f04d3b (0x639c bytes) 2024-09-18T00:18:32.394 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:32.394 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:32.394 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:32.394 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:32.394 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:32.400 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:32.398+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:32.403 INFO:teuthology.orchestra.run.smithi097.stdout:1777 trunc from 0xe6633f7 to 0x7957178 2024-09-18T00:18:32.415 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:18:32.426 INFO:teuthology.orchestra.run.smithi097.stdout:1778 read 0x4a32280 thru 0x4a33a53 (0x17d4 bytes) 2024-09-18T00:18:32.436 INFO:teuthology.orchestra.run.smithi097.stdout:1779 trunc from 0x7957178 to 0xc76f8b 2024-09-18T00:18:32.477 INFO:teuthology.orchestra.run.smithi097.stdout:1780 write 0x56870b4 thru 0x569553a (0xe487 bytes) 2024-09-18T00:18:32.484 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:32.501 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:32.498+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:32.505 INFO:teuthology.orchestra.run.smithi097.stdout:1783 write 0xc05b652 thru 0xc066c80 (0xb62f bytes) 2024-09-18T00:18:32.518 INFO:teuthology.orchestra.run.smithi097.stdout:1784 read 0x9d783c0 thru 0x9d86681 (0xe2c2 bytes) 2024-09-18T00:18:32.518 INFO:teuthology.orchestra.run.smithi097.stdout:1785 read 0x5eb1357 thru 0x5ebe3cd (0xd077 bytes) 2024-09-18T00:18:32.518 INFO:teuthology.orchestra.run.smithi097.stdout:1787 punch from 0x21e9e19 to 0x21f02ea, (0x64d1 bytes) 2024-09-18T00:18:32.518 INFO:teuthology.orchestra.run.smithi097.stdout:1790 clone 6 order 19 su 8192 sc 8 2024-09-18T00:18:32.558 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:32.559 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:32.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:32.559 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:32.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.559 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2230 3.17 3:ef492888:::rbd_data.112aa7ba4578.000000000000014f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:18:32.559 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:32.559 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.290208379999999, 2024-09-18T00:18:32.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075065283999999996, 2024-09-18T00:18:32.559 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.559 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.559 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:32.560 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:32.560 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:32.560 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:32.560 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2230 2024-09-18T00:18:32.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.560 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:32.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:32.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266014+0000", 2024-09-18T00:18:32.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:18:32.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266029+0000", 2024-09-18T00:18:32.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4616e-05 2024-09-18T00:18:32.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266053+0000", 2024-09-18T00:18:32.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4369999999999999e-05 2024-09-18T00:18:32.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266067+0000", 2024-09-18T00:18:32.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4243999999999999e-05 2024-09-18T00:18:32.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266157+0000", 2024-09-18T00:18:32.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9909999999999998e-05 2024-09-18T00:18:32.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266327+0000", 2024-09-18T00:18:32.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016981200000000001 2024-09-18T00:18:32.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:32.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269180+0000", 2024-09-18T00:18:32.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028526559999999999 2024-09-18T00:18:32.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:32.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340955+0000", 2024-09-18T00:18:32.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.071774904 2024-09-18T00:18:32.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:32.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340974+0000", 2024-09-18T00:18:32.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8824000000000001e-05 2024-09-18T00:18:32.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:32.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341028+0000", 2024-09-18T00:18:32.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4456999999999997e-05 2024-09-18T00:18:32.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341076+0000", 2024-09-18T00:18:32.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8284000000000001e-05 2024-09-18T00:18:32.565 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.565 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.565 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.566 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2229 3.10 e36/23 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head v 36'8, mlcod=35'6)", 2024-09-18T00:18:32.566 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.269628+0000", 2024-09-18T00:18:32.566 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.286591566, 2024-09-18T00:18:32.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013356332, 2024-09-18T00:18:32.566 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.566 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.566 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:32.566 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269628+0000", 2024-09-18T00:18:32.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269628+0000", 2024-09-18T00:18:32.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269630+0000", 2024-09-18T00:18:32.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2299999999999998e-06 2024-09-18T00:18:32.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.281469+0000", 2024-09-18T00:18:32.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011838589 2024-09-18T00:18:32.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.281505+0000", 2024-09-18T00:18:32.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6412999999999998e-05 2024-09-18T00:18:32.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.281523+0000", 2024-09-18T00:18:32.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8122999999999999e-05 2024-09-18T00:18:32.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.281695+0000", 2024-09-18T00:18:32.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017173500000000001 2024-09-18T00:18:32.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.281748+0000", 2024-09-18T00:18:32.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3059999999999997e-05 2024-09-18T00:18:32.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:32.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.282951+0000", 2024-09-18T00:18:32.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001203345 2024-09-18T00:18:32.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.282984+0000", 2024-09-18T00:18:32.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2836999999999997e-05 2024-09-18T00:18:32.570 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.570 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.570 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.570 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2230 3.17 e36/23 ondisk, result = 0)", 2024-09-18T00:18:32.570 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:32.570 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.272864799000001, 2024-09-18T00:18:32.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057609645000000001, 2024-09-18T00:18:32.570 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.570 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.571 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:32.571 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:32.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:32.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283356+0000", 2024-09-18T00:18:32.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.44e-06 2024-09-18T00:18:32.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283362+0000", 2024-09-18T00:18:32.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2539999999999998e-06 2024-09-18T00:18:32.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283382+0000", 2024-09-18T00:18:32.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9896000000000001e-05 2024-09-18T00:18:32.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283398+0000", 2024-09-18T00:18:32.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5489999999999999e-05 2024-09-18T00:18:32.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340936+0000", 2024-09-18T00:18:32.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057538649999999997 2024-09-18T00:18:32.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340947+0000", 2024-09-18T00:18:32.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0981e-05 2024-09-18T00:18:32.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340964+0000", 2024-09-18T00:18:32.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6934000000000001e-05 2024-09-18T00:18:32.575 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.575 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.575 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.575 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2419 3.17 3:ed94b133:::rbd_data.112aa7ba4578.00000000000001b5:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:32.575 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:18:32.575 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 11.818743326, 2024-09-18T00:18:32.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.064761812000000002, 2024-09-18T00:18:32.575 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.575 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.575 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:32.575 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:32.576 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:32.576 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:32.576 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2419 2024-09-18T00:18:32.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.576 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:18:32.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:18:32.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737479+0000", 2024-09-18T00:18:32.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6089999999999999e-06 2024-09-18T00:18:32.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737487+0000", 2024-09-18T00:18:32.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7900000000000005e-06 2024-09-18T00:18:32.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737507+0000", 2024-09-18T00:18:32.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9936e-05 2024-09-18T00:18:32.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737518+0000", 2024-09-18T00:18:32.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1046999999999999e-05 2024-09-18T00:18:32.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737599+0000", 2024-09-18T00:18:32.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0697000000000003e-05 2024-09-18T00:18:32.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737697+0000", 2024-09-18T00:18:32.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8058000000000002e-05 2024-09-18T00:18:32.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:32.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.740445+0000", 2024-09-18T00:18:32.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027481279999999999 2024-09-18T00:18:32.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:32.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802023+0000", 2024-09-18T00:18:32.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.061577321999999997 2024-09-18T00:18:32.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:32.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802060+0000", 2024-09-18T00:18:32.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7586000000000002e-05 2024-09-18T00:18:32.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:32.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802158+0000", 2024-09-18T00:18:32.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7583000000000004e-05 2024-09-18T00:18:32.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802238+0000", 2024-09-18T00:18:32.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0055999999999994e-05 2024-09-18T00:18:32.581 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.581 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.581 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.581 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2419 3.17 e37/23 ondisk, result = 0)", 2024-09-18T00:18:32.581 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:18:32.581 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 11.80651838, 2024-09-18T00:18:32.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052342948, 2024-09-18T00:18:32.581 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.582 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.582 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:32.582 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:18:32.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:18:32.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749702+0000", 2024-09-18T00:18:32.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:18:32.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749710+0000", 2024-09-18T00:18:32.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7279999999999995e-06 2024-09-18T00:18:32.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749736+0000", 2024-09-18T00:18:32.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5797999999999999e-05 2024-09-18T00:18:32.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749751+0000", 2024-09-18T00:18:32.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5238999999999999e-05 2024-09-18T00:18:32.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801745+0000", 2024-09-18T00:18:32.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051994022000000001 2024-09-18T00:18:32.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801770+0000", 2024-09-18T00:18:32.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5103e-05 2024-09-18T00:18:32.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802044+0000", 2024-09-18T00:18:32.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027395799999999998 2024-09-18T00:18:32.585 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.585 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.586 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.586 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:5715 3.1 3:8061bf9f:::rbd_data.112a3a39fc45.000000000000001a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:32.586 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.316873+0000", 2024-09-18T00:18:32.586 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.2393466860000002, 2024-09-18T00:18:32.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.024653944000000001, 2024-09-18T00:18:32.586 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.586 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.586 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:32.586 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:32.586 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:32.586 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:32.587 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 5715 2024-09-18T00:18:32.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.587 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.316873+0000", 2024-09-18T00:18:32.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.316873+0000", 2024-09-18T00:18:32.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.316874+0000", 2024-09-18T00:18:32.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4470000000000001e-06 2024-09-18T00:18:32.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.317484+0000", 2024-09-18T00:18:32.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00061002500000000004 2024-09-18T00:18:32.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.317497+0000", 2024-09-18T00:18:32.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.242e-05 2024-09-18T00:18:32.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.317503+0000", 2024-09-18T00:18:32.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.657e-06 2024-09-18T00:18:32.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.317556+0000", 2024-09-18T00:18:32.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2774e-05 2024-09-18T00:18:32.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.317670+0000", 2024-09-18T00:18:32.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000114082 2024-09-18T00:18:32.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:32.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.317833+0000", 2024-09-18T00:18:32.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000163225 2024-09-18T00:18:32.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:32.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.322911+0000", 2024-09-18T00:18:32.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0050774419999999997 2024-09-18T00:18:32.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:32.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.341448+0000", 2024-09-18T00:18:32.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018537413999999999 2024-09-18T00:18:32.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:32.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.341490+0000", 2024-09-18T00:18:32.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1545000000000003e-05 2024-09-18T00:18:32.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.341527+0000", 2024-09-18T00:18:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6912999999999997e-05 2024-09-18T00:18:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:5719 3.1b 3:d97acbb6:::rbd_data.112a3a39fc45.000000000000001b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.328713+0000", 2024-09-18T00:18:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.227506762, 2024-09-18T00:18:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017834642000000001, 2024-09-18T00:18:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 5719 2024-09-18T00:18:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.328713+0000", 2024-09-18T00:18:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.328713+0000", 2024-09-18T00:18:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.328714+0000", 2024-09-18T00:18:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3680000000000001e-06 2024-09-18T00:18:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.329168+0000", 2024-09-18T00:18:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00045356299999999998 2024-09-18T00:18:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.329181+0000", 2024-09-18T00:18:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.309e-05 2024-09-18T00:18:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.329190+0000", 2024-09-18T00:18:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3000000000000007e-06 2024-09-18T00:18:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.329247+0000", 2024-09-18T00:18:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7357e-05 2024-09-18T00:18:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.329359+0000", 2024-09-18T00:18:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011159599999999999 2024-09-18T00:18:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.329518+0000", 2024-09-18T00:18:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015909 2024-09-18T00:18:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336470+0000", 2024-09-18T00:18:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0069525139999999999 2024-09-18T00:18:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.346427+0000", 2024-09-18T00:18:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099561249999999997 2024-09-18T00:18:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.346497+0000", 2024-09-18T00:18:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0149999999999998e-05 2024-09-18T00:18:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.346547+0000", 2024-09-18T00:18:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0488999999999997e-05 2024-09-18T00:18:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:5802 3.17 3:eb061e39:::rbd_data.112a3a39fc45.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 1583919~42129] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.387585+0000", 2024-09-18T00:18:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.1686345469999999, 2024-09-18T00:18:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015255879999999999, 2024-09-18T00:18:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 5802 2024-09-18T00:18:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.387585+0000", 2024-09-18T00:18:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.387585+0000", 2024-09-18T00:18:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.387586+0000", 2024-09-18T00:18:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1820000000000001e-06 2024-09-18T00:18:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.388067+0000", 2024-09-18T00:18:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048087299999999999 2024-09-18T00:18:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.388082+0000", 2024-09-18T00:18:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4848000000000001e-05 2024-09-18T00:18:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.388088+0000", 2024-09-18T00:18:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6640000000000001e-06 2024-09-18T00:18:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.388139+0000", 2024-09-18T00:18:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0210000000000002e-05 2024-09-18T00:18:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.388241+0000", 2024-09-18T00:18:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102277 2024-09-18T00:18:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.388401+0000", 2024-09-18T00:18:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016011000000000001 2024-09-18T00:18:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.396686+0000", 2024-09-18T00:18:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0082845869999999995 2024-09-18T00:18:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.402743+0000", 2024-09-18T00:18:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0060578000000000003 2024-09-18T00:18:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.402788+0000", 2024-09-18T00:18:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4796999999999998e-05 2024-09-18T00:18:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.402841+0000", 2024-09-18T00:18:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2531999999999998e-05 2024-09-18T00:18:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6308 3.6 3:6104537f:::rbd_data.112a960351d8.0000000000000004:6 [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.476537+0000", 2024-09-18T00:18:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.0796822769999999, 2024-09-18T00:18:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010554559, 2024-09-18T00:18:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6308 2024-09-18T00:18:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476537+0000", 2024-09-18T00:18:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476537+0000", 2024-09-18T00:18:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476538+0000", 2024-09-18T00:18:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.006e-06 2024-09-18T00:18:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476543+0000", 2024-09-18T00:18:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7400000000000004e-06 2024-09-18T00:18:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476560+0000", 2024-09-18T00:18:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6837000000000002e-05 2024-09-18T00:18:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476573+0000", 2024-09-18T00:18:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.29e-05 2024-09-18T00:18:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.486979+0000", 2024-09-18T00:18:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010406500000000001 2024-09-18T00:18:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487032+0000", 2024-09-18T00:18:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2636999999999998e-05 2024-09-18T00:18:32.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487092+0000", 2024-09-18T00:18:32.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9939000000000003e-05 2024-09-18T00:18:32.609 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.609 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.609 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.609 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6357 3.6 3:6104537f:::rbd_data.112a960351d8.0000000000000004:6 [sparse-read 4849664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:32.609 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.476621+0000", 2024-09-18T00:18:32.609 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.079598764, 2024-09-18T00:18:32.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010676343, 2024-09-18T00:18:32.610 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.610 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.610 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:32.610 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:32.610 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:32.610 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:32.610 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6357 2024-09-18T00:18:32.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.610 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476621+0000", 2024-09-18T00:18:32.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476621+0000", 2024-09-18T00:18:32.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476622+0000", 2024-09-18T00:18:32.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2359999999999999e-06 2024-09-18T00:18:32.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476642+0000", 2024-09-18T00:18:32.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.039e-05 2024-09-18T00:18:32.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476662+0000", 2024-09-18T00:18:32.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9939999999999999e-05 2024-09-18T00:18:32.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476677+0000", 2024-09-18T00:18:32.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5276000000000001e-05 2024-09-18T00:18:32.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487116+0000", 2024-09-18T00:18:32.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010438517 2024-09-18T00:18:32.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487187+0000", 2024-09-18T00:18:32.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1271000000000003e-05 2024-09-18T00:18:32.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487297+0000", 2024-09-18T00:18:32.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000109713 2024-09-18T00:18:32.614 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.614 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.614 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.614 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6358 3.6 3:6104537f:::rbd_data.112a960351d8.0000000000000004:6 [sparse-read 5636096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:32.614 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.476698+0000", 2024-09-18T00:18:32.614 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.0795211819999997, 2024-09-18T00:18:32.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010837404, 2024-09-18T00:18:32.614 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.614 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.614 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:32.615 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:32.615 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:32.615 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:32.615 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6358 2024-09-18T00:18:32.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.615 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476698+0000", 2024-09-18T00:18:32.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476698+0000", 2024-09-18T00:18:32.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476699+0000", 2024-09-18T00:18:32.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.19e-06 2024-09-18T00:18:32.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476705+0000", 2024-09-18T00:18:32.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3070000000000002e-06 2024-09-18T00:18:32.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476722+0000", 2024-09-18T00:18:32.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7816999999999999e-05 2024-09-18T00:18:32.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476739+0000", 2024-09-18T00:18:32.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6633000000000002e-05 2024-09-18T00:18:32.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487336+0000", 2024-09-18T00:18:32.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01059647 2024-09-18T00:18:32.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487426+0000", 2024-09-18T00:18:32.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0166000000000004e-05 2024-09-18T00:18:32.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487536+0000", 2024-09-18T00:18:32.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000109821 2024-09-18T00:18:32.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.619 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.619 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6359 3.6 3:6104537f:::rbd_data.112a960351d8.0000000000000004:6 [sparse-read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:32.619 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.476756+0000", 2024-09-18T00:18:32.619 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.0794631749999999, 2024-09-18T00:18:32.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012421049999999999, 2024-09-18T00:18:32.619 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.620 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.620 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:32.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:32.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:32.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:32.620 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6359 2024-09-18T00:18:32.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.620 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476756+0000", 2024-09-18T00:18:32.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476756+0000", 2024-09-18T00:18:32.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476757+0000", 2024-09-18T00:18:32.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.136e-06 2024-09-18T00:18:32.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476762+0000", 2024-09-18T00:18:32.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0540000000000002e-06 2024-09-18T00:18:32.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476779+0000", 2024-09-18T00:18:32.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6883000000000001e-05 2024-09-18T00:18:32.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476792+0000", 2024-09-18T00:18:32.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3066e-05 2024-09-18T00:18:32.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488807+0000", 2024-09-18T00:18:32.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012014491 2024-09-18T00:18:32.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488966+0000", 2024-09-18T00:18:32.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000159374 2024-09-18T00:18:32.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.489177+0000", 2024-09-18T00:18:32.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000211046 2024-09-18T00:18:32.624 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.624 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.624 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.624 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6360 3.6 3:6104537f:::rbd_data.112a960351d8.0000000000000004:6 [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:32.624 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.476809+0000", 2024-09-18T00:18:32.624 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.0794099729999997, 2024-09-18T00:18:32.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012624163000000001, 2024-09-18T00:18:32.624 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.624 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.624 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:32.625 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:32.625 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:32.625 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:32.625 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6360 2024-09-18T00:18:32.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.625 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476809+0000", 2024-09-18T00:18:32.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476809+0000", 2024-09-18T00:18:32.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476811+0000", 2024-09-18T00:18:32.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.237e-06 2024-09-18T00:18:32.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476816+0000", 2024-09-18T00:18:32.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2469999999999999e-06 2024-09-18T00:18:32.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476834+0000", 2024-09-18T00:18:32.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7713000000000001e-05 2024-09-18T00:18:32.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476847+0000", 2024-09-18T00:18:32.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3360000000000001e-05 2024-09-18T00:18:32.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.489242+0000", 2024-09-18T00:18:32.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012394871 2024-09-18T00:18:32.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.489322+0000", 2024-09-18T00:18:32.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0192999999999996e-05 2024-09-18T00:18:32.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.489434+0000", 2024-09-18T00:18:32.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111542 2024-09-18T00:18:32.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.629 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.629 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6361 3.6 3:6104537f:::rbd_data.112a960351d8.0000000000000004:6 [sparse-read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:32.629 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.476864+0000", 2024-09-18T00:18:32.629 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.0793557060000003, 2024-09-18T00:18:32.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012947956, 2024-09-18T00:18:32.629 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.629 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.629 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:32.630 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:32.630 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:32.630 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:32.630 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6361 2024-09-18T00:18:32.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.630 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476864+0000", 2024-09-18T00:18:32.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476864+0000", 2024-09-18T00:18:32.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476865+0000", 2024-09-18T00:18:32.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1030000000000001e-06 2024-09-18T00:18:32.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476870+0000", 2024-09-18T00:18:32.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0669999999999999e-06 2024-09-18T00:18:32.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476887+0000", 2024-09-18T00:18:32.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7235999999999999e-05 2024-09-18T00:18:32.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.476900+0000", 2024-09-18T00:18:32.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3147e-05 2024-09-18T00:18:32.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.489506+0000", 2024-09-18T00:18:32.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012605705 2024-09-18T00:18:32.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.489639+0000", 2024-09-18T00:18:32.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013270600000000001 2024-09-18T00:18:32.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.489812+0000", 2024-09-18T00:18:32.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017299200000000001 2024-09-18T00:18:32.634 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.634 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.634 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.635 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6409 3.6 3:6104537f:::rbd_data.112a960351d8.0000000000000004:6 [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:32.635 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.480652+0000", 2024-09-18T00:18:32.635 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.0755677380000002, 2024-09-18T00:18:32.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010485694, 2024-09-18T00:18:32.635 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.635 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.635 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:32.635 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:32.635 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:32.635 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:32.635 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6409 2024-09-18T00:18:32.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.636 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.480652+0000", 2024-09-18T00:18:32.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.480652+0000", 2024-09-18T00:18:32.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.480653+0000", 2024-09-18T00:18:32.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1260000000000001e-06 2024-09-18T00:18:32.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.480658+0000", 2024-09-18T00:18:32.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4639999999999997e-06 2024-09-18T00:18:32.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.480679+0000", 2024-09-18T00:18:32.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1166e-05 2024-09-18T00:18:32.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.480693+0000", 2024-09-18T00:18:32.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.385e-05 2024-09-18T00:18:32.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.490778+0000", 2024-09-18T00:18:32.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010085034999999999 2024-09-18T00:18:32.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.490960+0000", 2024-09-18T00:18:32.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018194199999999999 2024-09-18T00:18:32.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491137+0000", 2024-09-18T00:18:32.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017711100000000001 2024-09-18T00:18:32.639 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.639 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.639 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.639 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6410 3.6 3:6104537f:::rbd_data.112a960351d8.0000000000000004:6 [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:32.639 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.480826+0000", 2024-09-18T00:18:32.639 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.0753929700000002, 2024-09-18T00:18:32.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010939914, 2024-09-18T00:18:32.640 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.640 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.640 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:32.640 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:32.640 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:32.640 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:32.640 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6410 2024-09-18T00:18:32.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.640 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.480826+0000", 2024-09-18T00:18:32.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.480826+0000", 2024-09-18T00:18:32.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.480828+0000", 2024-09-18T00:18:32.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.407e-06 2024-09-18T00:18:32.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.480836+0000", 2024-09-18T00:18:32.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7999999999999999e-06 2024-09-18T00:18:32.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.480866+0000", 2024-09-18T00:18:32.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0809000000000003e-05 2024-09-18T00:18:32.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.480883+0000", 2024-09-18T00:18:32.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6867e-05 2024-09-18T00:18:32.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491254+0000", 2024-09-18T00:18:32.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010371042 2024-09-18T00:18:32.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491443+0000", 2024-09-18T00:18:32.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018882600000000001 2024-09-18T00:18:32.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491766+0000", 2024-09-18T00:18:32.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032316300000000001 2024-09-18T00:18:32.644 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.644 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.644 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.644 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6503 3.1b 3:d9387a1f:::rbd_data.112a3a39fc45.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:32.644 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:18:32.644 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.9918454099999998, 2024-09-18T00:18:32.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050510547000000003, 2024-09-18T00:18:32.644 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.644 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.644 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:32.645 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:32.645 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:32.645 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:32.645 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6503 2024-09-18T00:18:32.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.645 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:18:32.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:18:32.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564376+0000", 2024-09-18T00:18:32.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-06 2024-09-18T00:18:32.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564386+0000", 2024-09-18T00:18:32.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0312999999999999e-05 2024-09-18T00:18:32.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564408+0000", 2024-09-18T00:18:32.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2001000000000002e-05 2024-09-18T00:18:32.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564421+0000", 2024-09-18T00:18:32.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2271e-05 2024-09-18T00:18:32.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564491+0000", 2024-09-18T00:18:32.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0630000000000006e-05 2024-09-18T00:18:32.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564606+0000", 2024-09-18T00:18:32.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011449 2024-09-18T00:18:32.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:32.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.566743+0000", 2024-09-18T00:18:32.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021373830000000001 2024-09-18T00:18:32.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:32.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614738+0000", 2024-09-18T00:18:32.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047995359000000001 2024-09-18T00:18:32.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:32.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614759+0000", 2024-09-18T00:18:32.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1010999999999999e-05 2024-09-18T00:18:32.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:32.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614826+0000", 2024-09-18T00:18:32.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6654000000000003e-05 2024-09-18T00:18:32.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614884+0000", 2024-09-18T00:18:32.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8332999999999999e-05 2024-09-18T00:18:32.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.650 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.650 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6503 3.1b e42/23 ondisk, result = 0)", 2024-09-18T00:18:32.650 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:18:32.651 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.982585942, 2024-09-18T00:18:32.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041115407, 2024-09-18T00:18:32.651 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.651 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:32.651 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:18:32.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:18:32.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573635+0000", 2024-09-18T00:18:32.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5689999999999999e-06 2024-09-18T00:18:32.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573643+0000", 2024-09-18T00:18:32.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9699999999999999e-06 2024-09-18T00:18:32.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573668+0000", 2024-09-18T00:18:32.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5536000000000001e-05 2024-09-18T00:18:32.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573683+0000", 2024-09-18T00:18:32.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4025e-05 2024-09-18T00:18:32.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614717+0000", 2024-09-18T00:18:32.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041034517 2024-09-18T00:18:32.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614730+0000", 2024-09-18T00:18:32.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2945999999999999e-05 2024-09-18T00:18:32.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614749+0000", 2024-09-18T00:18:32.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8844000000000001e-05 2024-09-18T00:18:32.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.655 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.655 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6518 3.1 3:8061bf9f:::rbd_data.112a3a39fc45.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:32.656 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:18:32.656 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.905407136, 2024-09-18T00:18:32.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062303123000000002, 2024-09-18T00:18:32.656 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.656 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.656 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:32.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:32.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:32.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:32.656 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6518 2024-09-18T00:18:32.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.656 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:18:32.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:18:32.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650814+0000", 2024-09-18T00:18:32.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.629e-06 2024-09-18T00:18:32.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650823+0000", 2024-09-18T00:18:32.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1800000000000002e-06 2024-09-18T00:18:32.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650843+0000", 2024-09-18T00:18:32.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9836000000000001e-05 2024-09-18T00:18:32.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650854+0000", 2024-09-18T00:18:32.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1564e-05 2024-09-18T00:18:32.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650917+0000", 2024-09-18T00:18:32.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3076000000000004e-05 2024-09-18T00:18:32.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.651029+0000", 2024-09-18T00:18:32.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111154 2024-09-18T00:18:32.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:32.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.655531+0000", 2024-09-18T00:18:32.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045027629999999999 2024-09-18T00:18:32.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:32.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712924+0000", 2024-09-18T00:18:32.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057393006000000003 2024-09-18T00:18:32.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:32.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712988+0000", 2024-09-18T00:18:32.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000005e-05 2024-09-18T00:18:32.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:32.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713056+0000", 2024-09-18T00:18:32.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7904999999999994e-05 2024-09-18T00:18:32.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713115+0000", 2024-09-18T00:18:32.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9339999999999998e-05 2024-09-18T00:18:32.661 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.662 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.662 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.662 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6518 3.1 e42/23 ondisk, result = 0)", 2024-09-18T00:18:32.662 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:18:32.662 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.8929820529999999, 2024-09-18T00:18:32.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049887304, 2024-09-18T00:18:32.662 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:32.662 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:32.662 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:32.662 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:32.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:32.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:18:32.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:32.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:18:32.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:32.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:32.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663239+0000", 2024-09-18T00:18:32.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:18:32.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:32.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663246+0000", 2024-09-18T00:18:32.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4189999999999996e-06 2024-09-18T00:18:32.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:32.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663265+0000", 2024-09-18T00:18:32.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8323000000000001e-05 2024-09-18T00:18:32.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:32.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663277+0000", 2024-09-18T00:18:32.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2065999999999999e-05 2024-09-18T00:18:32.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:32.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712965+0000", 2024-09-18T00:18:32.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049688317000000003 2024-09-18T00:18:32.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:32.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712977+0000", 2024-09-18T00:18:32.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.206e-05 2024-09-18T00:18:32.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:32.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:32.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:32.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713125+0000", 2024-09-18T00:18:32.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014749899999999999 2024-09-18T00:18:32.665 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.666 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.666 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.666 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:32.666 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:32.666 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:32.667 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:18:32.668 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:32.662+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:32.768 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:32.763+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:32.849 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:32.849 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:32.867 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:18:32.868 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:32.863+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:32.956 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-18T00:18:32.969 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:32.966+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:33.010 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:33.010 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:33.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:18:33.070 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:33.067+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:33.102 INFO:teuthology.orchestra.run.smithi097.stdout:1791 punch from 0x46a7d07 to 0x46b016f, (0x8468 bytes) 2024-09-18T00:18:33.109 INFO:teuthology.orchestra.run.smithi097.stdout:1793 trunc from 0xc066c81 to 0xbf62340 2024-09-18T00:18:33.147 INFO:teuthology.orchestra.run.smithi097.stdout:1794 write 0x6201056 thru 0x6208b78 (0x7b23 bytes) 2024-09-18T00:18:33.166 INFO:teuthology.orchestra.run.smithi097.stdout:1795 read 0x7d228d1 thru 0x7d2649d (0x3bcd bytes) 2024-09-18T00:18:33.166 INFO:teuthology.orchestra.run.smithi097.stdout:1797 write 0x354a707 thru 0x354c175 (0x1a6f bytes) 2024-09-18T00:18:33.170 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:33.166+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:33.174 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:33.174 INFO:teuthology.orchestra.run.smithi097.stdout:1800 write 0x9d9d866 thru 0x9da526a (0x7a05 bytes) 2024-09-18T00:18:33.192 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:33.193 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:33.193 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:33.208 INFO:teuthology.orchestra.run.smithi097.stdout:1802 write 0xdefa18e thru 0xdf0294e (0x87c1 bytes) 2024-09-18T00:18:33.209 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:18:33.224 INFO:teuthology.orchestra.run.smithi097.stdout:1803 punch from 0x784d424 to 0x785173e, (0x431a bytes) 2024-09-18T00:18:33.224 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:33.224 INFO:teuthology.orchestra.run.smithi097.stdout:1806 read 0xcf6e08a thru 0xcf79f17 (0xbe8e bytes) 2024-09-18T00:18:33.224 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:33.225 INFO:teuthology.orchestra.run.smithi097.stdout:1808 writesame 0xa055b5 thru 0xa0c5d6 data_size 0x1f(0x7022 bytes) 2024-09-18T00:18:33.246 INFO:teuthology.orchestra.run.smithi097.stdout:1809 trunc from 0xdf0294f to 0xab2dc9d 2024-09-18T00:18:33.271 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:33.266+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:33.290 INFO:teuthology.orchestra.run.smithi097.stdout:1811 writesame 0x9d21746 thru 0x9d240b8 data_size 0x1b(0x2973 bytes) 2024-09-18T00:18:33.293 INFO:teuthology.orchestra.run.smithi097.stdout:1812 read 0xa4d6387 thru 0xa4e4d62 (0xe9dc bytes) 2024-09-18T00:18:33.293 INFO:teuthology.orchestra.run.smithi097.stdout:1813 read 0x9fa3d42 thru 0x9fa5cb9 (0x1f78 bytes) 2024-09-18T00:18:33.294 INFO:teuthology.orchestra.run.smithi097.stdout:1814 read 0x38fa63b thru 0x390794c (0xd312 bytes) 2024-09-18T00:18:33.294 INFO:teuthology.orchestra.run.smithi097.stdout:1815 write 0x543463f thru 0x543b0e2 (0x6aa4 bytes) 2024-09-18T00:18:33.317 INFO:teuthology.orchestra.run.smithi097.stdout:1817 writesame 0x72a1b53 thru 0x72a44d4 data_size 0x2e(0x2982 bytes) 2024-09-18T00:18:33.342 INFO:teuthology.orchestra.run.smithi097.stdout:1819 write 0xcc42aa0 thru 0xcc47d59 (0x52ba bytes) 2024-09-18T00:18:33.352 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:33.352 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:33.353 INFO:teuthology.orchestra.run.smithi097.stdout:1821 read 0x3ded0b3 thru 0x3df82db (0xb229 bytes) 2024-09-18T00:18:33.354 INFO:teuthology.orchestra.run.smithi097.stdout:1823 write 0x2d6b063 thru 0x2d78c0e (0xdbac bytes) 2024-09-18T00:18:33.368 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:18:33.372 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:33.366+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:33.375 INFO:teuthology.orchestra.run.smithi097.stdout:1824 write 0x4f927da thru 0x4fa1b71 (0xf398 bytes) 2024-09-18T00:18:33.394 INFO:teuthology.orchestra.run.smithi097.stdout:1825 write 0x91749da thru 0x9175197 (0x7be bytes) 2024-09-18T00:18:33.399 INFO:teuthology.orchestra.run.smithi097.stdout:1826 writesame 0x2a832fe thru 0x2a8e4e9 data_size 0x3b(0xb1ec bytes) 2024-09-18T00:18:33.421 INFO:teuthology.orchestra.run.smithi097.stdout:1827 trunc from 0xcc47d5a to 0x6f6dd7b 2024-09-18T00:18:33.456 INFO:teuthology.orchestra.run.smithi097.stdout:1830 read 0x1c7db3c thru 0x1c88efa (0xb3bf bytes) 2024-09-18T00:18:33.466 INFO:teuthology.orchestra.run.smithi097.stdout:1831 write 0x81be925 thru 0x81c3479 (0x4b55 bytes) 2024-09-18T00:18:33.473 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:33.466+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:33.475 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:33.475 INFO:teuthology.orchestra.run.smithi097.stdout:1833 read 0x17fe04c thru 0x1809517 (0xb4cc bytes) 2024-09-18T00:18:33.494 INFO:teuthology.orchestra.run.smithi097.stdout:1834 write 0xd45c3c1 thru 0xd468ca7 (0xc8e7 bytes) 2024-09-18T00:18:33.501 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:33.501 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:33.515 INFO:teuthology.orchestra.run.smithi097.stdout:1835 read 0x426f4e4 thru 0x427d2f9 (0xde16 bytes) 2024-09-18T00:18:33.515 INFO:teuthology.orchestra.run.smithi097.stdout:1836 read 0xafc43b0 thru 0xafc7960 (0x35b1 bytes) 2024-09-18T00:18:33.515 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:33.515 INFO:teuthology.orchestra.run.smithi097.stdout:1838 read 0xa1a5645 thru 0xa1aaf6a (0x5926 bytes) 2024-09-18T00:18:33.515 INFO:teuthology.orchestra.run.smithi097.stdout:1839 writesame 0x3634103 thru 0x36364e6 data_size 0x2(0x23e4 bytes) 2024-09-18T00:18:33.517 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:18:33.524 INFO:teuthology.orchestra.run.smithi097.stdout:1841 punch from 0x4a4ced0 to 0x4a516b3, (0x47e3 bytes) 2024-09-18T00:18:33.524 INFO:teuthology.orchestra.run.smithi097.stdout:1843 write 0x50c03f4 thru 0x50c7262 (0x6e6f bytes) 2024-09-18T00:18:33.534 INFO:teuthology.orchestra.run.smithi097.stdout:1844 write 0xa888e81 thru 0xa89698f (0xdb0f bytes) 2024-09-18T00:18:33.551 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:33.552 INFO:teuthology.orchestra.run.smithi097.stdout:1847 write 0xbde15d9 thru 0xbdeb510 (0x9f38 bytes) 2024-09-18T00:18:33.565 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:33.565 INFO:teuthology.orchestra.run.smithi097.stdout:1849 read 0x53cbb25 thru 0x53d3ab9 (0x7f95 bytes) 2024-09-18T00:18:33.565 INFO:teuthology.orchestra.run.smithi097.stdout:1850 punch from 0xc3daedb to 0xc3e4296, (0x93bb bytes) 2024-09-18T00:18:33.565 INFO:teuthology.orchestra.run.smithi097.stdout:1852 read 0xcafb403 thru 0xcb00da8 (0x59a6 bytes) 2024-09-18T00:18:33.566 INFO:teuthology.orchestra.run.smithi097.stdout:1853 flatten 2024-09-18T00:18:33.574 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:33.570+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:33.590 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:18:33.632 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:33.632 INFO:teuthology.orchestra.run.smithi097.stdout:1855 read 0xa55a64c thru 0xa568d8b (0xe740 bytes) 2024-09-18T00:18:33.632 INFO:teuthology.orchestra.run.smithi097.stdout:1856 punch from 0x3c917c5 to 0x3c9748e, (0x5cc9 bytes) 2024-09-18T00:18:33.633 INFO:teuthology.orchestra.run.smithi097.stdout:1857 read 0x9f89814 thru 0x9f90fc9 (0x77b6 bytes) 2024-09-18T00:18:33.642 INFO:teuthology.orchestra.run.smithi097.stdout:1858 write 0xdfba6a7 thru 0xdfc0f47 (0x68a1 bytes) 2024-09-18T00:18:33.653 INFO:teuthology.orchestra.run.smithi097.stdout:1859 writesame 0x29e178c thru 0x29e6b7f data_size 0x24(0x53f4 bytes) 2024-09-18T00:18:33.658 INFO:teuthology.orchestra.run.smithi097.stdout:1860 read 0xeeab49 thru 0xeee9a6 (0x3e5e bytes) 2024-09-18T00:18:33.658 INFO:teuthology.orchestra.run.smithi097.stdout:1861 writesame 0x3333576 thru 0x333d136 data_size 0x1(0x9bc1 bytes) 2024-09-18T00:18:33.668 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:33.669 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:33.675 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:33.670+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:33.688 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:18:33.700 INFO:teuthology.orchestra.run.smithi097.stdout:1862 read 0xa5ac79f thru 0xa5bb8f5 (0xf157 bytes) 2024-09-18T00:18:33.700 INFO:teuthology.orchestra.run.smithi097.stdout:1863 punch from 0xbface55 to 0xbfb4820, (0x79cb bytes) 2024-09-18T00:18:33.700 INFO:teuthology.orchestra.run.smithi097.stdout:1864 read 0x8930348 thru 0x8931829 (0x14e2 bytes) 2024-09-18T00:18:33.700 INFO:teuthology.orchestra.run.smithi097.stdout:1865 write 0x7d6edc7 thru 0x7d72fae (0x41e8 bytes) 2024-09-18T00:18:33.705 INFO:teuthology.orchestra.run.smithi097.stdout:1866 write 0xcab51e6 thru 0xcab7011 (0x1e2c bytes) 2024-09-18T00:18:33.723 INFO:teuthology.orchestra.run.smithi097.stdout:1867 trunc from 0xdfc0f48 to 0x4d0065c 2024-09-18T00:18:33.748 INFO:teuthology.orchestra.run.smithi097.stdout:1868 read 0x7b869 thru 0x855ca (0x9d62 bytes) 2024-09-18T00:18:33.750 INFO:teuthology.orchestra.run.smithi097.stdout:1869 punch from 0x6f1527 to 0x6f4c52, (0x372b bytes) 2024-09-18T00:18:33.750 INFO:teuthology.orchestra.run.smithi097.stdout:1870 read 0x21f1286 thru 0x21ff090 (0xde0b bytes) 2024-09-18T00:18:33.750 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:33.763 INFO:teuthology.orchestra.run.smithi097.stdout:1874 write 0x984b446 thru 0x98589e6 (0xd5a1 bytes) 2024-09-18T00:18:33.773 INFO:teuthology.orchestra.run.smithi097.stdout:1875 write 0x6400ba5 thru 0x640137a (0x7d6 bytes) 2024-09-18T00:18:33.776 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:33.776 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:33.770+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:33.788 INFO:teuthology.orchestra.run.smithi097.stdout:1881 trunc from 0x98589e7 to 0xa8cd5e 2024-09-18T00:18:33.813 INFO:teuthology.orchestra.run.smithi097.stdout:1883 read 0x15449c thru 0x15f814 (0xb379 bytes) 2024-09-18T00:18:33.816 INFO:teuthology.orchestra.run.smithi097.stdout:1885 trunc from 0xa8cd5e to 0x1b67112 2024-09-18T00:18:33.823 INFO:teuthology.orchestra.run.smithi097.stdout:1886 punch from 0x1365fc2 to 0x136d0d2, (0x7110 bytes) 2024-09-18T00:18:33.823 INFO:teuthology.orchestra.run.smithi097.stdout:1888 punch from 0x1507667 to 0x150a272, (0x2c0b bytes) 2024-09-18T00:18:33.823 INFO:teuthology.orchestra.run.smithi097.stdout:1890 punch from 0xf08a02 to 0xf111cc, (0x87ca bytes) 2024-09-18T00:18:33.824 INFO:teuthology.orchestra.run.smithi097.stdout:1891 punch from 0x1a61be to 0x1ad642, (0x7484 bytes) 2024-09-18T00:18:33.824 INFO:teuthology.orchestra.run.smithi097.stdout:1892 trunc from 0x1b67112 to 0x1a9307e 2024-09-18T00:18:33.829 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:33.829 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:33.843 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:18:33.854 INFO:teuthology.orchestra.run.smithi097.stdout:1893 write 0xc5eb197 thru 0xc5f4810 (0x967a bytes) 2024-09-18T00:18:33.865 INFO:teuthology.orchestra.run.smithi097.stdout:1894 write 0x5549b4e thru 0x5554366 (0xa819 bytes) 2024-09-18T00:18:33.877 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:33.874+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:33.882 INFO:teuthology.orchestra.run.smithi097.stdout:1895 read 0x489f64d thru 0x48abff3 (0xc9a7 bytes) 2024-09-18T00:18:33.882 INFO:teuthology.orchestra.run.smithi097.stdout:1896 read 0x1c57396 thru 0x1c5b9e4 (0x464f bytes) 2024-09-18T00:18:33.884 INFO:teuthology.orchestra.run.smithi097.stdout:1897 trunc from 0xc5f4811 to 0xdf60a3c 2024-09-18T00:18:33.893 INFO:teuthology.orchestra.run.smithi097.stdout:1898 punch from 0x1d1dc73 to 0x1d2b232, (0xd5bf bytes) 2024-09-18T00:18:33.893 INFO:teuthology.orchestra.run.smithi097.stdout:1899 writesame 0x11b963b thru 0x11c2ba6 data_size 0x4(0x956c bytes) 2024-09-18T00:18:33.894 INFO:teuthology.orchestra.run.smithi097.stderr:Exporting 3.8 info 3.8( v 44'54 (0'0,44'54] local-lis/les=23/24 n=5 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) 2024-09-18T00:18:33.901 INFO:teuthology.orchestra.run.smithi097.stderr:Read #3:11b3ac8b:::rbd_data.112aa7ba4578.000000000000033a:head# 2024-09-18T00:18:33.902 INFO:teuthology.orchestra.run.smithi097.stderr:Read #3:12ac789b:::rbd_data.112aa7ba4578.0000000000000130:head# 2024-09-18T00:18:33.902 INFO:teuthology.orchestra.run.smithi097.stderr:Read #3:16b24e91:::rbd_data.112aa7ba4578.0000000000000161:head# 2024-09-18T00:18:33.903 INFO:teuthology.orchestra.run.smithi097.stderr:Read #3:17544c57:::rbd_data.112aa7ba4578.0000000000000142:head# 2024-09-18T00:18:33.904 INFO:teuthology.orchestra.run.smithi097.stderr:Read #3:17e81ee6:::rbd_id.image_client.0-clone4:head# 2024-09-18T00:18:33.904 INFO:teuthology.orchestra.run.smithi097.stderr:Export successful 2024-09-18T00:18:33.905 INFO:teuthology.orchestra.run.smithi097.stdout:1901 read 0x32d33fd thru 0x32e2eca (0xface bytes) 2024-09-18T00:18:33.905 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:33.905 INFO:teuthology.orchestra.run.smithi097.stdout:1903 punch from 0x31f787a to 0x320578f, (0xdf15 bytes) 2024-09-18T00:18:33.905 INFO:teuthology.orchestra.run.smithi097.stdout:1904 read 0x974cfc7 thru 0x9752366 (0x53a0 bytes) 2024-09-18T00:18:33.905 INFO:teuthology.orchestra.run.smithi097.stdout:1905 write 0x640eae6 thru 0x641755a (0x8a75 bytes) 2024-09-18T00:18:33.916 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:33.916 INFO:teuthology.orchestra.run.smithi097.stdout:1907 read 0xa639990 thru 0xa63acd0 (0x1341 bytes) 2024-09-18T00:18:33.916 INFO:teuthology.orchestra.run.smithi097.stdout:1908 write 0x41262a3 thru 0x412abc1 (0x491f bytes) 2024-09-18T00:18:33.919 INFO:teuthology.orchestra.run.smithi097.stdout:1909 write 0x26da5ec thru 0x26da8bd (0x2d2 bytes) 2024-09-18T00:18:33.923 INFO:teuthology.orchestra.run.smithi097.stdout:1910 punch from 0x476414f to 0x476685b, (0x270c bytes) 2024-09-18T00:18:33.923 INFO:teuthology.orchestra.run.smithi097.stdout:1911 write 0xdaf9fe4 thru 0xdb01b1b (0x7b38 bytes) 2024-09-18T00:18:33.934 INFO:teuthology.orchestra.run.smithi097.stdout:1912 punch from 0x5d8dd52 to 0x5d91ca5, (0x3f53 bytes) 2024-09-18T00:18:33.934 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:33.949 INFO:teuthology.orchestra.run.smithi097.stdout:1914 trunc from 0xdf60a3c to 0x390000e 2024-09-18T00:18:33.965 INFO:teuthology.orchestra.run.smithi097.stdout:1915 read 0x11e5d73 thru 0x11e8d9f (0x302d bytes) 2024-09-18T00:18:33.965 INFO:teuthology.orchestra.run.smithi097.stdout:1916 read 0x1215aaf thru 0x121eeef (0x9441 bytes) 2024-09-18T00:18:33.979 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:33.974+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:33.982 INFO:teuthology.orchestra.run.smithi097.stdout:1917 writesame 0x8cd0513 thru 0x8cdd188 data_size 0x1(0xcc76 bytes) 2024-09-18T00:18:33.982 INFO:teuthology.orchestra.run.smithi097.stdout:1920 read 0x58da4ed thru 0x58df337 (0x4e4b bytes) 2024-09-18T00:18:33.982 INFO:teuthology.orchestra.run.smithi097.stdout:1921 read 0x40953a1 thru 0x409abfa (0x585a bytes) 2024-09-18T00:18:33.982 INFO:teuthology.orchestra.run.smithi097.stdout:1922 writesame 0x3225d4d thru 0x322db6d data_size 0x2f(0x7e21 bytes) 2024-09-18T00:18:33.990 INFO:teuthology.orchestra.run.smithi097.stdout:1923 read 0x5a80af9 thru 0x5a8804f (0x7557 bytes) 2024-09-18T00:18:33.990 INFO:teuthology.orchestra.run.smithi097.stdout:1924 read 0x522b3b4 thru 0x523364f (0x829c bytes) 2024-09-18T00:18:33.995 INFO:teuthology.orchestra.run.smithi097.stdout:1925 trunc from 0x8cdd189 to 0xb87aa43 2024-09-18T00:18:34.003 INFO:teuthology.orchestra.run.smithi097.stdout:1926 write 0xb21f510 thru 0xb2245cf (0x50c0 bytes) 2024-09-18T00:18:34.024 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:34.025 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:34.029 INFO:teuthology.orchestra.run.smithi097.stdout:1927 write 0xd612f6c thru 0xd613434 (0x4c9 bytes) 2024-09-18T00:18:34.040 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:18:34.044 INFO:teuthology.orchestra.run.smithi097.stdout:1928 trunc from 0xd613435 to 0x93b2ac8 2024-09-18T00:18:34.061 INFO:teuthology.orchestra.run.smithi097.stdout:1929 read 0x562b99c thru 0x562fdf1 (0x4456 bytes) 2024-09-18T00:18:34.061 INFO:teuthology.orchestra.run.smithi097.stdout:1930 punch from 0x33b6106 to 0x33c573f, (0xf639 bytes) 2024-09-18T00:18:34.079 INFO:teuthology.orchestra.run.smithi097.stdout:1931 trunc from 0x93b2ac8 to 0x2e9d9d1 2024-09-18T00:18:34.082 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:34.078+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:34.098 INFO:teuthology.orchestra.run.smithi097.stdout:1932 punch from 0xc09a4e to 0xc188eb, (0xee9d bytes) 2024-09-18T00:18:34.122 INFO:teuthology.orchestra.run.smithi097.stdout:1933 trunc from 0x2e9d9d1 to 0xbf6788c 2024-09-18T00:18:34.133 INFO:teuthology.orchestra.run.smithi097.stdout:1934 read 0x4f56e15 thru 0x4f59d64 (0x2f50 bytes) 2024-09-18T00:18:34.133 INFO:teuthology.orchestra.run.smithi097.stdout:1935 writesame 0x8a6d4ac thru 0x8a6f515 data_size 0x9(0x206a bytes) 2024-09-18T00:18:34.151 INFO:teuthology.orchestra.run.smithi097.stdout:1936 write 0xbf8ed94 thru 0xbf926bf (0x392c bytes) 2024-09-18T00:18:34.160 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:34.160 INFO:teuthology.orchestra.run.smithi097.stdout:1939 read 0x740efc9 thru 0x7415aa2 (0x6ada bytes) 2024-09-18T00:18:34.160 INFO:teuthology.orchestra.run.smithi097.stdout:1940 writesame 0xb7bed1d thru 0xb7c7b51 data_size 0xf(0x8e35 bytes) 2024-09-18T00:18:34.174 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:34.174 INFO:teuthology.orchestra.run.smithi097.stdout:1944 punch from 0x1d4bab9 to 0x1d536d4, (0x7c1b bytes) 2024-09-18T00:18:34.181 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:34.178+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:34.198 INFO:teuthology.orchestra.run.smithi097.stdout:1945 trunc from 0xbf926c0 to 0x23456f6 2024-09-18T00:18:34.218 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:34.218 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:34.225 INFO:teuthology.orchestra.run.smithi097.stdout:1947 read 0x1ea1961 thru 0x1ea4232 (0x28d2 bytes) 2024-09-18T00:18:34.233 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:18:34.249 INFO:teuthology.orchestra.run.smithi097.stdout:1948 writesame 0x779da98 thru 0x77a4330 data_size 0x1(0x6899 bytes) 2024-09-18T00:18:34.249 INFO:teuthology.orchestra.run.smithi097.stdout:1950 write 0x225d83e thru 0x2263821 (0x5fe4 bytes) 2024-09-18T00:18:34.260 INFO:teuthology.orchestra.run.smithi097.stdout:1952 write 0x60b66d5 thru 0x60b9274 (0x2ba0 bytes) 2024-09-18T00:18:34.278 INFO:teuthology.orchestra.run.smithi097.stdout:1953 writesame 0x917b98c thru 0x917c5fe data_size 0x1(0xc73 bytes) 2024-09-18T00:18:34.279 INFO:teuthology.orchestra.run.smithi097.stdout:1954 punch from 0x5d82e67 to 0x5d88452, (0x55eb bytes) 2024-09-18T00:18:34.279 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:34.279 INFO:teuthology.orchestra.run.smithi097.stdout:1956 punch from 0x4c45730 to 0x4c4edb3, (0x9683 bytes) 2024-09-18T00:18:34.279 INFO:teuthology.orchestra.run.smithi097.stdout:1957 punch from 0x647b077 to 0x6487bf2, (0xcb7b bytes) 2024-09-18T00:18:34.279 INFO:teuthology.orchestra.run.smithi097.stdout:1959 write 0x3f1f04f thru 0x3f22553 (0x3505 bytes) 2024-09-18T00:18:34.281 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:34.274+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:34.286 INFO:teuthology.orchestra.run.smithi097.stdout:1962 punch from 0x6f4a317 to 0x6f4a7e5, (0x4ce bytes) 2024-09-18T00:18:34.297 INFO:teuthology.orchestra.run.smithi097.stdout:1963 write 0xa945610 thru 0xa947f23 (0x2914 bytes) 2024-09-18T00:18:34.303 INFO:teuthology.orchestra.run.smithi097.stdout:1966 read 0x3192526 thru 0x3193b31 (0x160c bytes) 2024-09-18T00:18:34.303 INFO:teuthology.orchestra.run.smithi097.stdout:1967 punch from 0x643ae97 to 0x6448507, (0xd670 bytes) 2024-09-18T00:18:34.303 INFO:teuthology.orchestra.run.smithi097.stdout:1969 punch from 0xca53f to 0xd82c0, (0xdd81 bytes) 2024-09-18T00:18:34.303 INFO:teuthology.orchestra.run.smithi097.stdout:1971 write 0x6d3149b thru 0x6d33d93 (0x28f9 bytes) 2024-09-18T00:18:34.307 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:34.308 INFO:teuthology.orchestra.run.smithi097.stdout:1975 read 0x26b593d thru 0x26bb679 (0x5d3d bytes) 2024-09-18T00:18:34.325 INFO:teuthology.orchestra.run.smithi097.stdout:1976 trunc from 0xa947f24 to 0x43ed607 2024-09-18T00:18:34.350 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:18:34.354 INFO:teuthology.orchestra.run.smithi097.stdout:1978 writesame 0x5c671f3 thru 0x5c68555 data_size 0x1(0x1363 bytes) 2024-09-18T00:18:34.354 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:34.354 INFO:teuthology.orchestra.run.smithi097.stdout:1980 read 0x17860ca thru 0x178cf0e (0x6e45 bytes) 2024-09-18T00:18:34.354 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:34.354 INFO:teuthology.orchestra.run.smithi097.stdout:1982 punch from 0x51a48de to 0x51a7697, (0x2db9 bytes) 2024-09-18T00:18:34.365 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:34.365 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:34.365 INFO:teuthology.orchestra.run.smithi097.stdout:1983 trunc from 0x5c68556 to 0xaa9ad02 2024-09-18T00:18:34.368 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:34.369 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:34.380 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:18:34.382 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:34.378+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:34.387 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:34.388 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:34.388 INFO:teuthology.orchestra.run.smithi097.stdout:1984 write 0xce47b4d thru 0xce4e8f9 (0x6dad bytes) 2024-09-18T00:18:34.410 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:34.410 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:34.418 INFO:teuthology.orchestra.run.smithi097.stdout:1985 writesame 0xe219a1a thru 0xe22253a data_size 0x1(0x8b21 bytes) 2024-09-18T00:18:34.419 INFO:teuthology.orchestra.run.smithi097.stdout:1988 read 0xd9c398a thru 0xd9c4d9b (0x1412 bytes) 2024-09-18T00:18:34.428 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:18:34.429 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:18:34.430 INFO:teuthology.orchestra.run.smithi097.stdout:1989 trunc from 0xe22253b to 0x8c10c2c 2024-09-18T00:18:34.448 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:34.448 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:34.461 INFO:teuthology.orchestra.run.smithi097.stdout:1990 trunc from 0x8c10c2c to 0x21e6ddb 2024-09-18T00:18:34.472 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:34.472 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:34.475 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:18:34.476 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:18:34.477 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --force --op remove --pgid 3.8 2024-09-18T00:18:34.482 INFO:teuthology.orchestra.run.smithi097.stdout:1991 read 0xd68b5a thru 0xd6e2ad (0x5754 bytes) 2024-09-18T00:18:34.482 INFO:teuthology.orchestra.run.smithi097.stdout:1993 read 0x21d6a0e thru 0x21dbf8a (0x557d bytes) 2024-09-18T00:18:34.482 INFO:teuthology.orchestra.run.smithi097.stdout:1994 read 0xb823dd thru 0xb8938b (0x6faf bytes) 2024-09-18T00:18:34.483 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:34.478+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:34.483 INFO:teuthology.orchestra.run.smithi097.stdout:1995 read 0x1902d82 thru 0x1906131 (0x33b0 bytes) 2024-09-18T00:18:34.495 INFO:teuthology.orchestra.run.smithi097.stdout:1997 trunc from 0x21e6ddb to 0x8f71d10 2024-09-18T00:18:34.505 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:34.505 INFO:teuthology.orchestra.run.smithi097.stdout:2000 read 0xc4fff6 thru 0xc55ed3 (0x5ede bytes) 2024-09-18T00:18:34.519 INFO:teuthology.orchestra.run.smithi097.stdout:2001 trunc from 0x8f71d10 to 0x11071c7 2024-09-18T00:18:34.555 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:34.556 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:34.556 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:34.556 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:34.556 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:34.558 INFO:teuthology.orchestra.run.smithi097.stdout:2004 write 0x25bb558 thru 0x25caec7 (0xf970 bytes) 2024-09-18T00:18:34.572 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:18:34.578 INFO:teuthology.orchestra.run.smithi097.stdout:2005 punch from 0x6c443e to 0x6c6bb5, (0x2777 bytes) 2024-09-18T00:18:34.578 INFO:teuthology.orchestra.run.smithi097.stdout:2006 read 0x1f8b5dc thru 0x1f9a03d (0xea62 bytes) 2024-09-18T00:18:34.591 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is unset 2024-09-18T00:18:34.607 INFO:teuthology.orchestra.run.smithi097.stdout:2008 write 0x9927e3b thru 0x9932ac1 (0xac87 bytes) 2024-09-18T00:18:34.614 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:34.610+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:34.640 INFO:teuthology.orchestra.run.smithi097.stdout:2009 trunc from 0x9932ac2 to 0x18a2b3 2024-09-18T00:18:34.684 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:34.678+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:34.694 INFO:teuthology.orchestra.run.smithi097.stdout:2012 write 0x8ba59de thru 0x8bb5421 (0xfa44 bytes) 2024-09-18T00:18:34.710 INFO:teuthology.orchestra.run.smithi097.stdout:2013 read 0x353774 thru 0x3621e5 (0xea72 bytes) 2024-09-18T00:18:34.719 INFO:teuthology.orchestra.run.smithi097.stdout:2014 trunc from 0x8bb5422 to 0xbcb35d3 2024-09-18T00:18:34.747 INFO:teuthology.orchestra.run.smithi097.stdout:2015 write 0xdb2d5b1 thru 0xdb39fbe (0xca0e bytes) 2024-09-18T00:18:34.751 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4263.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-18T00:18:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:55.470436+0000", 2024-09-18T00:18:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 39.276658759999997, 2024-09-18T00:18:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01376928, 2024-09-18T00:18:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4263", 2024-09-18T00:18:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1025497029", 2024-09-18T00:18:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-18T00:18:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.470436+0000", 2024-09-18T00:18:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.470436+0000", 2024-09-18T00:18:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.470442+0000", 2024-09-18T00:18:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7559999999999996e-06 2024-09-18T00:18:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.470453+0000", 2024-09-18T00:18:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.078e-05 2024-09-18T00:18:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.470474+0000", 2024-09-18T00:18:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1126999999999999e-05 2024-09-18T00:18:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.470491+0000", 2024-09-18T00:18:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7465999999999998e-05 2024-09-18T00:18:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.470674+0000", 2024-09-18T00:18:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000182719 2024-09-18T00:18:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:18:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.470693+0000", 2024-09-18T00:18:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9332999999999999e-05 2024-09-18T00:18:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.474770+0000", 2024-09-18T00:18:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040765649999999999 2024-09-18T00:18:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:18:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.474791+0000", 2024-09-18T00:18:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1107e-05 2024-09-18T00:18:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.478172+0000", 2024-09-18T00:18:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0033807139999999999 2024-09-18T00:18:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.478520+0000", 2024-09-18T00:18:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034797700000000002 2024-09-18T00:18:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 2", 2024-09-18T00:18:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.479136+0000", 2024-09-18T00:18:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061632899999999999 2024-09-18T00:18:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.482703+0000", 2024-09-18T00:18:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035675490000000002 2024-09-18T00:18:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.484057+0000", 2024-09-18T00:18:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0013534490000000001 2024-09-18T00:18:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.484144+0000", 2024-09-18T00:18:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6676999999999997e-05 2024-09-18T00:18:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:55.484205+0000", 2024-09-18T00:18:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1731999999999998e-05 2024-09-18T00:18:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4263.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-18T00:18:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:17:56.475935+0000", 2024-09-18T00:18:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.271159664999999, 2024-09-18T00:18:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0088589089999999999, 2024-09-18T00:18:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4263", 2024-09-18T00:18:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1025497029", 2024-09-18T00:18:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5 2024-09-18T00:18:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.475935+0000", 2024-09-18T00:18:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.475935+0000", 2024-09-18T00:18:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.475938+0000", 2024-09-18T00:18:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2899999999999998e-06 2024-09-18T00:18:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.475953+0000", 2024-09-18T00:18:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4267000000000001e-05 2024-09-18T00:18:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.475989+0000", 2024-09-18T00:18:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999997e-05 2024-09-18T00:18:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.476014+0000", 2024-09-18T00:18:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5052999999999999e-05 2024-09-18T00:18:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.476167+0000", 2024-09-18T00:18:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000153602 2024-09-18T00:18:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:18:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.476178+0000", 2024-09-18T00:18:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0560000000000001e-05 2024-09-18T00:18:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.481752+0000", 2024-09-18T00:18:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055741059999999997 2024-09-18T00:18:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.481969+0000", 2024-09-18T00:18:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000216739 2024-09-18T00:18:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-18T00:18:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.482181+0000", 2024-09-18T00:18:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021255199999999999 2024-09-18T00:18:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.483689+0000", 2024-09-18T00:18:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0015074509999999999 2024-09-18T00:18:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.484522+0000", 2024-09-18T00:18:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083324099999999997 2024-09-18T00:18:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.484678+0000", 2024-09-18T00:18:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015571500000000001 2024-09-18T00:18:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:17:56.484794+0000", 2024-09-18T00:18:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116273 2024-09-18T00:18:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:525 3.1d 3:bd3b4404:::rbd_data.112ae20746f5.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:11.792745+0000", 2024-09-18T00:18:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.954349761, 2024-09-18T00:18:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011496734999999999, 2024-09-18T00:18:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 525 2024-09-18T00:18:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:11.792745+0000", 2024-09-18T00:18:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:11.792745+0000", 2024-09-18T00:18:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:11.792747+0000", 2024-09-18T00:18:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8470000000000001e-06 2024-09-18T00:18:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:11.792757+0000", 2024-09-18T00:18:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0316000000000001e-05 2024-09-18T00:18:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:11.792786+0000", 2024-09-18T00:18:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9193e-05 2024-09-18T00:18:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:11.792804+0000", 2024-09-18T00:18:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7893999999999998e-05 2024-09-18T00:18:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:11.792900+0000", 2024-09-18T00:18:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5499000000000002e-05 2024-09-18T00:18:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:11.793151+0000", 2024-09-18T00:18:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025106800000000002 2024-09-18T00:18:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:11.794873+0000", 2024-09-18T00:18:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001722653 2024-09-18T00:18:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:11.803901+0000", 2024-09-18T00:18:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0090272540000000002 2024-09-18T00:18:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:11.803957+0000", 2024-09-18T00:18:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6132999999999999e-05 2024-09-18T00:18:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:11.804115+0000", 2024-09-18T00:18:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000157746 2024-09-18T00:18:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:11.804242+0000", 2024-09-18T00:18:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012713199999999999 2024-09-18T00:18:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:752 3.1c 3:3b5e52da:::rbd_data.112ae20746f5.0000000000000156:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:12.417363+0000", 2024-09-18T00:18:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.329732108000002, 2024-09-18T00:18:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014857838999999999, 2024-09-18T00:18:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 752 2024-09-18T00:18:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.417363+0000", 2024-09-18T00:18:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.417363+0000", 2024-09-18T00:18:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.417365+0000", 2024-09-18T00:18:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.99e-06 2024-09-18T00:18:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.417375+0000", 2024-09-18T00:18:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0597e-05 2024-09-18T00:18:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.417405+0000", 2024-09-18T00:18:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9935999999999999e-05 2024-09-18T00:18:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.417423+0000", 2024-09-18T00:18:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7897000000000002e-05 2024-09-18T00:18:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.417525+0000", 2024-09-18T00:18:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101869 2024-09-18T00:18:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.417738+0000", 2024-09-18T00:18:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021268799999999999 2024-09-18T00:18:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.418498+0000", 2024-09-18T00:18:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00076028499999999998 2024-09-18T00:18:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.431589+0000", 2024-09-18T00:18:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.013090689000000001 2024-09-18T00:18:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.432053+0000", 2024-09-18T00:18:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00046407600000000001 2024-09-18T00:18:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.432144+0000", 2024-09-18T00:18:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1252e-05 2024-09-18T00:18:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.432220+0000", 2024-09-18T00:18:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6559999999999999e-05 2024-09-18T00:18:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:752 3.1c e30/23 ondisk, result = 0)", 2024-09-18T00:18:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:12.421402+0000", 2024-09-18T00:18:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.325693162, 2024-09-18T00:18:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010205384, 2024-09-18T00:18:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.421402+0000", 2024-09-18T00:18:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.421402+0000", 2024-09-18T00:18:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.421403+0000", 2024-09-18T00:18:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9259999999999999e-06 2024-09-18T00:18:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.421412+0000", 2024-09-18T00:18:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7569999999999993e-06 2024-09-18T00:18:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.421440+0000", 2024-09-18T00:18:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7866000000000001e-05 2024-09-18T00:18:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.421453+0000", 2024-09-18T00:18:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2977e-05 2024-09-18T00:18:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.431560+0000", 2024-09-18T00:18:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010106412 2024-09-18T00:18:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.431578+0000", 2024-09-18T00:18:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8153000000000001e-05 2024-09-18T00:18:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.431607+0000", 2024-09-18T00:18:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9292999999999999e-05 2024-09-18T00:18:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:767 3.d 3:b6d4c6e5:::rbd_object_map.112ae20746f5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:12.468531+0000", 2024-09-18T00:18:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.278563726000002, 2024-09-18T00:18:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0093249839999999997, 2024-09-18T00:18:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 767 2024-09-18T00:18:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.468531+0000", 2024-09-18T00:18:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.468531+0000", 2024-09-18T00:18:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.468532+0000", 2024-09-18T00:18:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4600000000000003e-07 2024-09-18T00:18:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.468537+0000", 2024-09-18T00:18:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0239999999999997e-06 2024-09-18T00:18:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.468548+0000", 2024-09-18T00:18:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1343e-05 2024-09-18T00:18:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.468556+0000", 2024-09-18T00:18:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8869999999999992e-06 2024-09-18T00:18:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.472043+0000", 2024-09-18T00:18:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034867359999999998 2024-09-18T00:18:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.472309+0000", 2024-09-18T00:18:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026602800000000002 2024-09-18T00:18:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-18T00:18:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.473467+0000", 2024-09-18T00:18:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001158554 2024-09-18T00:18:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.477145+0000", 2024-09-18T00:18:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036778520000000001 2024-09-18T00:18:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.477659+0000", 2024-09-18T00:18:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00051422600000000001 2024-09-18T00:18:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.477749+0000", 2024-09-18T00:18:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9949000000000002e-05 2024-09-18T00:18:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.477856+0000", 2024-09-18T00:18:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106539 2024-09-18T00:18:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:819 3.d 3:b01084f4:::rbd_data.112ae20746f5.0000000000000071:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:12.574780+0000", 2024-09-18T00:18:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.172314929999999, 2024-09-18T00:18:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.157737555, 2024-09-18T00:18:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 819 2024-09-18T00:18:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.574780+0000", 2024-09-18T00:18:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.574780+0000", 2024-09-18T00:18:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.574782+0000", 2024-09-18T00:18:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9800000000000001e-06 2024-09-18T00:18:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.574792+0000", 2024-09-18T00:18:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0217000000000001e-05 2024-09-18T00:18:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.574816+0000", 2024-09-18T00:18:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4239999999999998e-05 2024-09-18T00:18:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.574828+0000", 2024-09-18T00:18:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2017e-05 2024-09-18T00:18:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.574948+0000", 2024-09-18T00:18:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119809 2024-09-18T00:18:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.575188+0000", 2024-09-18T00:18:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000239838 2024-09-18T00:18:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-18T00:18:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.584100+0000", 2024-09-18T00:18:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089120840000000007 2024-09-18T00:18:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.732437+0000", 2024-09-18T00:18:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14833724200000001 2024-09-18T00:18:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.732449+0000", 2024-09-18T00:18:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1741e-05 2024-09-18T00:18:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.732483+0000", 2024-09-18T00:18:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3810000000000003e-05 2024-09-18T00:18:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.732517+0000", 2024-09-18T00:18:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4576999999999998e-05 2024-09-18T00:18:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:819 3.d e30/27 ondisk, result = 0)", 2024-09-18T00:18:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:12.615330+0000", 2024-09-18T00:18:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.131764959000002, 2024-09-18T00:18:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.117112726, 2024-09-18T00:18:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.615330+0000", 2024-09-18T00:18:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.615330+0000", 2024-09-18T00:18:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.615332+0000", 2024-09-18T00:18:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:18:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.615340+0000", 2024-09-18T00:18:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3000000000000002e-06 2024-09-18T00:18:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.615370+0000", 2024-09-18T00:18:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0323000000000001e-05 2024-09-18T00:18:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.615397+0000", 2024-09-18T00:18:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6990000000000001e-05 2024-09-18T00:18:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.732425+0000", 2024-09-18T00:18:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.117028048 2024-09-18T00:18:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.732433+0000", 2024-09-18T00:18:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3069999999999997e-06 2024-09-18T00:18:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.732442+0000", 2024-09-18T00:18:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.978e-06 2024-09-18T00:18:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2255 3.1d 3:bea6c6e4:::rbd_object_map.112aa7ba4578:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.394808+0000", 2024-09-18T00:18:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.352286698, 2024-09-18T00:18:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0090183969999999992, 2024-09-18T00:18:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2255 2024-09-18T00:18:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.394808+0000", 2024-09-18T00:18:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.394808+0000", 2024-09-18T00:18:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.394812+0000", 2024-09-18T00:18:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.507e-06 2024-09-18T00:18:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.394828+0000", 2024-09-18T00:18:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6449000000000001e-05 2024-09-18T00:18:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.394874+0000", 2024-09-18T00:18:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5553999999999998e-05 2024-09-18T00:18:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.394899+0000", 2024-09-18T00:18:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5116000000000001e-05 2024-09-18T00:18:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.395127+0000", 2024-09-18T00:18:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022866499999999999 2024-09-18T00:18:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:18:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.395138+0000", 2024-09-18T00:18:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.027e-05 2024-09-18T00:18:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.400023+0000", 2024-09-18T00:18:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048849419999999998 2024-09-18T00:18:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.400194+0000", 2024-09-18T00:18:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000171655 2024-09-18T00:18:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.400713+0000", 2024-09-18T00:18:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00051857300000000004 2024-09-18T00:18:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.402786+0000", 2024-09-18T00:18:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020729300000000002 2024-09-18T00:18:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.403700+0000", 2024-09-18T00:18:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00091452500000000002 2024-09-18T00:18:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.403775+0000", 2024-09-18T00:18:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4668999999999999e-05 2024-09-18T00:18:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.403826+0000", 2024-09-18T00:18:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1542000000000002e-05 2024-09-18T00:18:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2256 3.1d 3:bea6c6e4:::rbd_object_map.112aa7ba4578:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.394956+0000", 2024-09-18T00:18:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.352138799, 2024-09-18T00:18:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0095355979999999993, 2024-09-18T00:18:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2256 2024-09-18T00:18:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.394956+0000", 2024-09-18T00:18:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.394956+0000", 2024-09-18T00:18:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.394958+0000", 2024-09-18T00:18:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.623e-06 2024-09-18T00:18:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.394968+0000", 2024-09-18T00:18:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0467e-05 2024-09-18T00:18:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.394991+0000", 2024-09-18T00:18:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3402999999999999e-05 2024-09-18T00:18:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.395008+0000", 2024-09-18T00:18:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6322999999999999e-05 2024-09-18T00:18:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.395248+0000", 2024-09-18T00:18:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024016799999999999 2024-09-18T00:18:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:18:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.395262+0000", 2024-09-18T00:18:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4097000000000001e-05 2024-09-18T00:18:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.401457+0000", 2024-09-18T00:18:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061947909999999998 2024-09-18T00:18:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.401572+0000", 2024-09-18T00:18:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000115479 2024-09-18T00:18:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.401982+0000", 2024-09-18T00:18:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00041021699999999999 2024-09-18T00:18:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.403318+0000", 2024-09-18T00:18:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0013355540000000001 2024-09-18T00:18:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.404390+0000", 2024-09-18T00:18:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010720720000000001 2024-09-18T00:18:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.404449+0000", 2024-09-18T00:18:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8897000000000002e-05 2024-09-18T00:18:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.404492+0000", 2024-09-18T00:18:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2506999999999998e-05 2024-09-18T00:18:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2375 3.1d 3:bea6c6e4:::rbd_object_map.112aa7ba4578:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.652515+0000", 2024-09-18T00:18:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.094580115999999, 2024-09-18T00:18:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0082362119999999997, 2024-09-18T00:18:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2375 2024-09-18T00:18:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652515+0000", 2024-09-18T00:18:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652515+0000", 2024-09-18T00:18:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652523+0000", 2024-09-18T00:18:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3930000000000008e-06 2024-09-18T00:18:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652533+0000", 2024-09-18T00:18:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0023e-05 2024-09-18T00:18:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652558+0000", 2024-09-18T00:18:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4476000000000001e-05 2024-09-18T00:18:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652576+0000", 2024-09-18T00:18:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8130000000000001e-05 2024-09-18T00:18:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.654220+0000", 2024-09-18T00:18:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016447969999999999 2024-09-18T00:18:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.654432+0000", 2024-09-18T00:18:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021130199999999999 2024-09-18T00:18:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.655106+0000", 2024-09-18T00:18:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00067395200000000001 2024-09-18T00:18:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.660481+0000", 2024-09-18T00:18:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053757309999999999 2024-09-18T00:18:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.660553+0000", 2024-09-18T00:18:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1125999999999996e-05 2024-09-18T00:18:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.660635+0000", 2024-09-18T00:18:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2360000000000004e-05 2024-09-18T00:18:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.660751+0000", 2024-09-18T00:18:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011592200000000001 2024-09-18T00:18:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2376 3.1d 3:bea6c6e4:::rbd_object_map.112aa7ba4578:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.652606+0000", 2024-09-18T00:18:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.094488821000001, 2024-09-18T00:18:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0084118849999999992, 2024-09-18T00:18:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2376 2024-09-18T00:18:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652606+0000", 2024-09-18T00:18:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652606+0000", 2024-09-18T00:18:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652608+0000", 2024-09-18T00:18:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6169999999999999e-06 2024-09-18T00:18:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652616+0000", 2024-09-18T00:18:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4670000000000004e-06 2024-09-18T00:18:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652636+0000", 2024-09-18T00:18:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9966000000000002e-05 2024-09-18T00:18:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652651+0000", 2024-09-18T00:18:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4847e-05 2024-09-18T00:18:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.655967+0000", 2024-09-18T00:18:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003316481 2024-09-18T00:18:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.656099+0000", 2024-09-18T00:18:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013155900000000001 2024-09-18T00:18:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.656927+0000", 2024-09-18T00:18:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00082777600000000001 2024-09-18T00:18:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.660858+0000", 2024-09-18T00:18:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0039318399999999998 2024-09-18T00:18:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.660885+0000", 2024-09-18T00:18:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6259999999999999e-05 2024-09-18T00:18:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.660957+0000", 2024-09-18T00:18:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2201999999999998e-05 2024-09-18T00:18:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.661018+0000", 2024-09-18T00:18:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0869999999999998e-05 2024-09-18T00:18:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2377 3.1d 3:bea6c6e4:::rbd_object_map.112aa7ba4578:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.652672+0000", 2024-09-18T00:18:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.094422998000001, 2024-09-18T00:18:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010395176000000001, 2024-09-18T00:18:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2377 2024-09-18T00:18:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652672+0000", 2024-09-18T00:18:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652672+0000", 2024-09-18T00:18:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652673+0000", 2024-09-18T00:18:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.277e-06 2024-09-18T00:18:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652687+0000", 2024-09-18T00:18:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3906e-05 2024-09-18T00:18:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652706+0000", 2024-09-18T00:18:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9207000000000001e-05 2024-09-18T00:18:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652720+0000", 2024-09-18T00:18:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4209999999999999e-05 2024-09-18T00:18:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.658203+0000", 2024-09-18T00:18:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005482364 2024-09-18T00:18:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.658475+0000", 2024-09-18T00:18:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027264100000000001 2024-09-18T00:18:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.659201+0000", 2024-09-18T00:18:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00072537100000000005 2024-09-18T00:18:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.662860+0000", 2024-09-18T00:18:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036590849999999999 2024-09-18T00:18:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.662921+0000", 2024-09-18T00:18:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1666000000000002e-05 2024-09-18T00:18:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.663004+0000", 2024-09-18T00:18:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2489000000000003e-05 2024-09-18T00:18:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.663067+0000", 2024-09-18T00:18:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2959999999999994e-05 2024-09-18T00:18:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2379 3.1d 3:bea6c6e4:::rbd_object_map.112aa7ba4578:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.652818+0000", 2024-09-18T00:18:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.094277203000001, 2024-09-18T00:18:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011884924999999999, 2024-09-18T00:18:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2379 2024-09-18T00:18:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652818+0000", 2024-09-18T00:18:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652818+0000", 2024-09-18T00:18:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652819+0000", 2024-09-18T00:18:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3340000000000001e-06 2024-09-18T00:18:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652827+0000", 2024-09-18T00:18:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6860000000000006e-06 2024-09-18T00:18:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652846+0000", 2024-09-18T00:18:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9063999999999999e-05 2024-09-18T00:18:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.652860+0000", 2024-09-18T00:18:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4715999999999999e-05 2024-09-18T00:18:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.661080+0000", 2024-09-18T00:18:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0082201229999999993 2024-09-18T00:18:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.661213+0000", 2024-09-18T00:18:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013233900000000001 2024-09-18T00:18:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.661958+0000", 2024-09-18T00:18:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00074481700000000005 2024-09-18T00:18:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.663877+0000", 2024-09-18T00:18:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019196759999999999 2024-09-18T00:18:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.664550+0000", 2024-09-18T00:18:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000672402 2024-09-18T00:18:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.664638+0000", 2024-09-18T00:18:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8691999999999997e-05 2024-09-18T00:18:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.664702+0000", 2024-09-18T00:18:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4076000000000001e-05 2024-09-18T00:18:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2492 3.1d 3:bea6c6e4:::rbd_object_map.112aa7ba4578:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.003109+0000", 2024-09-18T00:18:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.743985733000001, 2024-09-18T00:18:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0084639120000000005, 2024-09-18T00:18:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2492 2024-09-18T00:18:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.003109+0000", 2024-09-18T00:18:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.003109+0000", 2024-09-18T00:18:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.003111+0000", 2024-09-18T00:18:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1730000000000002e-06 2024-09-18T00:18:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.003123+0000", 2024-09-18T00:18:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.155e-05 2024-09-18T00:18:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.003149+0000", 2024-09-18T00:18:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6276999999999999e-05 2024-09-18T00:18:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.003168+0000", 2024-09-18T00:18:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.91e-05 2024-09-18T00:18:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.006818+0000", 2024-09-18T00:18:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003649902 2024-09-18T00:18:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.007043+0000", 2024-09-18T00:18:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022524399999999999 2024-09-18T00:18:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.007761+0000", 2024-09-18T00:18:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00071739500000000001 2024-09-18T00:18:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.011247+0000", 2024-09-18T00:18:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034862869999999998 2024-09-18T00:18:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.011344+0000", 2024-09-18T00:18:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7141999999999999e-05 2024-09-18T00:18:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.011501+0000", 2024-09-18T00:18:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015708900000000001 2024-09-18T00:18:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.011573+0000", 2024-09-18T00:18:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1753000000000002e-05 2024-09-18T00:18:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2493 3.1d 3:bea6c6e4:::rbd_object_map.112aa7ba4578:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.003412+0000", 2024-09-18T00:18:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.743682272000001, 2024-09-18T00:18:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0096729859999999997, 2024-09-18T00:18:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2493 2024-09-18T00:18:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.003412+0000", 2024-09-18T00:18:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.003412+0000", 2024-09-18T00:18:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.003421+0000", 2024-09-18T00:18:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.14e-06 2024-09-18T00:18:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.003433+0000", 2024-09-18T00:18:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2113000000000001e-05 2024-09-18T00:18:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.003478+0000", 2024-09-18T00:18:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5590000000000002e-05 2024-09-18T00:18:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.003494+0000", 2024-09-18T00:18:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5217e-05 2024-09-18T00:18:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.009545+0000", 2024-09-18T00:18:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060510759999999999 2024-09-18T00:18:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.009728+0000", 2024-09-18T00:18:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000183456 2024-09-18T00:18:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.010331+0000", 2024-09-18T00:18:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00060318800000000005 2024-09-18T00:18:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.011973+0000", 2024-09-18T00:18:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016416880000000001 2024-09-18T00:18:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.012949+0000", 2024-09-18T00:18:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00097649599999999998 2024-09-18T00:18:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.013028+0000", 2024-09-18T00:18:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8189000000000006e-05 2024-09-18T00:18:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.013085+0000", 2024-09-18T00:18:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7833e-05 2024-09-18T00:18:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4394.0:5802 3.17 e41/23 3:eb061e39:::rbd_data.112a3a39fc45.000000000000000f:head v 41'42, mlcod=39'40)", 2024-09-18T00:18:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.387779+0000", 2024-09-18T00:18:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3593154209999998, 2024-09-18T00:18:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.013746476000000001, 2024-09-18T00:18:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.387779+0000", 2024-09-18T00:18:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.387779+0000", 2024-09-18T00:18:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.387780+0000", 2024-09-18T00:18:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0100000000000005e-07 2024-09-18T00:18:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.388975+0000", 2024-09-18T00:18:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011955239999999999 2024-09-18T00:18:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.388995+0000", 2024-09-18T00:18:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9581000000000001e-05 2024-09-18T00:18:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.389005+0000", 2024-09-18T00:18:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0334e-05 2024-09-18T00:18:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.389094+0000", 2024-09-18T00:18:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8865999999999999e-05 2024-09-18T00:18:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.389119+0000", 2024-09-18T00:18:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4978e-05 2024-09-18T00:18:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.401426+0000", 2024-09-18T00:18:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012306923000000001 2024-09-18T00:18:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.401526+0000", 2024-09-18T00:18:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9769e-05 2024-09-18T00:18:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6466 3.16 3:694a37fd:::rbd_data.112a3a39fc45.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2752512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.519234+0000", 2024-09-18T00:18:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.2278608309999992, 2024-09-18T00:18:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010134663, 2024-09-18T00:18:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6466 2024-09-18T00:18:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.519234+0000", 2024-09-18T00:18:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.519234+0000", 2024-09-18T00:18:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.519237+0000", 2024-09-18T00:18:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7e-06 2024-09-18T00:18:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.520242+0000", 2024-09-18T00:18:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001005553 2024-09-18T00:18:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.520264+0000", 2024-09-18T00:18:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1424999999999999e-05 2024-09-18T00:18:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.520275+0000", 2024-09-18T00:18:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1484999999999999e-05 2024-09-18T00:18:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.520366+0000", 2024-09-18T00:18:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0735000000000005e-05 2024-09-18T00:18:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.520503+0000", 2024-09-18T00:18:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000137501 2024-09-18T00:18:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.520694+0000", 2024-09-18T00:18:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019088199999999999 2024-09-18T00:18:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.528299+0000", 2024-09-18T00:18:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0076044959999999997 2024-09-18T00:18:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.529285+0000", 2024-09-18T00:18:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00098625300000000009 2024-09-18T00:18:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.529333+0000", 2024-09-18T00:18:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7979999999999998e-05 2024-09-18T00:18:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.529369+0000", 2024-09-18T00:18:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5652999999999999e-05 2024-09-18T00:18:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6516 3.16 3:694a37fd:::rbd_data.112a3a39fc45.0000000000000013:head [stat,write 1441792~46130] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.637657+0000", 2024-09-18T00:18:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.1094373280000003, 2024-09-18T00:18:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0083751350000000006, 2024-09-18T00:18:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:34.861 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6516 2024-09-18T00:18:34.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.861 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.637657+0000", 2024-09-18T00:18:34.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.637657+0000", 2024-09-18T00:18:34.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.637661+0000", 2024-09-18T00:18:34.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.213e-06 2024-09-18T00:18:34.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.637724+0000", 2024-09-18T00:18:34.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2918999999999998e-05 2024-09-18T00:18:34.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.637768+0000", 2024-09-18T00:18:34.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4477000000000001e-05 2024-09-18T00:18:34.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.637792+0000", 2024-09-18T00:18:34.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3683e-05 2024-09-18T00:18:34.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.637951+0000", 2024-09-18T00:18:34.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015946499999999999 2024-09-18T00:18:34.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.638186+0000", 2024-09-18T00:18:34.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000234825 2024-09-18T00:18:34.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:34.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.638611+0000", 2024-09-18T00:18:34.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00042497999999999999 2024-09-18T00:18:34.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.645537+0000", 2024-09-18T00:18:34.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069264540000000003 2024-09-18T00:18:34.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.645868+0000", 2024-09-18T00:18:34.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033033999999999998 2024-09-18T00:18:34.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.645961+0000", 2024-09-18T00:18:34.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2800000000000006e-05 2024-09-18T00:18:34.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.646033+0000", 2024-09-18T00:18:34.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1978999999999996e-05 2024-09-18T00:18:34.865 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.865 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.865 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.866 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6631 3.1d 3:bd9647a1:::rbd_data.112a3a39fc45.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 227425~24616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:34.866 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.764107+0000", 2024-09-18T00:18:34.866 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.9829881539999992, 2024-09-18T00:18:34.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028313351, 2024-09-18T00:18:34.866 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:34.866 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:34.866 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:34.866 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:34.866 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:34.866 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:34.866 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6631 2024-09-18T00:18:34.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.866 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:34.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:34.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.764107+0000", 2024-09-18T00:18:34.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:34.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.764107+0000", 2024-09-18T00:18:34.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:34.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:34.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.764112+0000", 2024-09-18T00:18:34.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6099999999999997e-06 2024-09-18T00:18:34.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:34.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.764879+0000", 2024-09-18T00:18:34.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00076643000000000002 2024-09-18T00:18:34.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:34.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.764911+0000", 2024-09-18T00:18:34.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2663000000000002e-05 2024-09-18T00:18:34.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:34.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.764927+0000", 2024-09-18T00:18:34.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5999999999999999e-05 2024-09-18T00:18:34.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:34.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.765049+0000", 2024-09-18T00:18:34.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012181899999999999 2024-09-18T00:18:34.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:34.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.765291+0000", 2024-09-18T00:18:34.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024234500000000001 2024-09-18T00:18:34.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:34.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.765686+0000", 2024-09-18T00:18:34.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039434300000000002 2024-09-18T00:18:34.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:34.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.791151+0000", 2024-09-18T00:18:34.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025465338000000001 2024-09-18T00:18:34.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:34.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.792115+0000", 2024-09-18T00:18:34.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00096422499999999998 2024-09-18T00:18:34.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:34.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.792292+0000", 2024-09-18T00:18:34.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017714599999999999 2024-09-18T00:18:34.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:34.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:34.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:34.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.792420+0000", 2024-09-18T00:18:34.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000127432 2024-09-18T00:18:34.871 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.871 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.871 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.871 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:34.872 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:34.872 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:34.874 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:18:34.875 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:34.875 INFO:teuthology.orchestra.run.smithi097.stdout:2023 read 0xbc4d22b thru 0xbc53e0c (0x6be2 bytes) 2024-09-18T00:18:34.875 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:34.875 INFO:teuthology.orchestra.run.smithi097.stdout:2025 write 0xc3d7a15 thru 0xc3dba60 (0x404c bytes) 2024-09-18T00:18:34.875 INFO:teuthology.orchestra.run.smithi097.stdout:2026 write 0xeac69b thru 0xeb77ce (0xb134 bytes) 2024-09-18T00:18:34.875 INFO:teuthology.orchestra.run.smithi097.stdout:2027 write 0x78b63de thru 0x78c3fbf (0xdbe2 bytes) 2024-09-18T00:18:34.876 INFO:teuthology.orchestra.run.smithi097.stdout:2028 write 0x340daac thru 0x3412add (0x5032 bytes) 2024-09-18T00:18:34.876 INFO:teuthology.orchestra.run.smithi097.stdout:2029 punch from 0xc7226eb to 0xc72e796, (0xc0ab bytes) 2024-09-18T00:18:34.876 INFO:teuthology.orchestra.run.smithi097.stdout:2030 read 0xdade94d thru 0xdaee066 (0xf71a bytes) 2024-09-18T00:18:34.876 INFO:teuthology.orchestra.run.smithi097.stdout:2031 read 0x57548f0 thru 0x575b472 (0x6b83 bytes) 2024-09-18T00:18:34.876 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:34.876 INFO:teuthology.orchestra.run.smithi097.stdout:2034 writesame 0xd4580ac thru 0xd4596d3 data_size 0x4(0x1628 bytes) 2024-09-18T00:18:34.876 INFO:teuthology.orchestra.run.smithi097.stdout:2035 write 0xac5f5df thru 0xac6eb25 (0xf547 bytes) 2024-09-18T00:18:34.876 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:34.876 INFO:teuthology.orchestra.run.smithi097.stdout:2037 read 0x85b589c thru 0x85b8092 (0x27f7 bytes) 2024-09-18T00:18:34.876 INFO:teuthology.orchestra.run.smithi097.stdout:2039 read 0x147aaef thru 0x1482e4c (0x835e bytes) 2024-09-18T00:18:34.876 INFO:teuthology.orchestra.run.smithi097.stdout:2040 write 0x9aae6d8 thru 0x9ab06af (0x1fd8 bytes) 2024-09-18T00:18:34.876 INFO:teuthology.orchestra.run.smithi097.stdout:2044 trunc from 0xdb39fbf to 0xe1017cd 2024-09-18T00:18:34.877 INFO:teuthology.orchestra.run.smithi097.stdout:2046 writesame 0x1f073e0 thru 0x1f16163 data_size 0xc(0xed84 bytes) 2024-09-18T00:18:34.877 INFO:teuthology.orchestra.run.smithi097.stdout:2048 read 0x3ebd78d thru 0x3ecb91f (0xe193 bytes) 2024-09-18T00:18:34.877 INFO:teuthology.orchestra.run.smithi097.stdout:2049 read 0x78644c8 thru 0x786ed16 (0xa84f bytes) 2024-09-18T00:18:34.877 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:34.870+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:34.878 INFO:teuthology.orchestra.run.smithi097.stdout:2050 punch from 0x247daab to 0x247e293, (0x7e8 bytes) 2024-09-18T00:18:34.878 INFO:teuthology.orchestra.run.smithi097.stdout:2051 punch from 0x56941b1 to 0x5695d29, (0x1b78 bytes) 2024-09-18T00:18:34.878 INFO:teuthology.orchestra.run.smithi097.stdout:2052 read 0x28620a1 thru 0x286a4a5 (0x8405 bytes) 2024-09-18T00:18:34.878 INFO:teuthology.orchestra.run.smithi097.stdout:2053 write 0x2436e4f thru 0x244338a (0xc53c bytes) 2024-09-18T00:18:34.887 INFO:teuthology.orchestra.run.smithi097.stdout:2054 write 0xc298ab2 thru 0xc2a605d (0xd5ac bytes) 2024-09-18T00:18:34.898 INFO:teuthology.orchestra.run.smithi097.stdout:2055 write 0xcdcdc65 thru 0xcdd49c1 (0x6d5d bytes) 2024-09-18T00:18:34.922 INFO:teuthology.orchestra.run.smithi097.stdout:2056 write 0xe719085 thru 0xe71afdb (0x1f57 bytes) 2024-09-18T00:18:34.929 INFO:teuthology.orchestra.run.smithi097.stdout:2058 writesame 0x21cd70f thru 0x21d8067 data_size 0x3(0xa959 bytes) 2024-09-18T00:18:34.948 INFO:teuthology.orchestra.run.smithi097.stdout:2060 read 0x4ff33d0 thru 0x4ffb7b3 (0x83e4 bytes) 2024-09-18T00:18:34.948 INFO:teuthology.orchestra.run.smithi097.stdout:2061 read 0x8e492d thru 0x8f18a4 (0xcf78 bytes) 2024-09-18T00:18:34.951 INFO:teuthology.orchestra.run.smithi097.stdout:2062 trunc from 0xe71afdc to 0xd1f2b5a 2024-09-18T00:18:34.966 INFO:teuthology.orchestra.run.smithi097.stdout:2065 punch from 0x7b1608e to 0x7b216c6, (0xb638 bytes) 2024-09-18T00:18:34.974 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:34.970+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:34.987 INFO:teuthology.orchestra.run.smithi097.stdout:2067 trunc from 0xd1f2b5a to 0x1987633 2024-09-18T00:18:35.019 INFO:teuthology.orchestra.run.smithi097.stdout:2068 read 0x15cee57 thru 0x15d513f (0x62e9 bytes) 2024-09-18T00:18:35.019 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:35.019 INFO:teuthology.orchestra.run.smithi097.stdout:2072 read 0x19418fe thru 0x19451e4 (0x38e7 bytes) 2024-09-18T00:18:35.019 INFO:teuthology.orchestra.run.smithi097.stdout:2073 read 0x227fad thru 0x22f0ce (0x7122 bytes) 2024-09-18T00:18:35.040 INFO:teuthology.orchestra.run.smithi097.stdout:2075 trunc from 0x1987633 to 0xbb3b32f 2024-09-18T00:18:35.049 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:35.049 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:35.049 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:18:35.049 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:35.052 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:35.052 INFO:teuthology.orchestra.run.smithi097.stdout:2077 read 0x6d6ea4e thru 0x6d71bce (0x3181 bytes) 2024-09-18T00:18:35.052 INFO:teuthology.orchestra.run.smithi097.stdout:2078 read 0x5accda3 thru 0x5ad18c2 (0x4b20 bytes) 2024-09-18T00:18:35.052 INFO:teuthology.orchestra.run.smithi097.stdout:2079 read 0xb46e239 thru 0xb479e64 (0xbc2c bytes) 2024-09-18T00:18:35.065 INFO:teuthology.orchestra.run.smithi097.stdout:2080 write 0xc1bf505 thru 0xc1c883a (0x9336 bytes) 2024-09-18T00:18:35.066 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:18:35.075 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:35.070+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:35.080 INFO:teuthology.orchestra.run.smithi097.stdout:2081 writesame 0x3e02839 thru 0x3e053bb data_size 0x2f(0x2b83 bytes) 2024-09-18T00:18:35.087 INFO:teuthology.orchestra.run.smithi097.stdout:2082 writesame 0xa5d7800 thru 0xa5da3d6 data_size 0x2b(0x2bd7 bytes) 2024-09-18T00:18:35.095 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:35.099 INFO:teuthology.orchestra.run.smithi097.stdout:2084 trunc from 0xc1c883b to 0xaafeb8b 2024-09-18T00:18:35.115 INFO:teuthology.orchestra.run.smithi097.stdout:2085 write 0x8dd8e09 thru 0x8de0cd1 (0x7ec9 bytes) 2024-09-18T00:18:35.146 INFO:teuthology.orchestra.run.smithi097.stdout:2086 write 0xe095bec thru 0xe09a8fa (0x4d0f bytes) 2024-09-18T00:18:35.156 INFO:teuthology.orchestra.run.smithi097.stdout:2087 read 0x4127dfa thru 0x41366d4 (0xe8db bytes) 2024-09-18T00:18:35.176 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:35.170+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:35.180 INFO:teuthology.orchestra.run.smithi097.stdout:2088 trunc from 0xe09a8fb to 0x1c8ce0b 2024-09-18T00:18:35.202 INFO:teuthology.orchestra.run.smithi097.stdout:2089 read 0x14d6eef thru 0x14de1cc (0x72de bytes) 2024-09-18T00:18:35.232 INFO:teuthology.orchestra.run.smithi097.stdout:2090 write 0xd4ab50a thru 0xd4b75db (0xc0d2 bytes) 2024-09-18T00:18:35.235 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:35.236 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:35.236 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:35.236 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:35.236 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:35.251 INFO:teuthology.orchestra.run.smithi097.stdout:2091 writesame 0xaada6ce thru 0xaae93c0 data_size 0x1(0xecf3 bytes) 2024-09-18T00:18:35.255 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:18:35.302 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:35.298+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:2092 read 0x6fe58ca thru 0x6fe6da4 (0x14db bytes) 2024-09-18T00:18:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:2093 read 0xa17ff4e thru 0xa183344 (0x33f7 bytes) 2024-09-18T00:18:35.308 INFO:teuthology.orchestra.run.smithi097.stdout:2094 punch from 0x889766 to 0x8943de, (0xac78 bytes) 2024-09-18T00:18:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:2095 read 0xa19244f thru 0xa19866a (0x621c bytes) 2024-09-18T00:18:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:2096 read 0x802f2fe thru 0x8036dbc (0x7abf bytes) 2024-09-18T00:18:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:35.309 INFO:teuthology.orchestra.run.smithi097.stdout:2099 write 0x843f171 thru 0x844e837 (0xf6c7 bytes) 2024-09-18T00:18:35.320 INFO:teuthology.orchestra.run.smithi097.stdout:2100 read 0x3971b48 thru 0x397881c (0x6cd5 bytes) 2024-09-18T00:18:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:2101 punch from 0x8847e05 to 0x884ef5f, (0x715a bytes) 2024-09-18T00:18:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:35.321 INFO:teuthology.orchestra.run.smithi097.stdout:2106 write 0x803c9ea thru 0x8044f7e (0x8595 bytes) 2024-09-18T00:18:35.337 INFO:teuthology.orchestra.run.smithi097.stdout:2107 write 0xe6710b4 thru 0xe67d671 (0xc5be bytes) 2024-09-18T00:18:35.350 INFO:teuthology.orchestra.run.smithi097.stdout:2108 writesame 0x856c040 thru 0x85758f3 data_size 0x1d(0x98b4 bytes) 2024-09-18T00:18:35.353 INFO:teuthology.orchestra.run.smithi097.stdout:2109 write 0xb0b1ac6 thru 0xb0bf920 (0xde5b bytes) 2024-09-18T00:18:35.366 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:35.366 INFO:teuthology.orchestra.run.smithi097.stdout:2112 trunc from 0xe67d672 to 0xe868097 2024-09-18T00:18:35.374 INFO:teuthology.orchestra.run.smithi097.stdout:2113 clone 7 order 21 su 16384 sc 5 2024-09-18T00:18:35.377 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:35.370+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:35.395 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:35.396 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:35.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:35.396 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:35.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.396 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4394.0:792 3.14 e30/27 3:29eec297:::rbd_data.112ae20746f5.000000000000000c:head v 30'11, mlcod=29'9)", 2024-09-18T00:18:35.396 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:12.516730+0000", 2024-09-18T00:18:35.396 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.876841296999999, 2024-09-18T00:18:35.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049957619999999999, 2024-09-18T00:18:35.396 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.396 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.396 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:35.396 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.516730+0000", 2024-09-18T00:18:35.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.516730+0000", 2024-09-18T00:18:35.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.516731+0000", 2024-09-18T00:18:35.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6139999999999999e-06 2024-09-18T00:18:35.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.516769+0000", 2024-09-18T00:18:35.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7286000000000002e-05 2024-09-18T00:18:35.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.516792+0000", 2024-09-18T00:18:35.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.279e-05 2024-09-18T00:18:35.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.516805+0000", 2024-09-18T00:18:35.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3203999999999999e-05 2024-09-18T00:18:35.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.516945+0000", 2024-09-18T00:18:35.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014018200000000001 2024-09-18T00:18:35.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.517010+0000", 2024-09-18T00:18:35.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5046e-05 2024-09-18T00:18:35.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:35.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.521645+0000", 2024-09-18T00:18:35.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046348700000000001 2024-09-18T00:18:35.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:12.521726+0000", 2024-09-18T00:18:35.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0770000000000001e-05 2024-09-18T00:18:35.400 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.400 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.400 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.400 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2229 3.10 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:18:35.400 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:18:35.400 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.130212555, 2024-09-18T00:18:35.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048730086999999998, 2024-09-18T00:18:35.401 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.401 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.401 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:35.401 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:35.401 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:35.401 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:35.401 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2229 2024-09-18T00:18:35.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.401 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:18:35.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:18:35.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263362+0000", 2024-09-18T00:18:35.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3869999999999999e-06 2024-09-18T00:18:35.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263381+0000", 2024-09-18T00:18:35.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9153000000000001e-05 2024-09-18T00:18:35.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263418+0000", 2024-09-18T00:18:35.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6346000000000001e-05 2024-09-18T00:18:35.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263435+0000", 2024-09-18T00:18:35.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7309999999999999e-05 2024-09-18T00:18:35.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263589+0000", 2024-09-18T00:18:35.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015451199999999999 2024-09-18T00:18:35.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263902+0000", 2024-09-18T00:18:35.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031291799999999997 2024-09-18T00:18:35.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:35.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.265995+0000", 2024-09-18T00:18:35.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020931809999999999 2024-09-18T00:18:35.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:35.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311788+0000", 2024-09-18T00:18:35.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045792078 2024-09-18T00:18:35.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:35.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311903+0000", 2024-09-18T00:18:35.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011510699999999999 2024-09-18T00:18:35.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:35.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312009+0000", 2024-09-18T00:18:35.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010630599999999999 2024-09-18T00:18:35.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312089+0000", 2024-09-18T00:18:35.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9789000000000005e-05 2024-09-18T00:18:35.406 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.406 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.406 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.406 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2229 3.10 e36/23 ondisk, result = 0)", 2024-09-18T00:18:35.406 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:18:35.406 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.111066896000001, 2024-09-18T00:18:35.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029600955000000002, 2024-09-18T00:18:35.406 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.406 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.407 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:35.407 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:18:35.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:18:35.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282507+0000", 2024-09-18T00:18:35.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9160000000000001e-06 2024-09-18T00:18:35.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282522+0000", 2024-09-18T00:18:35.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5e-05 2024-09-18T00:18:35.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282558+0000", 2024-09-18T00:18:35.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5694000000000001e-05 2024-09-18T00:18:35.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282580+0000", 2024-09-18T00:18:35.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2323e-05 2024-09-18T00:18:35.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311873+0000", 2024-09-18T00:18:35.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029293013999999999 2024-09-18T00:18:35.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311892+0000", 2024-09-18T00:18:35.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8556e-05 2024-09-18T00:18:35.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312105+0000", 2024-09-18T00:18:35.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021345199999999999 2024-09-18T00:18:35.410 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.410 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.410 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.410 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2276 3.10 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head [truncate 98304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:35.410 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.451881+0000", 2024-09-18T00:18:35.410 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.941690267, 2024-09-18T00:18:35.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068466680000000002, 2024-09-18T00:18:35.411 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.411 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.411 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:35.411 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:35.411 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:35.411 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:35.411 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2276 2024-09-18T00:18:35.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.411 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.451881+0000", 2024-09-18T00:18:35.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.451881+0000", 2024-09-18T00:18:35.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.451884+0000", 2024-09-18T00:18:35.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.32e-06 2024-09-18T00:18:35.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.451902+0000", 2024-09-18T00:18:35.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7623000000000001e-05 2024-09-18T00:18:35.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.451941+0000", 2024-09-18T00:18:35.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.892e-05 2024-09-18T00:18:35.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.451964+0000", 2024-09-18T00:18:35.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3615999999999999e-05 2024-09-18T00:18:35.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.452149+0000", 2024-09-18T00:18:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018466399999999999 2024-09-18T00:18:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:18:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.452170+0000", 2024-09-18T00:18:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0448000000000001e-05 2024-09-18T00:18:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.455821+0000", 2024-09-18T00:18:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036518789999999998 2024-09-18T00:18:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.455988+0000", 2024-09-18T00:18:35.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016684800000000001 2024-09-18T00:18:35.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:35.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.456204+0000", 2024-09-18T00:18:35.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021615600000000001 2024-09-18T00:18:35.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:35.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.458193+0000", 2024-09-18T00:18:35.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019886349999999999 2024-09-18T00:18:35.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:35.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.458580+0000", 2024-09-18T00:18:35.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00038688999999999998 2024-09-18T00:18:35.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:35.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.458666+0000", 2024-09-18T00:18:35.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5935999999999998e-05 2024-09-18T00:18:35.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.458728+0000", 2024-09-18T00:18:35.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1733e-05 2024-09-18T00:18:35.417 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.417 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.417 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.417 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4394.0:2314 3.1e e37/23 3:785f9b7b:::rbd_data.112aa7ba4578.0000000000000005:head v 37'18, mlcod=36'16)", 2024-09-18T00:18:35.417 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.532988+0000", 2024-09-18T00:18:35.417 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.860582945999999, 2024-09-18T00:18:35.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055938359999999996, 2024-09-18T00:18:35.417 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.417 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.417 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:35.417 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.532988+0000", 2024-09-18T00:18:35.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.532988+0000", 2024-09-18T00:18:35.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.532989+0000", 2024-09-18T00:18:35.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6700000000000003e-07 2024-09-18T00:18:35.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.533082+0000", 2024-09-18T00:18:35.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3394999999999996e-05 2024-09-18T00:18:35.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.533113+0000", 2024-09-18T00:18:35.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0926999999999997e-05 2024-09-18T00:18:35.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.533131+0000", 2024-09-18T00:18:35.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7433000000000001e-05 2024-09-18T00:18:35.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.533286+0000", 2024-09-18T00:18:35.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015501600000000001 2024-09-18T00:18:35.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.533347+0000", 2024-09-18T00:18:35.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1415999999999996e-05 2024-09-18T00:18:35.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.421 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:35.421 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.538496+0000", 2024-09-18T00:18:35.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051492760000000004 2024-09-18T00:18:35.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.421 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.421 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.538582+0000", 2024-09-18T00:18:35.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5805999999999998e-05 2024-09-18T00:18:35.421 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.421 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.421 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.422 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2341 3.10 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:35.422 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.590152+0000", 2024-09-18T00:18:35.422 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.803419451, 2024-09-18T00:18:35.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081859440000000006, 2024-09-18T00:18:35.422 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.422 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.422 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:35.422 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:35.422 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:35.422 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:35.422 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2341 2024-09-18T00:18:35.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.422 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.590152+0000", 2024-09-18T00:18:35.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.590152+0000", 2024-09-18T00:18:35.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.590155+0000", 2024-09-18T00:18:35.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2799999999999999e-06 2024-09-18T00:18:35.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.590170+0000", 2024-09-18T00:18:35.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.455e-05 2024-09-18T00:18:35.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.590206+0000", 2024-09-18T00:18:35.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6056000000000002e-05 2024-09-18T00:18:35.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.590227+0000", 2024-09-18T00:18:35.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1189999999999999e-05 2024-09-18T00:18:35.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.425 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.425 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.590413+0000", 2024-09-18T00:18:35.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000185966 2024-09-18T00:18:35.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.425 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.425 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.590664+0000", 2024-09-18T00:18:35.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025139100000000001 2024-09-18T00:18:35.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.425 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:35.425 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.590937+0000", 2024-09-18T00:18:35.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027242800000000001 2024-09-18T00:18:35.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:35.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.594857+0000", 2024-09-18T00:18:35.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0039203229999999999 2024-09-18T00:18:35.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:35.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.598185+0000", 2024-09-18T00:18:35.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0033282749999999999 2024-09-18T00:18:35.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:35.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.598271+0000", 2024-09-18T00:18:35.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5482000000000006e-05 2024-09-18T00:18:35.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.598338+0000", 2024-09-18T00:18:35.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7002999999999995e-05 2024-09-18T00:18:35.427 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.427 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.427 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.427 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2819 3.a 3:565076c5:::rbd_data.112aa7ba4578.0000000000000078:head [set-alloc-hint object_size 262144 write_size 262144,write 32768~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:18:35.427 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.776873+0000", 2024-09-18T00:18:35.428 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.61669829, 2024-09-18T00:18:35.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006824681, 2024-09-18T00:18:35.428 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.428 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.428 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:35.428 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:35.428 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:35.428 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:35.428 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2819 2024-09-18T00:18:35.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.428 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.776873+0000", 2024-09-18T00:18:35.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.776873+0000", 2024-09-18T00:18:35.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.776877+0000", 2024-09-18T00:18:35.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1629999999999998e-06 2024-09-18T00:18:35.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.430 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.430 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.776998+0000", 2024-09-18T00:18:35.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120646 2024-09-18T00:18:35.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.430 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.430 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.777040+0000", 2024-09-18T00:18:35.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1922999999999998e-05 2024-09-18T00:18:35.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.430 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.430 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.777064+0000", 2024-09-18T00:18:35.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4447000000000002e-05 2024-09-18T00:18:35.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.777263+0000", 2024-09-18T00:18:35.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019846800000000001 2024-09-18T00:18:35.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:18:35.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.777282+0000", 2024-09-18T00:18:35.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9783e-05 2024-09-18T00:18:35.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.780740+0000", 2024-09-18T00:18:35.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034576110000000002 2024-09-18T00:18:35.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.780952+0000", 2024-09-18T00:18:35.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000211848 2024-09-18T00:18:35.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:35.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.781191+0000", 2024-09-18T00:18:35.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000239068 2024-09-18T00:18:35.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:35.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.783293+0000", 2024-09-18T00:18:35.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0021019810000000002 2024-09-18T00:18:35.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:35.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.783544+0000", 2024-09-18T00:18:35.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000251384 2024-09-18T00:18:35.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:35.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.783634+0000", 2024-09-18T00:18:35.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9957000000000006e-05 2024-09-18T00:18:35.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.783698+0000", 2024-09-18T00:18:35.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3402e-05 2024-09-18T00:18:35.434 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.434 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.434 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.434 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:4828 3.a 3:56b25a6e:::rbd_data.112aa7ba4578.0000000000000216:head [sparse-read 0~2750] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:35.434 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:24.991934+0000", 2024-09-18T00:18:35.435 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.401636694, 2024-09-18T00:18:35.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049260349999999996, 2024-09-18T00:18:35.435 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.435 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.435 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:35.435 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:35.435 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:35.435 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:35.435 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 4828 2024-09-18T00:18:35.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.435 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.991934+0000", 2024-09-18T00:18:35.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.991934+0000", 2024-09-18T00:18:35.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.991936+0000", 2024-09-18T00:18:35.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1459999999999999e-06 2024-09-18T00:18:35.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.991941+0000", 2024-09-18T00:18:35.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3140000000000003e-06 2024-09-18T00:18:35.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.991958+0000", 2024-09-18T00:18:35.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7146000000000002e-05 2024-09-18T00:18:35.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.991983+0000", 2024-09-18T00:18:35.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4757e-05 2024-09-18T00:18:35.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.996362+0000", 2024-09-18T00:18:35.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0043790230000000001 2024-09-18T00:18:35.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.996543+0000", 2024-09-18T00:18:35.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018138200000000001 2024-09-18T00:18:35.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.996861+0000", 2024-09-18T00:18:35.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031726699999999998 2024-09-18T00:18:35.439 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.439 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.439 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.439 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:4832 3.a 3:57dc2db7:::rbd_data.112aa7ba4578.000000000000021a:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:35.439 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:24.992012+0000", 2024-09-18T00:18:35.439 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.401559560999999, 2024-09-18T00:18:35.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051891569999999998, 2024-09-18T00:18:35.439 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.439 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.440 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:18:35.440 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:35.440 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:35.440 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:35.440 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 4832 2024-09-18T00:18:35.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.440 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.992012+0000", 2024-09-18T00:18:35.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.992012+0000", 2024-09-18T00:18:35.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.992013+0000", 2024-09-18T00:18:35.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.42e-06 2024-09-18T00:18:35.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.992019+0000", 2024-09-18T00:18:35.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9730000000000002e-06 2024-09-18T00:18:35.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.992036+0000", 2024-09-18T00:18:35.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6895999999999999e-05 2024-09-18T00:18:35.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.992049+0000", 2024-09-18T00:18:35.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3526999999999999e-05 2024-09-18T00:18:35.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.996945+0000", 2024-09-18T00:18:35.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048952129999999998 2024-09-18T00:18:35.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.997201+0000", 2024-09-18T00:18:35.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000256128 2024-09-18T00:18:35.443 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.443 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.443 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.443 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:4841 3.a 3:52c13ab5:::rbd_data.112aa7ba4578.0000000000000223:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:35.443 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:24.992089+0000", 2024-09-18T00:18:35.443 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.401482649, 2024-09-18T00:18:35.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0058043859999999999, 2024-09-18T00:18:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:18:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 4841 2024-09-18T00:18:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.992089+0000", 2024-09-18T00:18:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.992089+0000", 2024-09-18T00:18:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.992090+0000", 2024-09-18T00:18:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.277e-06 2024-09-18T00:18:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.992096+0000", 2024-09-18T00:18:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4799999999999998e-06 2024-09-18T00:18:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.992115+0000", 2024-09-18T00:18:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8283000000000001e-05 2024-09-18T00:18:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.992129+0000", 2024-09-18T00:18:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.413e-05 2024-09-18T00:18:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.997341+0000", 2024-09-18T00:18:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052119269999999999 2024-09-18T00:18:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:24.997893+0000", 2024-09-18T00:18:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055228899999999997 2024-09-18T00:18:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5714 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:18:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.083364393, 2024-09-18T00:18:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027448727999999999, 2024-09-18T00:18:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5714 2024-09-18T00:18:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:18:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:18:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310211+0000", 2024-09-18T00:18:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5639999999999998e-06 2024-09-18T00:18:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312678+0000", 2024-09-18T00:18:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002466761 2024-09-18T00:18:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312746+0000", 2024-09-18T00:18:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8046000000000005e-05 2024-09-18T00:18:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312774+0000", 2024-09-18T00:18:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7813e-05 2024-09-18T00:18:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312927+0000", 2024-09-18T00:18:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015328499999999999 2024-09-18T00:18:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313214+0000", 2024-09-18T00:18:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028683099999999998 2024-09-18T00:18:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313708+0000", 2024-09-18T00:18:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049379100000000004 2024-09-18T00:18:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336895+0000", 2024-09-18T00:18:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023186801 2024-09-18T00:18:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337454+0000", 2024-09-18T00:18:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055979399999999996 2024-09-18T00:18:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337571+0000", 2024-09-18T00:18:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116821 2024-09-18T00:18:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337656+0000", 2024-09-18T00:18:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4221000000000006e-05 2024-09-18T00:18:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:5714 3.10 e41/23 ondisk, result = 0)", 2024-09-18T00:18:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.321328+0000", 2024-09-18T00:18:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.072243346, 2024-09-18T00:18:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015589295, 2024-09-18T00:18:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.321328+0000", 2024-09-18T00:18:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.321328+0000", 2024-09-18T00:18:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.321330+0000", 2024-09-18T00:18:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.813e-06 2024-09-18T00:18:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.321341+0000", 2024-09-18T00:18:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1174e-05 2024-09-18T00:18:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.321372+0000", 2024-09-18T00:18:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0800000000000003e-05 2024-09-18T00:18:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.321390+0000", 2024-09-18T00:18:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7995e-05 2024-09-18T00:18:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336863+0000", 2024-09-18T00:18:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015472921000000001 2024-09-18T00:18:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336883+0000", 2024-09-18T00:18:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0069999999999999e-05 2024-09-18T00:18:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336917+0000", 2024-09-18T00:18:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4521999999999999e-05 2024-09-18T00:18:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4394.0:5724 3.1e e41/23 3:78324145:::rbd_data.112a3a39fc45.000000000000001e:head v 41'25, mlcod=39'23)", 2024-09-18T00:18:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.343133+0000", 2024-09-18T00:18:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.050437970999999, 2024-09-18T00:18:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012348726000000001, 2024-09-18T00:18:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343133+0000", 2024-09-18T00:18:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343133+0000", 2024-09-18T00:18:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343134+0000", 2024-09-18T00:18:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9999999999999998e-07 2024-09-18T00:18:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345249+0000", 2024-09-18T00:18:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002115025 2024-09-18T00:18:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345278+0000", 2024-09-18T00:18:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9648000000000001e-05 2024-09-18T00:18:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345296+0000", 2024-09-18T00:18:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7268000000000001e-05 2024-09-18T00:18:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345421+0000", 2024-09-18T00:18:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012594900000000001 2024-09-18T00:18:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345459+0000", 2024-09-18T00:18:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7425999999999997e-05 2024-09-18T00:18:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.355406+0000", 2024-09-18T00:18:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0099466039999999995 2024-09-18T00:18:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.355482+0000", 2024-09-18T00:18:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6406e-05 2024-09-18T00:18:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5729 3.a 3:5683c0f9:::rbd_data.112a3a39fc45.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.345361+0000", 2024-09-18T00:18:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.048210144, 2024-09-18T00:18:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052351749999999999, 2024-09-18T00:18:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5729 2024-09-18T00:18:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345361+0000", 2024-09-18T00:18:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345361+0000", 2024-09-18T00:18:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345364+0000", 2024-09-18T00:18:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6800000000000002e-06 2024-09-18T00:18:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345377+0000", 2024-09-18T00:18:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3386e-05 2024-09-18T00:18:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345401+0000", 2024-09-18T00:18:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3710000000000002e-05 2024-09-18T00:18:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345420+0000", 2024-09-18T00:18:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9250999999999999e-05 2024-09-18T00:18:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345531+0000", 2024-09-18T00:18:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011063 2024-09-18T00:18:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345782+0000", 2024-09-18T00:18:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025128599999999998 2024-09-18T00:18:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346112+0000", 2024-09-18T00:18:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033004900000000001 2024-09-18T00:18:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347307+0000", 2024-09-18T00:18:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001195057 2024-09-18T00:18:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.350405+0000", 2024-09-18T00:18:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00309801 2024-09-18T00:18:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.350518+0000", 2024-09-18T00:18:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112617 2024-09-18T00:18:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.350596+0000", 2024-09-18T00:18:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8498999999999995e-05 2024-09-18T00:18:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4394.0:5869 3.1e e41/23 3:78324145:::rbd_data.112a3a39fc45.000000000000001e:head v 41'26, mlcod=39'24)", 2024-09-18T00:18:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.453873+0000", 2024-09-18T00:18:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9396984780000004, 2024-09-18T00:18:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005189, 2024-09-18T00:18:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.453873+0000", 2024-09-18T00:18:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.453873+0000", 2024-09-18T00:18:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.453875+0000", 2024-09-18T00:18:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8199999999999999e-06 2024-09-18T00:18:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.453888+0000", 2024-09-18T00:18:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2982999999999999e-05 2024-09-18T00:18:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.453916+0000", 2024-09-18T00:18:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8203000000000001e-05 2024-09-18T00:18:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.453933+0000", 2024-09-18T00:18:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7456999999999999e-05 2024-09-18T00:18:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.454080+0000", 2024-09-18T00:18:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000147059 2024-09-18T00:18:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.454159+0000", 2024-09-18T00:18:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9039e-05 2024-09-18T00:18:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.458979+0000", 2024-09-18T00:18:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048198199999999998 2024-09-18T00:18:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.459062+0000", 2024-09-18T00:18:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2619000000000003e-05 2024-09-18T00:18:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4394.0:6468 3.1e e42/23 3:79a5efb9:::rbd_data.112a3a39fc45.0000000000000015:head v 42'27, mlcod=41'25)", 2024-09-18T00:18:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.521049+0000", 2024-09-18T00:18:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.8725221629999993, 2024-09-18T00:18:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005705796, 2024-09-18T00:18:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.521049+0000", 2024-09-18T00:18:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.521049+0000", 2024-09-18T00:18:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.521049+0000", 2024-09-18T00:18:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.58e-07 2024-09-18T00:18:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.522051+0000", 2024-09-18T00:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010019880000000001 2024-09-18T00:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.522129+0000", 2024-09-18T00:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7891000000000004e-05 2024-09-18T00:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.522144+0000", 2024-09-18T00:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5149e-05 2024-09-18T00:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.522211+0000", 2024-09-18T00:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6701999999999999e-05 2024-09-18T00:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.522229+0000", 2024-09-18T00:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7909000000000001e-05 2024-09-18T00:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.526700+0000", 2024-09-18T00:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044709789999999999 2024-09-18T00:18:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.526755+0000", 2024-09-18T00:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.482e-05 2024-09-18T00:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6752 3.a 3:5683c0f9:::rbd_data.112a3a39fc45.0000000000000021:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2621440~24935] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.836400+0000", 2024-09-18T00:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5571712269999995, 2024-09-18T00:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054133580000000001, 2024-09-18T00:18:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6752 2024-09-18T00:18:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.836400+0000", 2024-09-18T00:18:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.836400+0000", 2024-09-18T00:18:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.836403+0000", 2024-09-18T00:18:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9969999999999999e-06 2024-09-18T00:18:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.836467+0000", 2024-09-18T00:18:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4035000000000006e-05 2024-09-18T00:18:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.836500+0000", 2024-09-18T00:18:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3317e-05 2024-09-18T00:18:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.836516+0000", 2024-09-18T00:18:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5622999999999999e-05 2024-09-18T00:18:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.836663+0000", 2024-09-18T00:18:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014699900000000001 2024-09-18T00:18:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.836799+0000", 2024-09-18T00:18:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013638300000000001 2024-09-18T00:18:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.837019+0000", 2024-09-18T00:18:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000219874 2024-09-18T00:18:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.841415+0000", 2024-09-18T00:18:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0043961759999999999 2024-09-18T00:18:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.841659+0000", 2024-09-18T00:18:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000243988 2024-09-18T00:18:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.841746+0000", 2024-09-18T00:18:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7070000000000005e-05 2024-09-18T00:18:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.841813+0000", 2024-09-18T00:18:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6896000000000005e-05 2024-09-18T00:18:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6801 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:18:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3565279449999998, 2024-09-18T00:18:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051953602000000002, 2024-09-18T00:18:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6801 2024-09-18T00:18:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:18:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:18:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037046+0000", 2024-09-18T00:18:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.19e-06 2024-09-18T00:18:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037062+0000", 2024-09-18T00:18:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586999999999998e-05 2024-09-18T00:18:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037096+0000", 2024-09-18T00:18:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4112000000000001e-05 2024-09-18T00:18:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037120+0000", 2024-09-18T00:18:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.355e-05 2024-09-18T00:18:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037246+0000", 2024-09-18T00:18:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012652200000000001 2024-09-18T00:18:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037385+0000", 2024-09-18T00:18:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013924599999999999 2024-09-18T00:18:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.040881+0000", 2024-09-18T00:18:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034954830000000002 2024-09-18T00:18:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088794+0000", 2024-09-18T00:18:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047912897000000003 2024-09-18T00:18:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088828+0000", 2024-09-18T00:18:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3880000000000001e-05 2024-09-18T00:18:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088916+0000", 2024-09-18T00:18:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.865e-05 2024-09-18T00:18:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088997+0000", 2024-09-18T00:18:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0484999999999999e-05 2024-09-18T00:18:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:6801 3.10 e42/23 ondisk, result = 0)", 2024-09-18T00:18:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:18:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3427205349999998, 2024-09-18T00:18:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037960394000000001, 2024-09-18T00:18:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:18:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:18:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050852+0000", 2024-09-18T00:18:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:18:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050860+0000", 2024-09-18T00:18:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7729999999999999e-06 2024-09-18T00:18:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050891+0000", 2024-09-18T00:18:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0617000000000002e-05 2024-09-18T00:18:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050910+0000", 2024-09-18T00:18:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.944e-05 2024-09-18T00:18:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088766+0000", 2024-09-18T00:18:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037855633999999999 2024-09-18T00:18:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088784+0000", 2024-09-18T00:18:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7767000000000001e-05 2024-09-18T00:18:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088811+0000", 2024-09-18T00:18:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7382999999999999e-05 2024-09-18T00:18:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:10939 3.a 3:565c2ebb:::rbd_data.112ab1fa102a.0000000000000134:head [set-alloc-hint object_size 524288 write_size 524288,write 147456~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:34.619286+0000", 2024-09-18T00:18:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.77428532400000005, 2024-09-18T00:18:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055090640000000001, 2024-09-18T00:18:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 10939 2024-09-18T00:18:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:34.619286+0000", 2024-09-18T00:18:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:34.619286+0000", 2024-09-18T00:18:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:34.619289+0000", 2024-09-18T00:18:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7099999999999999e-06 2024-09-18T00:18:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:34.619306+0000", 2024-09-18T00:18:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7702000000000001e-05 2024-09-18T00:18:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:34.619327+0000", 2024-09-18T00:18:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0817000000000001e-05 2024-09-18T00:18:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:34.619336+0000", 2024-09-18T00:18:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1509999999999993e-06 2024-09-18T00:18:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:34.619409+0000", 2024-09-18T00:18:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2583999999999995e-05 2024-09-18T00:18:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:18:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:34.619415+0000", 2024-09-18T00:18:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1809999999999998e-06 2024-09-18T00:18:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:34.621625+0000", 2024-09-18T00:18:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002209801 2024-09-18T00:18:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:34.621677+0000", 2024-09-18T00:18:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2298e-05 2024-09-18T00:18:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:34.621791+0000", 2024-09-18T00:18:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000113608 2024-09-18T00:18:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:34.623714+0000", 2024-09-18T00:18:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001922792 2024-09-18T00:18:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:34.624699+0000", 2024-09-18T00:18:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00098573699999999998 2024-09-18T00:18:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:34.624755+0000", 2024-09-18T00:18:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6183000000000001e-05 2024-09-18T00:18:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:34.624795+0000", 2024-09-18T00:18:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9499999999999998e-05 2024-09-18T00:18:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:35.500 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:35.501 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:18:35.502 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:35.498+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:35.602 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:35.598+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:35.673 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:35.673 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:35.673 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:18:35.673 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:35.687 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:18:35.702 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:35.698+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:35.803 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:35.798+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:35.805 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:35.805 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:35.805 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:35.805 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:35.806 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:35.820 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:18:35.910 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:35.906+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:35.930 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-18T00:18:35.995 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:35.995 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:35.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:35.995 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:35.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.995 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2228 3.15 3:aa2fc059:::rbd_data.112aa7ba4578.000000000000014a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:18:35.995 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:18:35.995 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.727349411000001, 2024-09-18T00:18:35.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.068966377999999995, 2024-09-18T00:18:35.996 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:35.996 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:35.996 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:35.996 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:35.996 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:35.996 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:35.996 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2228 2024-09-18T00:18:35.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.996 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:35.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:35.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:18:35.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:35.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:18:35.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:35.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:35.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263099+0000", 2024-09-18T00:18:35.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2359999999999999e-06 2024-09-18T00:18:35.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:35.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263115+0000", 2024-09-18T00:18:35.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5996999999999999e-05 2024-09-18T00:18:35.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:35.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263156+0000", 2024-09-18T00:18:35.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:18:35.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:35.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263183+0000", 2024-09-18T00:18:35.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7233000000000002e-05 2024-09-18T00:18:35.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:35.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263304+0000", 2024-09-18T00:18:35.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121209 2024-09-18T00:18:35.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:35.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263528+0000", 2024-09-18T00:18:35.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022401800000000001 2024-09-18T00:18:35.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:35.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:35.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:18:35.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.266458+0000", 2024-09-18T00:18:35.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029296980000000001 2024-09-18T00:18:36.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:36.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331628+0000", 2024-09-18T00:18:36.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.065170045999999995 2024-09-18T00:18:36.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:36.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331819+0000", 2024-09-18T00:18:36.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000190819 2024-09-18T00:18:36.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:36.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331971+0000", 2024-09-18T00:18:36.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151972 2024-09-18T00:18:36.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332063+0000", 2024-09-18T00:18:36.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2083999999999995e-05 2024-09-18T00:18:36.001 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.001 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.001 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.001 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2228 3.15 e36/32 ondisk, result = 0)", 2024-09-18T00:18:36.001 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:18:36.002 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.714296299000001, 2024-09-18T00:18:36.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055930148999999998, 2024-09-18T00:18:36.002 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.002 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.002 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.002 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:18:36.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:18:36.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276152+0000", 2024-09-18T00:18:36.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0200000000000001e-06 2024-09-18T00:18:36.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276162+0000", 2024-09-18T00:18:36.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0767e-05 2024-09-18T00:18:36.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276202+0000", 2024-09-18T00:18:36.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000003e-05 2024-09-18T00:18:36.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276226+0000", 2024-09-18T00:18:36.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.351e-05 2024-09-18T00:18:36.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331782+0000", 2024-09-18T00:18:36.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055556296999999998 2024-09-18T00:18:36.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331801+0000", 2024-09-18T00:18:36.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9357000000000001e-05 2024-09-18T00:18:36.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332080+0000", 2024-09-18T00:18:36.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027849800000000002 2024-09-18T00:18:36.005 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.005 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.005 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.006 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2854 3.f 3:f02b0bca:::rbd_data.112aa7ba4578.0000000000000135:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:18:36.006 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:18:36.006 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.120376921, 2024-09-18T00:18:36.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.082582802999999996, 2024-09-18T00:18:36.006 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.006 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.006 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:36.006 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.006 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.006 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.006 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2854 2024-09-18T00:18:36.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.006 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:18:36.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:18:36.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870071+0000", 2024-09-18T00:18:36.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.227e-06 2024-09-18T00:18:36.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870083+0000", 2024-09-18T00:18:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1909999999999999e-05 2024-09-18T00:18:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870114+0000", 2024-09-18T00:18:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0576e-05 2024-09-18T00:18:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870129+0000", 2024-09-18T00:18:36.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.485e-05 2024-09-18T00:18:36.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870256+0000", 2024-09-18T00:18:36.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012691300000000001 2024-09-18T00:18:36.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870493+0000", 2024-09-18T00:18:36.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023737399999999999 2024-09-18T00:18:36.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:36.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.874933+0000", 2024-09-18T00:18:36.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044398299999999996 2024-09-18T00:18:36.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:36.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952452+0000", 2024-09-18T00:18:36.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.077518918000000006 2024-09-18T00:18:36.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:36.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952486+0000", 2024-09-18T00:18:36.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4035999999999997e-05 2024-09-18T00:18:36.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:36.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952575+0000", 2024-09-18T00:18:36.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.954e-05 2024-09-18T00:18:36.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952652+0000", 2024-09-18T00:18:36.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6629000000000001e-05 2024-09-18T00:18:36.011 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.011 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.011 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.012 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2854 3.f e38/32 ondisk, result = 0)", 2024-09-18T00:18:36.012 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:18:36.012 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.102846105999999, 2024-09-18T00:18:36.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064868625999999999, 2024-09-18T00:18:36.012 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.012 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.012 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.012 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:18:36.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:18:36.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887602+0000", 2024-09-18T00:18:36.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.677e-06 2024-09-18T00:18:36.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887609+0000", 2024-09-18T00:18:36.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6729999999999992e-06 2024-09-18T00:18:36.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887637+0000", 2024-09-18T00:18:36.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8206000000000001e-05 2024-09-18T00:18:36.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887651+0000", 2024-09-18T00:18:36.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3387000000000001e-05 2024-09-18T00:18:36.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952422+0000", 2024-09-18T00:18:36.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064771366999999996 2024-09-18T00:18:36.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952441+0000", 2024-09-18T00:18:36.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8737e-05 2024-09-18T00:18:36.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952469+0000", 2024-09-18T00:18:36.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7579000000000001e-05 2024-09-18T00:18:36.015 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.016 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.016 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.016 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5631 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:36.016 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297078+0000", 2024-09-18T00:18:36.016 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.693367718999999, 2024-09-18T00:18:36.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046621463000000002, 2024-09-18T00:18:36.016 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.016 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.016 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.016 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.016 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.016 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.017 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5631 2024-09-18T00:18:36.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.017 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297078+0000", 2024-09-18T00:18:36.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297078+0000", 2024-09-18T00:18:36.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297080+0000", 2024-09-18T00:18:36.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2929999999999999e-06 2024-09-18T00:18:36.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297085+0000", 2024-09-18T00:18:36.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6799999999999998e-06 2024-09-18T00:18:36.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297103+0000", 2024-09-18T00:18:36.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7479e-05 2024-09-18T00:18:36.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297116+0000", 2024-09-18T00:18:36.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2894000000000001e-05 2024-09-18T00:18:36.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343417+0000", 2024-09-18T00:18:36.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046301006999999998 2024-09-18T00:18:36.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343551+0000", 2024-09-18T00:18:36.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013466300000000001 2024-09-18T00:18:36.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343700+0000", 2024-09-18T00:18:36.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014844700000000001 2024-09-18T00:18:36.020 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.020 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.020 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.020 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5632 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:36.020 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297135+0000", 2024-09-18T00:18:36.020 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.693310607000001, 2024-09-18T00:18:36.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046771002999999998, 2024-09-18T00:18:36.021 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.021 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.021 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.021 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.021 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.021 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.021 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5632 2024-09-18T00:18:36.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.021 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297135+0000", 2024-09-18T00:18:36.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297135+0000", 2024-09-18T00:18:36.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297137+0000", 2024-09-18T00:18:36.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1999999999999999e-06 2024-09-18T00:18:36.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297141+0000", 2024-09-18T00:18:36.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8239999999999999e-06 2024-09-18T00:18:36.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297158+0000", 2024-09-18T00:18:36.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6566e-05 2024-09-18T00:18:36.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297175+0000", 2024-09-18T00:18:36.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7280000000000001e-05 2024-09-18T00:18:36.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343749+0000", 2024-09-18T00:18:36.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046574084000000002 2024-09-18T00:18:36.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343817+0000", 2024-09-18T00:18:36.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7198000000000003e-05 2024-09-18T00:18:36.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343906+0000", 2024-09-18T00:18:36.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9851000000000004e-05 2024-09-18T00:18:36.024 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.024 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.024 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.025 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5633 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:36.025 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297194+0000", 2024-09-18T00:18:36.025 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.693252276999999, 2024-09-18T00:18:36.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046979053, 2024-09-18T00:18:36.025 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.025 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.025 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.025 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.025 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.025 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.025 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5633 2024-09-18T00:18:36.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.026 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297194+0000", 2024-09-18T00:18:36.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297194+0000", 2024-09-18T00:18:36.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297195+0000", 2024-09-18T00:18:36.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2440000000000001e-06 2024-09-18T00:18:36.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297201+0000", 2024-09-18T00:18:36.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5489999999999997e-06 2024-09-18T00:18:36.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297218+0000", 2024-09-18T00:18:36.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7652999999999999e-05 2024-09-18T00:18:36.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297245+0000", 2024-09-18T00:18:36.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7212999999999998e-05 2024-09-18T00:18:36.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343955+0000", 2024-09-18T00:18:36.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046709564000000002 2024-09-18T00:18:36.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344028+0000", 2024-09-18T00:18:36.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3370999999999999e-05 2024-09-18T00:18:36.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344173+0000", 2024-09-18T00:18:36.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014445899999999999 2024-09-18T00:18:36.029 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.029 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.029 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.029 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5634 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:36.029 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297263+0000", 2024-09-18T00:18:36.029 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.693183001, 2024-09-18T00:18:36.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047153893000000002, 2024-09-18T00:18:36.030 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.030 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.030 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.030 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.030 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.030 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.030 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5634 2024-09-18T00:18:36.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.030 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297263+0000", 2024-09-18T00:18:36.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297263+0000", 2024-09-18T00:18:36.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297264+0000", 2024-09-18T00:18:36.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4169999999999999e-06 2024-09-18T00:18:36.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297270+0000", 2024-09-18T00:18:36.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4829999999999999e-06 2024-09-18T00:18:36.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297287+0000", 2024-09-18T00:18:36.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6993000000000001e-05 2024-09-18T00:18:36.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297300+0000", 2024-09-18T00:18:36.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.324e-05 2024-09-18T00:18:36.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344221+0000", 2024-09-18T00:18:36.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046921030000000002 2024-09-18T00:18:36.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344310+0000", 2024-09-18T00:18:36.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8533000000000001e-05 2024-09-18T00:18:36.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344417+0000", 2024-09-18T00:18:36.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107197 2024-09-18T00:18:36.033 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.033 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.034 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.034 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5635 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:36.034 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297317+0000", 2024-09-18T00:18:36.034 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.693129251, 2024-09-18T00:18:36.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047292222000000002, 2024-09-18T00:18:36.034 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.034 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.034 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.034 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.034 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.034 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.034 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5635 2024-09-18T00:18:36.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.035 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297317+0000", 2024-09-18T00:18:36.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297317+0000", 2024-09-18T00:18:36.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297318+0000", 2024-09-18T00:18:36.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2869999999999999e-06 2024-09-18T00:18:36.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297323+0000", 2024-09-18T00:18:36.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1760000000000001e-06 2024-09-18T00:18:36.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297343+0000", 2024-09-18T00:18:36.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9473000000000001e-05 2024-09-18T00:18:36.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297356+0000", 2024-09-18T00:18:36.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3767e-05 2024-09-18T00:18:36.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344455+0000", 2024-09-18T00:18:36.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047098220000000003 2024-09-18T00:18:36.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344518+0000", 2024-09-18T00:18:36.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2856999999999998e-05 2024-09-18T00:18:36.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344609+0000", 2024-09-18T00:18:36.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1441999999999996e-05 2024-09-18T00:18:36.038 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.038 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.038 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.038 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5672 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 9109504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:36.038 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298045+0000", 2024-09-18T00:18:36.038 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.69240091, 2024-09-18T00:18:36.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046729202999999997, 2024-09-18T00:18:36.039 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.039 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.039 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.039 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.039 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.039 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.039 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5672 2024-09-18T00:18:36.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.039 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298045+0000", 2024-09-18T00:18:36.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298045+0000", 2024-09-18T00:18:36.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298046+0000", 2024-09-18T00:18:36.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1260000000000001e-06 2024-09-18T00:18:36.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298051+0000", 2024-09-18T00:18:36.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9300000000000002e-06 2024-09-18T00:18:36.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298068+0000", 2024-09-18T00:18:36.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7153999999999999e-05 2024-09-18T00:18:36.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298082+0000", 2024-09-18T00:18:36.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3329999999999999e-05 2024-09-18T00:18:36.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344636+0000", 2024-09-18T00:18:36.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046554460999999998 2024-09-18T00:18:36.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344696+0000", 2024-09-18T00:18:36.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0034999999999997e-05 2024-09-18T00:18:36.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344774+0000", 2024-09-18T00:18:36.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8166999999999999e-05 2024-09-18T00:18:36.043 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.043 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.043 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.043 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5673 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 9895936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:36.043 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298098+0000", 2024-09-18T00:18:36.043 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.692347534, 2024-09-18T00:18:36.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046867918000000001, 2024-09-18T00:18:36.043 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.043 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.043 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.044 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.044 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.044 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.044 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5673 2024-09-18T00:18:36.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.044 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298098+0000", 2024-09-18T00:18:36.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298098+0000", 2024-09-18T00:18:36.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298100+0000", 2024-09-18T00:18:36.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0699999999999999e-06 2024-09-18T00:18:36.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298104+0000", 2024-09-18T00:18:36.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9339999999999997e-06 2024-09-18T00:18:36.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298121+0000", 2024-09-18T00:18:36.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6656000000000001e-05 2024-09-18T00:18:36.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298149+0000", 2024-09-18T00:18:36.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8243e-05 2024-09-18T00:18:36.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344801+0000", 2024-09-18T00:18:36.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046651199999999997 2024-09-18T00:18:36.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344860+0000", 2024-09-18T00:18:36.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9537999999999998e-05 2024-09-18T00:18:36.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344966+0000", 2024-09-18T00:18:36.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106277 2024-09-18T00:18:36.047 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.047 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.047 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.048 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5674 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 10682368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:36.048 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298168+0000", 2024-09-18T00:18:36.048 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.692278448, 2024-09-18T00:18:36.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047098783999999998, 2024-09-18T00:18:36.048 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.048 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.048 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.048 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.048 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.048 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.048 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5674 2024-09-18T00:18:36.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.048 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298168+0000", 2024-09-18T00:18:36.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298168+0000", 2024-09-18T00:18:36.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298169+0000", 2024-09-18T00:18:36.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1370000000000001e-06 2024-09-18T00:18:36.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298174+0000", 2024-09-18T00:18:36.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6500000000000001e-06 2024-09-18T00:18:36.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298192+0000", 2024-09-18T00:18:36.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7620000000000001e-05 2024-09-18T00:18:36.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298205+0000", 2024-09-18T00:18:36.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3363000000000001e-05 2024-09-18T00:18:36.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345013+0000", 2024-09-18T00:18:36.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046807418000000003 2024-09-18T00:18:36.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345125+0000", 2024-09-18T00:18:36.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111901 2024-09-18T00:18:36.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345266+0000", 2024-09-18T00:18:36.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000141695 2024-09-18T00:18:36.052 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.052 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.052 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.052 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5675 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:36.052 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298223+0000", 2024-09-18T00:18:36.052 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.692222555000001, 2024-09-18T00:18:36.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047324643999999999, 2024-09-18T00:18:36.052 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.052 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.052 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.053 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.053 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.053 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.053 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5675 2024-09-18T00:18:36.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.053 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298223+0000", 2024-09-18T00:18:36.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298223+0000", 2024-09-18T00:18:36.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298225+0000", 2024-09-18T00:18:36.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.13e-06 2024-09-18T00:18:36.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298230+0000", 2024-09-18T00:18:36.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1599999999999997e-06 2024-09-18T00:18:36.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298247+0000", 2024-09-18T00:18:36.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6813999999999999e-05 2024-09-18T00:18:36.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298260+0000", 2024-09-18T00:18:36.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3036e-05 2024-09-18T00:18:36.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345315+0000", 2024-09-18T00:18:36.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047055420000000001 2024-09-18T00:18:36.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345384+0000", 2024-09-18T00:18:36.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9375999999999994e-05 2024-09-18T00:18:36.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345548+0000", 2024-09-18T00:18:36.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016370800000000001 2024-09-18T00:18:36.056 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.056 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.056 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.057 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5676 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 12255232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:36.057 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:18:36.057 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.692170249, 2024-09-18T00:18:36.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047571816000000003, 2024-09-18T00:18:36.057 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.057 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.057 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.057 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.057 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.057 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.057 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5676 2024-09-18T00:18:36.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.058 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:18:36.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:18:36.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298277+0000", 2024-09-18T00:18:36.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.454e-06 2024-09-18T00:18:36.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298282+0000", 2024-09-18T00:18:36.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6299999999999997e-06 2024-09-18T00:18:36.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298299+0000", 2024-09-18T00:18:36.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6959000000000001e-05 2024-09-18T00:18:36.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298313+0000", 2024-09-18T00:18:36.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.376e-05 2024-09-18T00:18:36.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345610+0000", 2024-09-18T00:18:36.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047297128000000001 2024-09-18T00:18:36.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345708+0000", 2024-09-18T00:18:36.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8245999999999994e-05 2024-09-18T00:18:36.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345848+0000", 2024-09-18T00:18:36.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000139639 2024-09-18T00:18:36.061 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.061 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.061 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.061 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5677 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 12779520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:36.061 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:18:36.061 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.692116735999999, 2024-09-18T00:18:36.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047810679000000002, 2024-09-18T00:18:36.062 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.062 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.062 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.062 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.062 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.062 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.062 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5677 2024-09-18T00:18:36.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.062 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:18:36.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:18:36.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298330+0000", 2024-09-18T00:18:36.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.217e-06 2024-09-18T00:18:36.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298354+0000", 2024-09-18T00:18:36.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3753000000000001e-05 2024-09-18T00:18:36.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298388+0000", 2024-09-18T00:18:36.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4276999999999997e-05 2024-09-18T00:18:36.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298415+0000", 2024-09-18T00:18:36.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6403000000000001e-05 2024-09-18T00:18:36.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345896+0000", 2024-09-18T00:18:36.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047481007999999998 2024-09-18T00:18:36.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345992+0000", 2024-09-18T00:18:36.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6141e-05 2024-09-18T00:18:36.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346140+0000", 2024-09-18T00:18:36.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014788 2024-09-18T00:18:36.066 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.066 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.066 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.066 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5678 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:36.066 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:18:36.066 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.692010557, 2024-09-18T00:18:36.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047985466999999997, 2024-09-18T00:18:36.066 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.066 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.066 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.067 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5678 2024-09-18T00:18:36.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.067 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:18:36.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:18:36.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298437+0000", 2024-09-18T00:18:36.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1799999999999999e-06 2024-09-18T00:18:36.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298443+0000", 2024-09-18T00:18:36.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1140000000000002e-06 2024-09-18T00:18:36.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298460+0000", 2024-09-18T00:18:36.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7615999999999999e-05 2024-09-18T00:18:36.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298474+0000", 2024-09-18T00:18:36.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4017e-05 2024-09-18T00:18:36.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346180+0000", 2024-09-18T00:18:36.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047705801999999999 2024-09-18T00:18:36.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346274+0000", 2024-09-18T00:18:36.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3814999999999995e-05 2024-09-18T00:18:36.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346421+0000", 2024-09-18T00:18:36.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000146923 2024-09-18T00:18:36.070 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.070 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.070 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.071 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5679 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:36.071 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:18:36.071 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.691951512999999, 2024-09-18T00:18:36.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048190771, 2024-09-18T00:18:36.071 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.071 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.071 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.071 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5679 2024-09-18T00:18:36.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.071 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:18:36.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:18:36.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298496+0000", 2024-09-18T00:18:36.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4699999999999999e-06 2024-09-18T00:18:36.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298501+0000", 2024-09-18T00:18:36.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:18:36.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298519+0000", 2024-09-18T00:18:36.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7329e-05 2024-09-18T00:18:36.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298532+0000", 2024-09-18T00:18:36.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3083e-05 2024-09-18T00:18:36.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346462+0000", 2024-09-18T00:18:36.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047930449999999999 2024-09-18T00:18:36.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346556+0000", 2024-09-18T00:18:36.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3222999999999998e-05 2024-09-18T00:18:36.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346685+0000", 2024-09-18T00:18:36.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000129696 2024-09-18T00:18:36.075 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.075 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.075 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.075 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5680 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:36.076 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:18:36.076 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.691894103999999, 2024-09-18T00:18:36.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048535847, 2024-09-18T00:18:36.076 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.076 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.076 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.076 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5680 2024-09-18T00:18:36.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.076 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:18:36.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:18:36.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298553+0000", 2024-09-18T00:18:36.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0759999999999999e-06 2024-09-18T00:18:36.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298575+0000", 2024-09-18T00:18:36.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2019999999999999e-05 2024-09-18T00:18:36.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298593+0000", 2024-09-18T00:18:36.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7900000000000001e-05 2024-09-18T00:18:36.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298607+0000", 2024-09-18T00:18:36.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3844e-05 2024-09-18T00:18:36.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346791+0000", 2024-09-18T00:18:36.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048184449999999997 2024-09-18T00:18:36.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346921+0000", 2024-09-18T00:18:36.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012994699999999999 2024-09-18T00:18:36.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347088+0000", 2024-09-18T00:18:36.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016661000000000001 2024-09-18T00:18:36.080 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.080 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.080 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.080 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5681 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:36.080 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:18:36.080 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.691821694, 2024-09-18T00:18:36.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048743337999999997, 2024-09-18T00:18:36.080 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.081 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.081 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.081 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.081 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.081 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.081 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5681 2024-09-18T00:18:36.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.081 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:18:36.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:18:36.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298626+0000", 2024-09-18T00:18:36.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4899999999999999e-06 2024-09-18T00:18:36.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298631+0000", 2024-09-18T00:18:36.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0300000000000001e-06 2024-09-18T00:18:36.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298650+0000", 2024-09-18T00:18:36.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9499000000000001e-05 2024-09-18T00:18:36.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298664+0000", 2024-09-18T00:18:36.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3767e-05 2024-09-18T00:18:36.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347139+0000", 2024-09-18T00:18:36.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048475348000000001 2024-09-18T00:18:36.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347228+0000", 2024-09-18T00:18:36.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8843000000000003e-05 2024-09-18T00:18:36.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347368+0000", 2024-09-18T00:18:36.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013936100000000001 2024-09-18T00:18:36.084 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.085 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.085 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.085 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5682 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:36.085 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:18:36.085 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.691764879000001, 2024-09-18T00:18:36.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048916864999999997, 2024-09-18T00:18:36.085 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:36.085 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:36.085 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:36.085 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:36.085 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:36.085 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.086 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5682 2024-09-18T00:18:36.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.086 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:36.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:36.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:18:36.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:36.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:18:36.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:36.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:36.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298682+0000", 2024-09-18T00:18:36.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:18:36.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:36.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298687+0000", 2024-09-18T00:18:36.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9239999999999998e-06 2024-09-18T00:18:36.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:36.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298704+0000", 2024-09-18T00:18:36.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6569999999999999e-05 2024-09-18T00:18:36.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298731+0000", 2024-09-18T00:18:36.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6752999999999999e-05 2024-09-18T00:18:36.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:36.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347416+0000", 2024-09-18T00:18:36.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048685056999999997 2024-09-18T00:18:36.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:36.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347506+0000", 2024-09-18T00:18:36.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9936000000000001e-05 2024-09-18T00:18:36.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:36.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:36.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:36.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347598+0000", 2024-09-18T00:18:36.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2335000000000003e-05 2024-09-18T00:18:36.089 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.089 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.089 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.089 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:36.089 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:36.089 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:36.090 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:36.086+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:36.091 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:18:36.192 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:36.186+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:36.214 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:36.214 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:36.214 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:36.215 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:36.224 INFO:teuthology.orchestra.run.smithi097.stdout: marking collection for removal 2024-09-18T00:18:36.224 INFO:teuthology.orchestra.run.smithi097.stdout:setting '_remove' omap key 2024-09-18T00:18:36.224 INFO:teuthology.orchestra.run.smithi097.stdout:finish_remove_pgs 3.8_head removing 3.8 2024-09-18T00:18:36.231 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:18:36.233 INFO:teuthology.orchestra.run.smithi097.stdout:Remove successful 2024-09-18T00:18:36.288 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:36.289 INFO:teuthology.orchestra.run.smithi097.stdout:2115 write 0x3d6f8e5 thru 0x3d79550 (0x9c6c bytes) 2024-09-18T00:18:36.293 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:36.286+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:36.314 INFO:teuthology.orchestra.run.smithi097.stdout:2116 trunc from 0xe868097 to 0x6da60e1 2024-09-18T00:18:36.332 INFO:teuthology.orchestra.run.smithi097.stdout:2117 write 0x571f98c thru 0x572a228 (0xa89d bytes) 2024-09-18T00:18:36.354 INFO:teuthology.orchestra.run.smithi097.stdout:2119 write 0xa01e531 thru 0xa023c39 (0x5709 bytes) 2024-09-18T00:18:36.359 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:36.359 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:36.359 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:36.360 INFO:teuthology.orchestra.run.smithi097.stdout:2120 read 0x1d9d8ce thru 0x1daa173 (0xc8a6 bytes) 2024-09-18T00:18:36.373 INFO:teuthology.orchestra.run.smithi097.stdout:2121 trunc from 0xa023c3a to 0x713fed 2024-09-18T00:18:36.382 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:18:36.396 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:36.390+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:36.457 INFO:teuthology.orchestra.run.smithi097.stdout:2122 punch from 0x21a54f to 0x21d01a, (0x2acb bytes) 2024-09-18T00:18:36.457 INFO:teuthology.orchestra.run.smithi097.stdout:2123 read 0x624753 thru 0x6253a8 (0xc56 bytes) 2024-09-18T00:18:36.457 INFO:teuthology.orchestra.run.smithi097.stdout:2124 read 0x315202 thru 0x31fc5d (0xaa5c bytes) 2024-09-18T00:18:36.467 INFO:teuthology.orchestra.run.smithi097.stdout:2125 write 0x24398f2 thru 0x243c5a0 (0x2caf bytes) 2024-09-18T00:18:36.477 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:18:36.482 INFO:teuthology.orchestra.run.smithi097.stdout:2126 trunc from 0x243c5a1 to 0x91ba524 2024-09-18T00:18:36.490 INFO:teuthology.orchestra.run.smithi097.stdout:2127 write 0x80b3ba3 thru 0x80b5e2c (0x228a bytes) 2024-09-18T00:18:36.495 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:36.490+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:36.496 INFO:teuthology.orchestra.run.smithi097.stdout:2133 punch from 0x4993c75 to 0x49946f0, (0xa7b bytes) 2024-09-18T00:18:36.496 INFO:teuthology.orchestra.run.smithi097.stdout:2134 writesame 0x19434cd thru 0x194b5f8 data_size 0xe(0x812c bytes) 2024-09-18T00:18:36.514 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:36.514 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:36.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:36.514 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:36.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.515 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2230 3.17 3:ef492888:::rbd_data.112aa7ba4578.000000000000014f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:18:36.515 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:36.515 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 16.247214722999999, 2024-09-18T00:18:36.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075065283999999996, 2024-09-18T00:18:36.515 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.515 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.515 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:36.515 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:36.515 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:36.515 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.515 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2230 2024-09-18T00:18:36.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.515 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:36.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:36.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266014+0000", 2024-09-18T00:18:36.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:18:36.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266029+0000", 2024-09-18T00:18:36.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4616e-05 2024-09-18T00:18:36.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266053+0000", 2024-09-18T00:18:36.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4369999999999999e-05 2024-09-18T00:18:36.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266067+0000", 2024-09-18T00:18:36.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4243999999999999e-05 2024-09-18T00:18:36.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266157+0000", 2024-09-18T00:18:36.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9909999999999998e-05 2024-09-18T00:18:36.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266327+0000", 2024-09-18T00:18:36.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016981200000000001 2024-09-18T00:18:36.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:36.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269180+0000", 2024-09-18T00:18:36.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028526559999999999 2024-09-18T00:18:36.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:36.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340955+0000", 2024-09-18T00:18:36.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.071774904 2024-09-18T00:18:36.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:36.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340974+0000", 2024-09-18T00:18:36.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8824000000000001e-05 2024-09-18T00:18:36.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341028+0000", 2024-09-18T00:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4456999999999997e-05 2024-09-18T00:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341076+0000", 2024-09-18T00:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8284000000000001e-05 2024-09-18T00:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2230 3.17 e36/23 ondisk, result = 0)", 2024-09-18T00:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:36.520 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 16.229871142, 2024-09-18T00:18:36.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057609645000000001, 2024-09-18T00:18:36.521 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.521 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.521 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:36.521 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:36.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:36.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283356+0000", 2024-09-18T00:18:36.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.44e-06 2024-09-18T00:18:36.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283362+0000", 2024-09-18T00:18:36.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2539999999999998e-06 2024-09-18T00:18:36.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283382+0000", 2024-09-18T00:18:36.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9896000000000001e-05 2024-09-18T00:18:36.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283398+0000", 2024-09-18T00:18:36.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5489999999999999e-05 2024-09-18T00:18:36.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340936+0000", 2024-09-18T00:18:36.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057538649999999997 2024-09-18T00:18:36.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340947+0000", 2024-09-18T00:18:36.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0981e-05 2024-09-18T00:18:36.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340964+0000", 2024-09-18T00:18:36.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6934000000000001e-05 2024-09-18T00:18:36.524 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.524 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.524 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.524 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2419 3.17 3:ed94b133:::rbd_data.112aa7ba4578.00000000000001b5:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:36.524 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:18:36.524 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.775749669, 2024-09-18T00:18:36.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.064761812000000002, 2024-09-18T00:18:36.524 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.525 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.525 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:36.525 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:36.525 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:36.525 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.525 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2419 2024-09-18T00:18:36.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.525 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:18:36.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:18:36.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737479+0000", 2024-09-18T00:18:36.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6089999999999999e-06 2024-09-18T00:18:36.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737487+0000", 2024-09-18T00:18:36.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7900000000000005e-06 2024-09-18T00:18:36.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737507+0000", 2024-09-18T00:18:36.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9936e-05 2024-09-18T00:18:36.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737518+0000", 2024-09-18T00:18:36.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1046999999999999e-05 2024-09-18T00:18:36.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737599+0000", 2024-09-18T00:18:36.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0697000000000003e-05 2024-09-18T00:18:36.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737697+0000", 2024-09-18T00:18:36.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8058000000000002e-05 2024-09-18T00:18:36.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:36.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.740445+0000", 2024-09-18T00:18:36.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027481279999999999 2024-09-18T00:18:36.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:36.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802023+0000", 2024-09-18T00:18:36.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.061577321999999997 2024-09-18T00:18:36.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:36.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802060+0000", 2024-09-18T00:18:36.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7586000000000002e-05 2024-09-18T00:18:36.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:36.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802158+0000", 2024-09-18T00:18:36.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7583000000000004e-05 2024-09-18T00:18:36.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802238+0000", 2024-09-18T00:18:36.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0055999999999994e-05 2024-09-18T00:18:36.530 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.530 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.530 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.530 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2419 3.17 e37/23 ondisk, result = 0)", 2024-09-18T00:18:36.530 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:18:36.530 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.763524723, 2024-09-18T00:18:36.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052342948, 2024-09-18T00:18:36.530 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.530 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.530 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:36.531 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:18:36.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:18:36.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749702+0000", 2024-09-18T00:18:36.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:18:36.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749710+0000", 2024-09-18T00:18:36.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7279999999999995e-06 2024-09-18T00:18:36.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749736+0000", 2024-09-18T00:18:36.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5797999999999999e-05 2024-09-18T00:18:36.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749751+0000", 2024-09-18T00:18:36.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5238999999999999e-05 2024-09-18T00:18:36.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801745+0000", 2024-09-18T00:18:36.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051994022000000001 2024-09-18T00:18:36.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801770+0000", 2024-09-18T00:18:36.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5103e-05 2024-09-18T00:18:36.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802044+0000", 2024-09-18T00:18:36.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027395799999999998 2024-09-18T00:18:36.534 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.534 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.534 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.534 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6503 3.1b 3:d9387a1f:::rbd_data.112a3a39fc45.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:36.534 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:18:36.534 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.948851753, 2024-09-18T00:18:36.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050510547000000003, 2024-09-18T00:18:36.534 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.535 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.535 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:36.535 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:36.535 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:36.535 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.535 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6503 2024-09-18T00:18:36.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.535 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:18:36.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:18:36.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564376+0000", 2024-09-18T00:18:36.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-06 2024-09-18T00:18:36.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564386+0000", 2024-09-18T00:18:36.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0312999999999999e-05 2024-09-18T00:18:36.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564408+0000", 2024-09-18T00:18:36.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2001000000000002e-05 2024-09-18T00:18:36.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564421+0000", 2024-09-18T00:18:36.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2271e-05 2024-09-18T00:18:36.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564491+0000", 2024-09-18T00:18:36.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0630000000000006e-05 2024-09-18T00:18:36.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564606+0000", 2024-09-18T00:18:36.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011449 2024-09-18T00:18:36.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:36.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.566743+0000", 2024-09-18T00:18:36.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021373830000000001 2024-09-18T00:18:36.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:36.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614738+0000", 2024-09-18T00:18:36.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047995359000000001 2024-09-18T00:18:36.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:36.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614759+0000", 2024-09-18T00:18:36.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1010999999999999e-05 2024-09-18T00:18:36.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:36.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614826+0000", 2024-09-18T00:18:36.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6654000000000003e-05 2024-09-18T00:18:36.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614884+0000", 2024-09-18T00:18:36.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8332999999999999e-05 2024-09-18T00:18:36.540 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.540 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.540 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.540 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6503 3.1b e42/23 ondisk, result = 0)", 2024-09-18T00:18:36.540 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:18:36.540 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.939592285, 2024-09-18T00:18:36.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041115407, 2024-09-18T00:18:36.540 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.540 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.540 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:36.540 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:18:36.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:18:36.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573635+0000", 2024-09-18T00:18:36.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5689999999999999e-06 2024-09-18T00:18:36.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573643+0000", 2024-09-18T00:18:36.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9699999999999999e-06 2024-09-18T00:18:36.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573668+0000", 2024-09-18T00:18:36.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5536000000000001e-05 2024-09-18T00:18:36.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573683+0000", 2024-09-18T00:18:36.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4025e-05 2024-09-18T00:18:36.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614717+0000", 2024-09-18T00:18:36.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041034517 2024-09-18T00:18:36.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614730+0000", 2024-09-18T00:18:36.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2945999999999999e-05 2024-09-18T00:18:36.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614749+0000", 2024-09-18T00:18:36.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8844000000000001e-05 2024-09-18T00:18:36.544 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.544 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.544 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.544 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6518 3.1 3:8061bf9f:::rbd_data.112a3a39fc45.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:36.544 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:18:36.544 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.862413479000001, 2024-09-18T00:18:36.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062303123000000002, 2024-09-18T00:18:36.544 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.545 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.545 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:36.545 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:36.545 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:36.545 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.545 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6518 2024-09-18T00:18:36.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.545 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:18:36.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:18:36.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650814+0000", 2024-09-18T00:18:36.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.629e-06 2024-09-18T00:18:36.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650823+0000", 2024-09-18T00:18:36.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1800000000000002e-06 2024-09-18T00:18:36.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650843+0000", 2024-09-18T00:18:36.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9836000000000001e-05 2024-09-18T00:18:36.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650854+0000", 2024-09-18T00:18:36.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1564e-05 2024-09-18T00:18:36.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650917+0000", 2024-09-18T00:18:36.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3076000000000004e-05 2024-09-18T00:18:36.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.651029+0000", 2024-09-18T00:18:36.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111154 2024-09-18T00:18:36.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:36.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.655531+0000", 2024-09-18T00:18:36.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045027629999999999 2024-09-18T00:18:36.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:36.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712924+0000", 2024-09-18T00:18:36.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057393006000000003 2024-09-18T00:18:36.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:36.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712988+0000", 2024-09-18T00:18:36.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000005e-05 2024-09-18T00:18:36.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:36.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713056+0000", 2024-09-18T00:18:36.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7904999999999994e-05 2024-09-18T00:18:36.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713115+0000", 2024-09-18T00:18:36.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9339999999999998e-05 2024-09-18T00:18:36.550 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.550 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.550 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.550 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6518 3.1 e42/23 ondisk, result = 0)", 2024-09-18T00:18:36.550 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:18:36.550 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.849988396000001, 2024-09-18T00:18:36.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049887304, 2024-09-18T00:18:36.550 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.550 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.550 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:36.551 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:18:36.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:18:36.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663239+0000", 2024-09-18T00:18:36.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:18:36.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663246+0000", 2024-09-18T00:18:36.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4189999999999996e-06 2024-09-18T00:18:36.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663265+0000", 2024-09-18T00:18:36.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8323000000000001e-05 2024-09-18T00:18:36.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663277+0000", 2024-09-18T00:18:36.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2065999999999999e-05 2024-09-18T00:18:36.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712965+0000", 2024-09-18T00:18:36.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049688317000000003 2024-09-18T00:18:36.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712977+0000", 2024-09-18T00:18:36.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.206e-05 2024-09-18T00:18:36.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713125+0000", 2024-09-18T00:18:36.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014749899999999999 2024-09-18T00:18:36.554 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.554 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.554 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.554 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:8831 3.12 3:49cac8cf:::rbd_data.112a9f4259df.0000000000000130:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T00:18:36.554 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:18:36.554 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2321281470000001, 2024-09-18T00:18:36.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104125047, 2024-09-18T00:18:36.554 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.554 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.554 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:36.554 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:36.555 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:36.555 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.555 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 8831 2024-09-18T00:18:36.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.555 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:18:36.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:18:36.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281099+0000", 2024-09-18T00:18:36.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.029e-06 2024-09-18T00:18:36.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281103+0000", 2024-09-18T00:18:36.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7759999999999997e-06 2024-09-18T00:18:36.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281114+0000", 2024-09-18T00:18:36.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1090000000000001e-05 2024-09-18T00:18:36.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281120+0000", 2024-09-18T00:18:36.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.604e-06 2024-09-18T00:18:36.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281164+0000", 2024-09-18T00:18:36.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3696999999999999e-05 2024-09-18T00:18:36.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281270+0000", 2024-09-18T00:18:36.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106041 2024-09-18T00:18:36.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:36.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.285596+0000", 2024-09-18T00:18:36.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043266659999999998 2024-09-18T00:18:36.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:36.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385028+0000", 2024-09-18T00:18:36.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.099431808999999996 2024-09-18T00:18:36.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:36.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385094+0000", 2024-09-18T00:18:36.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5740000000000004e-05 2024-09-18T00:18:36.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:36.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385166+0000", 2024-09-18T00:18:36.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1637000000000005e-05 2024-09-18T00:18:36.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385223+0000", 2024-09-18T00:18:36.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6957999999999999e-05 2024-09-18T00:18:36.560 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.560 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.560 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.560 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:8831 3.12 e48/32 ondisk, result = 0)", 2024-09-18T00:18:36.560 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:18:36.560 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2156140369999999, 2024-09-18T00:18:36.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087621558000000002, 2024-09-18T00:18:36.560 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.560 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.560 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:36.560 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:18:36.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:18:36.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297613+0000", 2024-09-18T00:18:36.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3400000000000001e-06 2024-09-18T00:18:36.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297620+0000", 2024-09-18T00:18:36.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5869999999999998e-06 2024-09-18T00:18:36.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297639+0000", 2024-09-18T00:18:36.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9888999999999999e-05 2024-09-18T00:18:36.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297661+0000", 2024-09-18T00:18:36.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1772000000000001e-05 2024-09-18T00:18:36.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385073+0000", 2024-09-18T00:18:36.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087411303999999995 2024-09-18T00:18:36.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385086+0000", 2024-09-18T00:18:36.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3040000000000001e-05 2024-09-18T00:18:36.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385233+0000", 2024-09-18T00:18:36.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014762600000000001 2024-09-18T00:18:36.564 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.564 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.564 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.564 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10099 3.6 3:62e7f20b:::rbd_data.112a9f4259df.0000000000000001:8 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:36.564 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.572938+0000", 2024-09-18T00:18:36.564 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.9402873970000001, 2024-09-18T00:18:36.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026138630999999999, 2024-09-18T00:18:36.564 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.564 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.564 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:36.565 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:36.565 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:36.565 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.565 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10099 2024-09-18T00:18:36.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.565 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572938+0000", 2024-09-18T00:18:36.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572938+0000", 2024-09-18T00:18:36.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572939+0000", 2024-09-18T00:18:36.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7400000000000003e-07 2024-09-18T00:18:36.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572942+0000", 2024-09-18T00:18:36.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7159999999999999e-06 2024-09-18T00:18:36.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572949+0000", 2024-09-18T00:18:36.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7379999999999994e-06 2024-09-18T00:18:36.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572955+0000", 2024-09-18T00:18:36.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7860000000000002e-06 2024-09-18T00:18:36.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.598855+0000", 2024-09-18T00:18:36.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025900175000000001 2024-09-18T00:18:36.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.598939+0000", 2024-09-18T00:18:36.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3244000000000005e-05 2024-09-18T00:18:36.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599077+0000", 2024-09-18T00:18:36.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138398 2024-09-18T00:18:36.568 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.568 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.568 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.568 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10108 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:36.569 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.572963+0000", 2024-09-18T00:18:36.569 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.940262954, 2024-09-18T00:18:36.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026310093999999999, 2024-09-18T00:18:36.569 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.569 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.569 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:36.569 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:36.569 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:36.569 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.569 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10108 2024-09-18T00:18:36.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.569 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572963+0000", 2024-09-18T00:18:36.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572963+0000", 2024-09-18T00:18:36.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572963+0000", 2024-09-18T00:18:36.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7299999999999996e-07 2024-09-18T00:18:36.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572966+0000", 2024-09-18T00:18:36.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4389999999999999e-06 2024-09-18T00:18:36.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572973+0000", 2024-09-18T00:18:36.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6520000000000006e-06 2024-09-18T00:18:36.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572980+0000", 2024-09-18T00:18:36.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:18:36.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599154+0000", 2024-09-18T00:18:36.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026174096000000001 2024-09-18T00:18:36.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599196+0000", 2024-09-18T00:18:36.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1993000000000003e-05 2024-09-18T00:18:36.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599273+0000", 2024-09-18T00:18:36.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7144999999999994e-05 2024-09-18T00:18:36.573 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.573 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.573 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.573 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10109 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:36.573 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:18:36.573 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.9402384769999999, 2024-09-18T00:18:36.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026513858000000001, 2024-09-18T00:18:36.573 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.573 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.574 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:36.574 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:36.574 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:36.574 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.574 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10109 2024-09-18T00:18:36.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.574 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:18:36.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:18:36.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572988+0000", 2024-09-18T00:18:36.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1099999999999996e-07 2024-09-18T00:18:36.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572990+0000", 2024-09-18T00:18:36.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3329999999999999e-06 2024-09-18T00:18:36.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572998+0000", 2024-09-18T00:18:36.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2039999999999997e-06 2024-09-18T00:18:36.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573012+0000", 2024-09-18T00:18:36.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3879e-05 2024-09-18T00:18:36.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599312+0000", 2024-09-18T00:18:36.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026300093 2024-09-18T00:18:36.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599380+0000", 2024-09-18T00:18:36.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7401000000000001e-05 2024-09-18T00:18:36.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599501+0000", 2024-09-18T00:18:36.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000121437 2024-09-18T00:18:36.577 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.577 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.577 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.577 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10110 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:36.577 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:18:36.578 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.9402050179999999, 2024-09-18T00:18:36.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026746066999999998, 2024-09-18T00:18:36.578 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.578 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.578 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:36.578 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:36.578 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:36.578 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.578 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10110 2024-09-18T00:18:36.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.578 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:18:36.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:18:36.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:18:36.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2799999999999996e-07 2024-09-18T00:18:36.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573030+0000", 2024-09-18T00:18:36.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5469999999999997e-06 2024-09-18T00:18:36.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573039+0000", 2024-09-18T00:18:36.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6080000000000002e-06 2024-09-18T00:18:36.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573049+0000", 2024-09-18T00:18:36.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0759999999999992e-06 2024-09-18T00:18:36.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599539+0000", 2024-09-18T00:18:36.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026490952000000002 2024-09-18T00:18:36.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599621+0000", 2024-09-18T00:18:36.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1601999999999996e-05 2024-09-18T00:18:36.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599767+0000", 2024-09-18T00:18:36.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145654 2024-09-18T00:18:36.581 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10111 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:18:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.9401668029999999, 2024-09-18T00:18:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027006651999999999, 2024-09-18T00:18:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:36.582 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.583 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10111 2024-09-18T00:18:36.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.583 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:18:36.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:18:36.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:18:36.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9500000000000002e-07 2024-09-18T00:18:36.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573062+0000", 2024-09-18T00:18:36.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6539999999999999e-06 2024-09-18T00:18:36.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573071+0000", 2024-09-18T00:18:36.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5380000000000002e-06 2024-09-18T00:18:36.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573077+0000", 2024-09-18T00:18:36.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999999e-06 2024-09-18T00:18:36.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599818+0000", 2024-09-18T00:18:36.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02674093 2024-09-18T00:18:36.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599923+0000", 2024-09-18T00:18:36.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010490200000000001 2024-09-18T00:18:36.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600066+0000", 2024-09-18T00:18:36.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014284500000000001 2024-09-18T00:18:36.586 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.586 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.586 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.586 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10112 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:36.586 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:18:36.586 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.940141009, 2024-09-18T00:18:36.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027148835, 2024-09-18T00:18:36.587 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.587 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.587 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:36.587 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:36.587 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:36.587 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.587 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10112 2024-09-18T00:18:36.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.587 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:18:36.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:18:36.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:18:36.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9500000000000003e-07 2024-09-18T00:18:36.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573088+0000", 2024-09-18T00:18:36.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9569999999999998e-06 2024-09-18T00:18:36.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573097+0000", 2024-09-18T00:18:36.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8599999999999999e-06 2024-09-18T00:18:36.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573104+0000", 2024-09-18T00:18:36.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5710000000000003e-06 2024-09-18T00:18:36.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600091+0000", 2024-09-18T00:18:36.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026987315000000001 2024-09-18T00:18:36.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600158+0000", 2024-09-18T00:18:36.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6727e-05 2024-09-18T00:18:36.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600234+0000", 2024-09-18T00:18:36.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5909999999999997e-05 2024-09-18T00:18:36.590 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.590 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.590 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.591 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10113 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:36.591 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:18:36.591 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.940096751, 2024-09-18T00:18:36.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027356020000000002, 2024-09-18T00:18:36.591 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.591 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.591 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:36.591 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:36.591 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:36.591 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.591 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10113 2024-09-18T00:18:36.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.592 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:18:36.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:18:36.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573130+0000", 2024-09-18T00:18:36.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1999999999999999e-07 2024-09-18T00:18:36.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573133+0000", 2024-09-18T00:18:36.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:18:36.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573141+0000", 2024-09-18T00:18:36.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5110000000000003e-06 2024-09-18T00:18:36.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573155+0000", 2024-09-18T00:18:36.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3730000000000001e-05 2024-09-18T00:18:36.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600275+0000", 2024-09-18T00:18:36.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027119621999999999 2024-09-18T00:18:36.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600349+0000", 2024-09-18T00:18:36.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4624999999999998e-05 2024-09-18T00:18:36.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600485+0000", 2024-09-18T00:18:36.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013570499999999999 2024-09-18T00:18:36.595 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.595 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.595 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.595 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10114 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:36.595 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:18:36.595 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.9400613880000002, 2024-09-18T00:18:36.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027471173000000002, 2024-09-18T00:18:36.595 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.596 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.596 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:36.596 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:36.596 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:36.596 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.596 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10114 2024-09-18T00:18:36.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.596 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:18:36.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:18:36.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573165+0000", 2024-09-18T00:18:36.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.99e-07 2024-09-18T00:18:36.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573168+0000", 2024-09-18T00:18:36.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8339999999999999e-06 2024-09-18T00:18:36.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573176+0000", 2024-09-18T00:18:36.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0199999999999994e-06 2024-09-18T00:18:36.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573182+0000", 2024-09-18T00:18:36.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.455e-06 2024-09-18T00:18:36.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600521+0000", 2024-09-18T00:18:36.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027338989000000001 2024-09-18T00:18:36.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600577+0000", 2024-09-18T00:18:36.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5717000000000002e-05 2024-09-18T00:18:36.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600635+0000", 2024-09-18T00:18:36.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8559e-05 2024-09-18T00:18:36.599 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.599 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.599 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.600 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10115 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:36.600 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:18:36.600 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.9400085110000003, 2024-09-18T00:18:36.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027643300999999999, 2024-09-18T00:18:36.600 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.600 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.600 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:36.600 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:36.600 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:36.600 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.600 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10115 2024-09-18T00:18:36.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.601 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:18:36.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:18:36.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573218+0000", 2024-09-18T00:18:36.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-18T00:18:36.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573221+0000", 2024-09-18T00:18:36.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3079999999999999e-06 2024-09-18T00:18:36.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573233+0000", 2024-09-18T00:18:36.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1544e-05 2024-09-18T00:18:36.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573240+0000", 2024-09-18T00:18:36.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4070000000000004e-06 2024-09-18T00:18:36.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600665+0000", 2024-09-18T00:18:36.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02742522 2024-09-18T00:18:36.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600745+0000", 2024-09-18T00:18:36.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0014999999999999e-05 2024-09-18T00:18:36.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600860+0000", 2024-09-18T00:18:36.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115127 2024-09-18T00:18:36.604 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.604 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.604 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.604 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11459 3.1b 3:db862ee8:::rbd_data.112ab1fa102a.0000000000000152:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:36.604 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:18:36.604 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.2515797, 2024-09-18T00:18:36.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.032246431999999998, 2024-09-18T00:18:36.605 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:36.605 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:36.605 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:36.605 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:36.605 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:36.605 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:36.605 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11459 2024-09-18T00:18:36.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.605 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:36.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:36.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:18:36.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:36.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:18:36.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:36.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:36.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261649+0000", 2024-09-18T00:18:36.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1200000000000002e-06 2024-09-18T00:18:36.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:36.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261665+0000", 2024-09-18T00:18:36.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5668999999999999e-05 2024-09-18T00:18:36.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:36.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261700+0000", 2024-09-18T00:18:36.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4925000000000002e-05 2024-09-18T00:18:36.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:36.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261723+0000", 2024-09-18T00:18:36.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3189000000000001e-05 2024-09-18T00:18:36.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:36.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261815+0000", 2024-09-18T00:18:36.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1662000000000004e-05 2024-09-18T00:18:36.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:36.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261904+0000", 2024-09-18T00:18:36.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9201000000000002e-05 2024-09-18T00:18:36.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:36.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.262955+0000", 2024-09-18T00:18:36.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00105156 2024-09-18T00:18:36.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:36.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293760+0000", 2024-09-18T00:18:36.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030804292 2024-09-18T00:18:36.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:36.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293773+0000", 2024-09-18T00:18:36.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3361e-05 2024-09-18T00:18:36.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:36.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293833+0000", 2024-09-18T00:18:36.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9626000000000001e-05 2024-09-18T00:18:36.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:36.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:36.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:36.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293892+0000", 2024-09-18T00:18:36.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9827000000000002e-05 2024-09-18T00:18:36.610 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.610 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.610 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.610 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:36.610 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:36.610 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:36.611 INFO:teuthology.orchestra.run.smithi097.stdout:2135 write 0xd090f3a thru 0xd091b8f (0xc56 bytes) 2024-09-18T00:18:36.611 INFO:teuthology.orchestra.run.smithi097.stdout:2136 punch from 0xa5eaa0 to 0xa6b734, (0xcc94 bytes) 2024-09-18T00:18:36.611 INFO:teuthology.orchestra.run.smithi097.stdout:2137 writesame 0x14c3b9 thru 0x1560d9 data_size 0x5(0x9d21 bytes) 2024-09-18T00:18:36.612 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:18:36.613 INFO:teuthology.orchestra.run.smithi097.stdout:2140 trunc from 0xd091b90 to 0x437fc64 2024-09-18T00:18:36.613 INFO:teuthology.orchestra.run.smithi097.stdout:2142 write 0x7be0cdc thru 0x7bee493 (0xd7b8 bytes) 2024-09-18T00:18:36.613 INFO:teuthology.orchestra.run.smithi097.stdout:2143 write 0x486a0f5 thru 0x487779a (0xd6a6 bytes) 2024-09-18T00:18:36.613 INFO:teuthology.orchestra.run.smithi097.stdout:2144 trunc from 0x7bee494 to 0xcd241dd 2024-09-18T00:18:36.613 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:36.614 INFO:teuthology.orchestra.run.smithi097.stdout:2146 write 0xb510d16 thru 0xb516459 (0x5744 bytes) 2024-09-18T00:18:36.614 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:36.606+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:36.614 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:18:36.615 INFO:teuthology.orchestra.run.smithi097.stdout:2147 writesame 0x3296d47 thru 0x32a1431 data_size 0x13(0xa6eb bytes) 2024-09-18T00:18:36.638 INFO:teuthology.orchestra.run.smithi097.stdout:2148 trunc from 0xcd241dd to 0x1ec4e48 2024-09-18T00:18:36.641 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:36.641 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:36.641 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:18:36.672 INFO:teuthology.orchestra.run.smithi097.stdout:2149 trunc from 0x1ec4e48 to 0xb146647 2024-09-18T00:18:36.675 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:36.675 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:36.680 INFO:teuthology.orchestra.run.smithi097.stdout:2150 trunc from 0xb146647 to 0xafbb2c5 2024-09-18T00:18:36.702 INFO:teuthology.orchestra.run.smithi097.stdout:2151 trunc from 0xafbb2c5 to 0x711e15 2024-09-18T00:18:36.706 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:36.706 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:36.713 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:36.706+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:36.737 INFO:teuthology.orchestra.run.smithi097.stdout:2152 writesame 0x5915dc5 thru 0x591800d data_size 0x1(0x2249 bytes) 2024-09-18T00:18:36.737 INFO:teuthology.orchestra.run.smithi097.stdout:2153 read 0x7d613d thru 0x7d7c34 (0x1af8 bytes) 2024-09-18T00:18:36.743 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:36.743 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:36.755 INFO:teuthology.orchestra.run.smithi097.stdout:2154 write 0xd239a5c thru 0xd23dc0f (0x41b4 bytes) 2024-09-18T00:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout:2155 read 0x5a26b8b thru 0x5a2e3b2 (0x7828 bytes) 2024-09-18T00:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout:2157 punch from 0x7800efd to 0x78023c8, (0x14cb bytes) 2024-09-18T00:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout:2158 read 0x58101d4 thru 0x5818425 (0x8252 bytes) 2024-09-18T00:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout:2161 read 0x7f70617 thru 0x7f76355 (0x5d3f bytes) 2024-09-18T00:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout:2163 read 0xaf4521 thru 0xafc4f2 (0x7fd2 bytes) 2024-09-18T00:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout:2165 punch from 0x747dd2b to 0x7481ce2, (0x3fb7 bytes) 2024-09-18T00:18:36.762 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:36.763 INFO:teuthology.orchestra.run.smithi097.stdout:2168 trunc from 0xd23dc10 to 0xc063372 2024-09-18T00:18:36.768 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:18:36.768 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:18:36.772 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:36.772 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:36.778 INFO:teuthology.orchestra.run.smithi097.stdout:2169 write 0x4e04e66 thru 0x4e0b5d2 (0x676d bytes) 2024-09-18T00:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:36.783 INFO:teuthology.orchestra.run.smithi097.stdout:2171 write 0x193c650 thru 0x194b91c (0xf2cd bytes) 2024-09-18T00:18:36.789 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.8.1 2024-09-18T00:18:36.794 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout:2173 punch from 0xa7c71ef to 0xa7d5e93, (0xeca4 bytes) 2024-09-18T00:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout:2175 punch from 0x258e5dc to 0x259c781, (0xe1a5 bytes) 2024-09-18T00:18:36.794 INFO:teuthology.orchestra.run.smithi097.stdout:2176 write 0x99f35c0 thru 0x99f908a (0x5acb bytes) 2024-09-18T00:18:36.795 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:36.795 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:36.809 INFO:teuthology.orchestra.run.smithi097.stdout:2177 trunc from 0xc063372 to 0x8b6702c 2024-09-18T00:18:36.813 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:36.806+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:36.819 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:36.820 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:36.823 INFO:teuthology.orchestra.run.smithi097.stdout:2180 punch from 0x4884c44 to 0x4886e1d, (0x21d9 bytes) 2024-09-18T00:18:36.823 INFO:teuthology.orchestra.run.smithi097.stdout:2181 read 0x6d56d0b thru 0x6d5998e (0x2c84 bytes) 2024-09-18T00:18:36.823 INFO:teuthology.orchestra.run.smithi097.stdout:2182 read 0x8aa9439 thru 0x8ab8a99 (0xf661 bytes) 2024-09-18T00:18:36.825 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:18:36.825 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:18:36.829 INFO:teuthology.orchestra.run.smithi097.stdout:2184 trunc from 0x8b6702c to 0x436c5ee 2024-09-18T00:18:36.844 INFO:teuthology.orchestra.run.smithi097.stdout:2185 writesame 0x1e92459 thru 0x1e9c6dc data_size 0x6(0xa284 bytes) 2024-09-18T00:18:36.854 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:36.876 INFO:teuthology.orchestra.run.smithi097.stdout:2189 write 0xe88f530 thru 0xe89633b (0x6e0c bytes) 2024-09-18T00:18:36.882 INFO:teuthology.orchestra.run.smithi097.stdout:2190 writesame 0xb13e6b6 thru 0xb14d207 data_size 0x7(0xeb52 bytes) 2024-09-18T00:18:36.892 INFO:teuthology.orchestra.run.smithi097.stdout:2191 read 0x2ef35dd thru 0x2ef946a (0x5e8e bytes) 2024-09-18T00:18:36.892 INFO:teuthology.orchestra.run.smithi097.stdout:2192 write 0x88438e1 thru 0x884c90e (0x902e bytes) 2024-09-18T00:18:36.899 INFO:teuthology.orchestra.run.smithi097.stdout:2193 write 0x60b5863 thru 0x60c369e (0xde3c bytes) 2024-09-18T00:18:36.907 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:36.908 INFO:teuthology.orchestra.run.smithi097.stdout:2198 writesame 0x62247d7 thru 0x6233094 data_size 0x1f(0xe8be bytes) 2024-09-18T00:18:36.913 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:36.913 INFO:teuthology.orchestra.run.smithi097.stdout:2200 read 0x968987e thru 0x9690b67 (0x72ea bytes) 2024-09-18T00:18:36.914 INFO:teuthology.orchestra.run.smithi097.stdout:2201 write 0x3175311 thru 0x3176973 (0x1663 bytes) 2024-09-18T00:18:36.914 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:36.910+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:36.916 INFO:teuthology.orchestra.run.smithi097.stdout:2202 read 0x19f8e3c thru 0x19f92f7 (0x4bc bytes) 2024-09-18T00:18:36.916 INFO:teuthology.orchestra.run.smithi097.stdout:2203 punch from 0x2d35645 to 0x2d43c56, (0xe611 bytes) 2024-09-18T00:18:36.933 INFO:teuthology.orchestra.run.smithi097.stdout:2205 trunc from 0xe89633c to 0x20eafc4 2024-09-18T00:18:36.937 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:36.937 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:36.952 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:18:36.956 INFO:teuthology.orchestra.run.smithi097.stdout:2207 read 0x63d82d thru 0x644e5a (0x762e bytes) 2024-09-18T00:18:36.970 INFO:teuthology.orchestra.run.smithi097.stdout:2208 write 0x6fc65be thru 0x6fc7726 (0x1169 bytes) 2024-09-18T00:18:36.987 INFO:teuthology.orchestra.run.smithi097.stdout:2210 writesame 0xb090efb thru 0xb0996d9 data_size 0x1(0x87df bytes) 2024-09-18T00:18:36.987 INFO:teuthology.orchestra.run.smithi097.stdout:2211 read 0x4654f92 thru 0x46631ad (0xe21c bytes) 2024-09-18T00:18:36.987 INFO:teuthology.orchestra.run.smithi097.stdout:2214 read 0x42ce1cd thru 0x42cf524 (0x1358 bytes) 2024-09-18T00:18:36.987 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:36.987 INFO:teuthology.orchestra.run.smithi097.stdout:2216 read 0x36c9fde thru 0x36cfcc5 (0x5ce8 bytes) 2024-09-18T00:18:36.987 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:36.988 INFO:teuthology.orchestra.run.smithi097.stdout:2218 clone 8 order 21 su 131072 sc 2 2024-09-18T00:18:37.015 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:37.010+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:37.065 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:37.065 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:37.077 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:18:37.121 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:37.118+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:37.216 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:37.214+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:37.234 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:37.234 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:37.246 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:18:37.317 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:37.314+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:37.357 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:37.358 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:37.372 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:18:37.417 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:37.410+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:37.518 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:37.514+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:37.525 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:37.526 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:37.539 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:18:37.610 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is unset 2024-09-18T00:18:37.618 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:37.614+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:37.678 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:37.679 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:37.694 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:18:37.719 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:37.714+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:37.724 INFO:teuthology.orchestra.run.smithi097.stdout:truncating image image_client.0-clone7 from 0xb0996da (overlap 0x711e15) to 0x4f2ea8 2024-09-18T00:18:37.819 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:37.819 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:37.820 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:37.814+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:37.834 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:18:37.837 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-18T00:18:37.921 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:37.918+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:38.008 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:38.008 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:38.022 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:18:38.023 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:38.018+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:38.123 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:38.118+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:38.145 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:38.146 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:38.159 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:18:38.238 INFO:teuthology.orchestra.run.smithi097.stdout:2219 write 0x3116982 thru 0x3117eb2 (0x1531 bytes) 2024-09-18T00:18:38.250 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:38.246+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:38.264 INFO:teuthology.orchestra.run.smithi097.stdout:2220 trunc from 0xb0996da to 0x489b15 2024-09-18T00:18:38.298 INFO:teuthology.orchestra.run.smithi097.stdout:2221 read 0x3b788a thru 0x3bc5f0 (0x4d67 bytes) 2024-09-18T00:18:38.299 INFO:teuthology.orchestra.run.smithi097.stdout:2222 punch from 0x76cbb to 0x84340, (0xd685 bytes) 2024-09-18T00:18:38.316 INFO:teuthology.orchestra.run.smithi097.stdout:2223 trunc from 0x489b15 to 0xdf4a134 2024-09-18T00:18:38.324 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:38.318+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:38.329 INFO:teuthology.orchestra.run.smithi097.stdout:2226 writesame 0xb97fc5 thru 0xb9a548 data_size 0xe(0x2584 bytes) 2024-09-18T00:18:38.335 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:38.335 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:38.336 INFO:teuthology.orchestra.run.smithi097.stdout:2227 write 0x88c8352 thru 0x88d5b33 (0xd7e2 bytes) 2024-09-18T00:18:38.348 INFO:teuthology.orchestra.run.smithi097.stdout:2228 read 0x33e5ab thru 0x3456f6 (0x714c bytes) 2024-09-18T00:18:38.350 INFO:teuthology.orchestra.run.smithi097.stdout:2229 read 0x623513e thru 0x623c9af (0x7872 bytes) 2024-09-18T00:18:38.350 INFO:teuthology.orchestra.run.smithi097.stdout:2230 write 0xbce92eb thru 0xbcf4dd5 (0xbaeb bytes) 2024-09-18T00:18:38.350 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:18:38.356 INFO:teuthology.orchestra.run.smithi097.stdout:2231 write 0x61caf9 thru 0x62b6f2 (0xebfa bytes) 2024-09-18T00:18:38.361 INFO:teuthology.orchestra.run.smithi097.stdout:2232 write 0x8f5616f thru 0x8f5e8f7 (0x8789 bytes) 2024-09-18T00:18:38.367 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:38.368 INFO:teuthology.orchestra.run.smithi097.stdout:2234 writesame 0x318bfb1 thru 0x31954b2 data_size 0x1(0x9502 bytes) 2024-09-18T00:18:38.417 INFO:teuthology.orchestra.run.smithi097.stdout:get_pg_num_history pg_num_history pg_num_history(e45 pg_nums {1={16=1},2={19=8},3={23=32}} deleted_pools ) 2024-09-18T00:18:38.417 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pgid 3.8 2024-09-18T00:18:38.425 INFO:teuthology.orchestra.run.smithi097.stdout:Write #3:11b3ac8b:::rbd_data.112aa7ba4578.000000000000033a:head# 2024-09-18T00:18:38.425 INFO:teuthology.orchestra.run.smithi097.stdout:snapset 0=[]:{} 2024-09-18T00:18:38.425 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:38.422+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:38.426 INFO:teuthology.orchestra.run.smithi097.stdout:Write #3:12ac789b:::rbd_data.112aa7ba4578.0000000000000130:head# 2024-09-18T00:18:38.426 INFO:teuthology.orchestra.run.smithi097.stdout:snapset 0=[]:{} 2024-09-18T00:18:38.428 INFO:teuthology.orchestra.run.smithi097.stdout:Write #3:16b24e91:::rbd_data.112aa7ba4578.0000000000000161:head# 2024-09-18T00:18:38.428 INFO:teuthology.orchestra.run.smithi097.stdout:snapset 0=[]:{} 2024-09-18T00:18:38.430 INFO:teuthology.orchestra.run.smithi097.stdout:Write #3:17544c57:::rbd_data.112aa7ba4578.0000000000000142:head# 2024-09-18T00:18:38.430 INFO:teuthology.orchestra.run.smithi097.stdout:snapset 0=[]:{} 2024-09-18T00:18:38.431 INFO:teuthology.orchestra.run.smithi097.stdout:Write #3:17e81ee6:::rbd_id.image_client.0-clone4:head# 2024-09-18T00:18:38.432 INFO:teuthology.orchestra.run.smithi097.stdout:snapset 0=[]:{} 2024-09-18T00:18:38.432 INFO:teuthology.orchestra.run.smithi097.stdout:write_pg epoch 45 info 3.8( v 44'54 (0'0,44'54] local-lis/les=23/24 n=5 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) 2024-09-18T00:18:38.432 INFO:teuthology.orchestra.run.smithi097.stdout:Import successful 2024-09-18T00:18:38.476 INFO:teuthology.orchestra.run.smithi097.stdout:2235 read 0xb9d4f63 thru 0xb9da52b (0x55c9 bytes) 2024-09-18T00:18:38.476 INFO:teuthology.orchestra.run.smithi097.stdout:2236 writesame 0xb977ce5 thru 0xb97b014 data_size 0x38(0x3330 bytes) 2024-09-18T00:18:38.480 INFO:teuthology.orchestra.run.smithi097.stdout:2237 write 0x668aeb9 thru 0x6697644 (0xc78c bytes) 2024-09-18T00:18:38.486 INFO:teuthology.orchestra.run.smithi097.stdout:2239 read 0xd4a97a6 thru 0xd4ac17a (0x29d5 bytes) 2024-09-18T00:18:38.486 INFO:teuthology.orchestra.run.smithi097.stdout:2240 read 0xd238ad5 thru 0xd23e879 (0x5da5 bytes) 2024-09-18T00:18:38.486 INFO:teuthology.orchestra.run.smithi097.stdout:2241 write 0x751997e thru 0x7525f86 (0xc609 bytes) 2024-09-18T00:18:38.494 INFO:teuthology.orchestra.run.smithi097.stdout:2242 read 0x39b67bc thru 0x39c38cc (0xd111 bytes) 2024-09-18T00:18:38.494 INFO:teuthology.orchestra.run.smithi097.stdout:2246 write 0xb829378 thru 0xb82e1ee (0x4e77 bytes) 2024-09-18T00:18:38.497 INFO:teuthology.orchestra.run.smithi097.stdout:2247 write 0x4a42a8d thru 0x4a454d3 (0x2a47 bytes) 2024-09-18T00:18:38.504 INFO:teuthology.orchestra.run.smithi097.stdout:2248 write 0x5cc5f6d thru 0x5ccb899 (0x592d bytes) 2024-09-18T00:18:38.513 INFO:teuthology.orchestra.run.smithi097.stdout:2249 write 0x3079a54 thru 0x3088dbd (0xf36a bytes) 2024-09-18T00:18:38.521 INFO:teuthology.orchestra.run.smithi097.stdout:2251 punch from 0xb01a69 to 0xb05b97, (0x412e bytes) 2024-09-18T00:18:38.521 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:38.524 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:38.524 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:38.526 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:38.522+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:38.538 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:18:38.547 INFO:teuthology.orchestra.run.smithi097.stdout:2253 trunc from 0xdf4a134 to 0x3963ea8 2024-09-18T00:18:38.603 INFO:teuthology.orchestra.run.smithi097.stdout:2257 write 0xe8379ae thru 0xe83c46f (0x4ac2 bytes) 2024-09-18T00:18:38.613 INFO:teuthology.orchestra.run.smithi097.stdout:2259 read 0x5dd8aaf thru 0x5de0be5 (0x8137 bytes) 2024-09-18T00:18:38.613 INFO:teuthology.orchestra.run.smithi097.stdout:2260 punch from 0xcba3107 to 0xcbadd63, (0xac5c bytes) 2024-09-18T00:18:38.613 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:38.613 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:38.613 INFO:teuthology.orchestra.run.smithi097.stdout:2264 writesame 0x6e82bc2 thru 0x6e84ee0 data_size 0x1b(0x231f bytes) 2024-09-18T00:18:38.618 INFO:teuthology.orchestra.run.smithi097.stdout:2265 read 0x6c703f4 thru 0x6c7539a (0x4fa7 bytes) 2024-09-18T00:18:38.618 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:38.618 INFO:teuthology.orchestra.run.smithi097.stdout:2267 read 0xb863ef2 thru 0xb873011 (0xf120 bytes) 2024-09-18T00:18:38.618 INFO:teuthology.orchestra.run.smithi097.stdout:2268 read 0xc8c3958 thru 0xc8c8a09 (0x50b2 bytes) 2024-09-18T00:18:38.618 INFO:teuthology.orchestra.run.smithi097.stdout:2270 read 0xcbf6621 thru 0xcc010a4 (0xaa84 bytes) 2024-09-18T00:18:38.618 INFO:teuthology.orchestra.run.smithi097.stdout:2271 punch from 0x7ae9f89 to 0x7af1fa0, (0x8017 bytes) 2024-09-18T00:18:38.618 INFO:teuthology.orchestra.run.smithi097.stdout:2272 read 0x7835a43 thru 0x784382d (0xddeb bytes) 2024-09-18T00:18:38.619 INFO:teuthology.orchestra.run.smithi097.stdout:2273 read 0x7100862 thru 0x710bd57 (0xb4f6 bytes) 2024-09-18T00:18:38.619 INFO:teuthology.orchestra.run.smithi097.stdout:2274 write 0x48022e7 thru 0x4811566 (0xf280 bytes) 2024-09-18T00:18:38.627 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:38.622+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:38.632 INFO:teuthology.orchestra.run.smithi097.stdout:2275 read 0x83cf914 thru 0x83cfcba (0x3a7 bytes) 2024-09-18T00:18:38.632 INFO:teuthology.orchestra.run.smithi097.stdout:2276 punch from 0x67a6d71 to 0x67aea82, (0x7d11 bytes) 2024-09-18T00:18:38.632 INFO:teuthology.orchestra.run.smithi097.stdout:2277 write 0x27c1f19 thru 0x27c2937 (0xa1f bytes) 2024-09-18T00:18:38.659 INFO:teuthology.orchestra.run.smithi097.stdout:2279 trunc from 0xe83c470 to 0x7263157 2024-09-18T00:18:38.670 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:38.670 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:38.685 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:18:38.708 INFO:teuthology.orchestra.run.smithi097.stdout:2280 write 0xdb9bd42 thru 0xdba0b22 (0x4de1 bytes) 2024-09-18T00:18:38.728 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:38.722+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:38.728 INFO:teuthology.orchestra.run.smithi097.stdout:2281 write 0xeba8f03 thru 0xebadfec (0x50ea bytes) 2024-09-18T00:18:38.733 INFO:teuthology.orchestra.run.smithi097.stdout:2282 read 0x29e283c thru 0x29e4633 (0x1df8 bytes) 2024-09-18T00:18:38.733 INFO:teuthology.orchestra.run.smithi097.stdout:2283 writesame 0x78667aa thru 0x78679c9 data_size 0x28(0x1220 bytes) 2024-09-18T00:18:38.738 INFO:teuthology.orchestra.run.smithi097.stdout:2284 write 0x7a8f057 thru 0x7a93c09 (0x4bb3 bytes) 2024-09-18T00:18:38.775 INFO:teuthology.orchestra.run.smithi097.stdout:2285 trunc from 0xebadfed to 0x2940326 2024-09-18T00:18:38.796 INFO:teuthology.orchestra.run.smithi097.stdout:2287 read 0x2474b2d thru 0x247a2de (0x57b2 bytes) 2024-09-18T00:18:38.796 INFO:teuthology.orchestra.run.smithi097.stdout:2290 read 0xab034f thru 0xabf833 (0xf4e5 bytes) 2024-09-18T00:18:38.796 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:38.797 INFO:teuthology.orchestra.run.smithi097.stdout:2294 read 0x13b0e0d thru 0x13ba0a9 (0x929d bytes) 2024-09-18T00:18:38.797 INFO:teuthology.orchestra.run.smithi097.stdout:2297 read 0x1eb3a2b thru 0x1ebfc45 (0xc21b bytes) 2024-09-18T00:18:38.814 INFO:teuthology.orchestra.run.smithi097.stdout:2298 trunc from 0x2940326 to 0x96575e1 2024-09-18T00:18:38.826 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:18:38.829 INFO:teuthology.orchestra.run.smithi097.stdout:2299 writesame 0x13e996e thru 0x13ec70c data_size 0x11(0x2d9f bytes) 2024-09-18T00:18:38.829 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:38.826+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:38.836 INFO:teuthology.orchestra.run.smithi097.stdout:2300 write 0x433dcd thru 0x435517 (0x174b bytes) 2024-09-18T00:18:38.839 INFO:teuthology.orchestra.run.smithi097.stdout:2301 read 0x2cb2e89 thru 0x2cb3339 (0x4b1 bytes) 2024-09-18T00:18:38.854 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:38.854 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:38.859 INFO:teuthology.orchestra.run.smithi097.stdout:2302 writesame 0xbf90a34 thru 0xbf98784 data_size 0x1(0x7d51 bytes) 2024-09-18T00:18:38.859 INFO:teuthology.orchestra.run.smithi097.stdout:2303 punch from 0xacb3781 to 0xacb8c1b, (0x549a bytes) 2024-09-18T00:18:38.859 INFO:teuthology.orchestra.run.smithi097.stdout:2304 write 0x7cc25b7 thru 0x7cd1161 (0xebab bytes) 2024-09-18T00:18:38.867 INFO:teuthology.orchestra.run.smithi097.stdout:2305 writesame 0x7c2fea0 thru 0x7c39bca data_size 0xd(0x9d2b bytes) 2024-09-18T00:18:38.870 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:18:38.877 INFO:teuthology.orchestra.run.smithi097.stdout:2307 write 0xadb08e4 thru 0xadb5684 (0x4da1 bytes) 2024-09-18T00:18:38.882 INFO:teuthology.orchestra.run.smithi097.stdout:2308 write 0x8ac063c thru 0x8ac3407 (0x2dcc bytes) 2024-09-18T00:18:38.886 INFO:teuthology.orchestra.run.smithi097.stdout:2309 read 0x4373594 thru 0x4376aaf (0x351c bytes) 2024-09-18T00:18:38.887 INFO:teuthology.orchestra.run.smithi097.stdout:2310 read 0x107edaf thru 0x1081e4f (0x30a1 bytes) 2024-09-18T00:18:38.887 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:38.897 INFO:teuthology.orchestra.run.smithi097.stdout:2312 write 0xc83b102 thru 0xc83de5d (0x2d5c bytes) 2024-09-18T00:18:38.902 INFO:teuthology.orchestra.run.smithi097.stdout:2313 punch from 0x57a6c04 to 0x57b2cf1, (0xc0ed bytes) 2024-09-18T00:18:38.902 INFO:teuthology.orchestra.run.smithi097.stdout:2314 write 0x47ce892 thru 0x47d0044 (0x17b3 bytes) 2024-09-18T00:18:38.906 INFO:teuthology.orchestra.run.smithi097.stdout:2316 write 0xbaa9e3a thru 0xbab7ec3 (0xe08a bytes) 2024-09-18T00:18:38.913 INFO:teuthology.orchestra.run.smithi097.stdout:2319 write 0x8e64339 thru 0x8e6712f (0x2df7 bytes) 2024-09-18T00:18:38.918 INFO:teuthology.orchestra.run.smithi097.stdout:2321 read 0x7cd8aca thru 0x7cdb58e (0x2ac5 bytes) 2024-09-18T00:18:38.919 INFO:teuthology.orchestra.run.smithi097.stdout:2322 read 0xb7b3150 thru 0xb7c187a (0xe72b bytes) 2024-09-18T00:18:38.919 INFO:teuthology.orchestra.run.smithi097.stdout:2324 punch from 0x172b419 to 0x17362dd, (0xaec4 bytes) 2024-09-18T00:18:38.930 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:38.926+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:38.933 INFO:teuthology.orchestra.run.smithi097.stdout:2325 trunc from 0xc83de5e to 0x33355ba 2024-09-18T00:18:38.948 INFO:teuthology.orchestra.run.smithi097.stdout:2326 write 0x2fdc83f thru 0x2fe628f (0x9a51 bytes) 2024-09-18T00:18:38.957 INFO:teuthology.orchestra.run.smithi097.stdout:2329 punch from 0x2f90e1f to 0x2f9cc28, (0xbe09 bytes) 2024-09-18T00:18:38.957 INFO:teuthology.orchestra.run.smithi097.stdout:2330 punch from 0x226017c to 0x22667df, (0x6663 bytes) 2024-09-18T00:18:38.980 INFO:teuthology.orchestra.run.smithi097.stdout:2331 write 0xbc40c67 thru 0xbc4416f (0x3509 bytes) 2024-09-18T00:18:38.984 INFO:teuthology.orchestra.run.smithi097.stdout:2332 read 0x1846665 thru 0x1853ad6 (0xd472 bytes) 2024-09-18T00:18:38.984 INFO:teuthology.orchestra.run.smithi097.stdout:2333 writesame 0x725ba5a thru 0x72670a0 data_size 0x1(0xb647 bytes) 2024-09-18T00:18:38.986 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /var/log/ceph/exp.3.8.1 2024-09-18T00:18:38.986 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:38.987 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:38.987 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:18:39.001 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:39.002 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:39.004 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:39.004 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:39.020 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:39.020 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:39.030 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:18:39.031 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:39.026+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:39.040 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:39.040 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:39.058 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:18:39.059 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:18:39.071 INFO:teuthology.orchestra.run.smithi097.stdout:2335 write 0xbf89b04 thru 0xbf8d214 (0x3711 bytes) 2024-09-18T00:18:39.076 INFO:teuthology.orchestra.run.smithi097.stdout:2336 trunc from 0xbf8d215 to 0xe418bb3 2024-09-18T00:18:39.077 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:39.077 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:39.084 INFO:teuthology.orchestra.run.smithi097.stdout:2338 read 0x581f52e thru 0x582b2e0 (0xbdb3 bytes) 2024-09-18T00:18:39.085 INFO:teuthology.orchestra.run.smithi097.stdout:2339 write 0x1842b82 thru 0x184af03 (0x8382 bytes) 2024-09-18T00:18:39.090 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.090 INFO:teuthology.orchestra.run.smithi097.stdout:2341 punch from 0x9b1b714 to 0x9b27cca, (0xc5b6 bytes) 2024-09-18T00:18:39.091 INFO:teuthology.orchestra.run.smithi097.stdout:2342 flatten 2024-09-18T00:18:39.098 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:39.098 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:39.107 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:18:39.107 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:18:39.132 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:39.126+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:39.147 INFO:teuthology.orchestra.run.smithi097.stdout:2343 trunc from 0xe418bb3 to 0x6fa3185 2024-09-18T00:18:39.160 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:39.160 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:39.160 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:18:39.160 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:39.160 INFO:teuthology.orchestra.run.smithi097.stdout:2344 read 0x1de361b thru 0x1dee5c9 (0xafaf bytes) 2024-09-18T00:18:39.174 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:18:39.175 INFO:teuthology.orchestra.run.smithi097.stdout:2345 writesame 0xaaa1166 thru 0xaaad4c9 data_size 0x52(0xc364 bytes) 2024-09-18T00:18:39.175 INFO:teuthology.orchestra.run.smithi097.stdout:2346 read 0x886c153 thru 0x887bbe8 (0xfa96 bytes) 2024-09-18T00:18:39.187 INFO:teuthology.orchestra.run.smithi097.stdout:2347 trunc from 0xaaad4ca to 0x32c0824 2024-09-18T00:18:39.197 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.221 INFO:teuthology.orchestra.run.smithi097.stdout:2352 write 0x8c21369 thru 0x8c2f134 (0xddcc bytes) 2024-09-18T00:18:39.227 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.227 INFO:teuthology.orchestra.run.smithi097.stdout:2356 writesame 0x27e97bb thru 0x27eefdc data_size 0x1d(0x5822 bytes) 2024-09-18T00:18:39.235 INFO:teuthology.orchestra.run.smithi097.stdout:2357 write 0x93976f thru 0x949767 (0xfff9 bytes) 2024-09-18T00:18:39.242 INFO:teuthology.orchestra.run.smithi097.stdout:2358 trunc from 0x8c2f135 to 0x913e7dd 2024-09-18T00:18:39.251 INFO:teuthology.orchestra.run.smithi097.stdout:2359 writesame 0x94e539 thru 0x959ebf data_size 0x73(0xb987 bytes) 2024-09-18T00:18:39.262 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:39.258+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:39.270 INFO:teuthology.orchestra.run.smithi097.stdout:2360 write 0xaabd43f thru 0xaac517f (0x7d41 bytes) 2024-09-18T00:18:39.279 INFO:teuthology.orchestra.run.smithi097.stdout:2361 punch from 0x975c85d to 0x976c410, (0xfbb3 bytes) 2024-09-18T00:18:39.279 INFO:teuthology.orchestra.run.smithi097.stdout:2362 writesame 0x66e0e48 thru 0x66eafeb data_size 0xa(0xa1a4 bytes) 2024-09-18T00:18:39.299 INFO:teuthology.orchestra.run.smithi097.stdout:2364 punch from 0x254a288 to 0x254b929, (0x16a1 bytes) 2024-09-18T00:18:39.299 INFO:teuthology.orchestra.run.smithi097.stdout:2365 read 0x36ca5ae thru 0x36d5b9c (0xb5ef bytes) 2024-09-18T00:18:39.299 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.317 INFO:teuthology.orchestra.run.smithi097.stdout:2367 trunc from 0xaac5180 to 0x2b911d5 2024-09-18T00:18:39.334 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:39.330+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:39.341 INFO:teuthology.orchestra.run.smithi097.stdout:2368 read 0x1fab6bd thru 0x1fb5f82 (0xa8c6 bytes) 2024-09-18T00:18:39.341 INFO:teuthology.orchestra.run.smithi097.stdout:2369 read 0x1f9e275 thru 0x1fad1b7 (0xef43 bytes) 2024-09-18T00:18:39.348 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:39.348 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:39.348 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:39.348 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:39.348 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:39.360 INFO:teuthology.orchestra.run.smithi097.stdout:2371 write 0x46571b4 thru 0x4665e01 (0xec4e bytes) 2024-09-18T00:18:39.363 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:18:39.369 INFO:teuthology.orchestra.run.smithi097.stdout:2372 read 0x35e5d44 thru 0x35f4e52 (0xf10f bytes) 2024-09-18T00:18:39.369 INFO:teuthology.orchestra.run.smithi097.stdout:2374 read 0x250f4b8 thru 0x2510a5b (0x15a4 bytes) 2024-09-18T00:18:39.369 INFO:teuthology.orchestra.run.smithi097.stdout:2375 write 0x3efe153 thru 0x3f09f20 (0xbdce bytes) 2024-09-18T00:18:39.382 INFO:teuthology.orchestra.run.smithi097.stdout:2376 read 0x2b16140 thru 0x2b258b0 (0xf771 bytes) 2024-09-18T00:18:39.382 INFO:teuthology.orchestra.run.smithi097.stdout:2378 read 0x37a9184 thru 0x37ac763 (0x35e0 bytes) 2024-09-18T00:18:39.382 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.382 INFO:teuthology.orchestra.run.smithi097.stdout:2381 read 0x866bdf thru 0x875d14 (0xf136 bytes) 2024-09-18T00:18:39.383 INFO:teuthology.orchestra.run.smithi097.stdout:2382 read 0x3e60f91 thru 0x3e68e58 (0x7ec8 bytes) 2024-09-18T00:18:39.391 INFO:teuthology.orchestra.run.smithi097.stdout:2383 trunc from 0x4665e02 to 0x1fcd55a 2024-09-18T00:18:39.433 INFO:teuthology.orchestra.run.smithi097.stdout:2384 trunc from 0x1fcd55a to 0xa1bc4ba 2024-09-18T00:18:39.450 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:39.446+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:39.456 INFO:teuthology.orchestra.run.smithi097.stdout:2385 writesame 0xb5acf3d thru 0xb5aeffb data_size 0x1(0x20bf bytes) 2024-09-18T00:18:39.468 INFO:teuthology.orchestra.run.smithi097.stdout:2387 writesame 0xc3051dd thru 0xc30ff5a data_size 0x2(0xad7e bytes) 2024-09-18T00:18:39.468 INFO:teuthology.orchestra.run.smithi097.stdout:2389 write 0x8446b0d thru 0x844e5c0 (0x7ab4 bytes) 2024-09-18T00:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout:2396 punch from 0xa4686e5 to 0xa4708bc, (0x81d7 bytes) 2024-09-18T00:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout:2397 punch from 0x64c0857 to 0x64c9063, (0x880c bytes) 2024-09-18T00:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout:2399 read 0x17c7461 thru 0x17d0c54 (0x97f4 bytes) 2024-09-18T00:18:39.475 INFO:teuthology.orchestra.run.smithi097.stdout:2400 write 0xaa1982d thru 0xaa1e28c (0x4a60 bytes) 2024-09-18T00:18:39.482 INFO:teuthology.orchestra.run.smithi097.stdout:2402 read 0x64eacb thru 0x651d3a (0x3270 bytes) 2024-09-18T00:18:39.482 INFO:teuthology.orchestra.run.smithi097.stdout:2403 punch from 0x9a77004 to 0x9a7ed87, (0x7d83 bytes) 2024-09-18T00:18:39.493 INFO:teuthology.orchestra.run.smithi097.stdout:2404 write 0xc43a35e thru 0xc448de7 (0xea8a bytes) 2024-09-18T00:18:39.502 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.503 INFO:teuthology.orchestra.run.smithi097.stdout:2407 writesame 0x76e808b thru 0x76f6b2d data_size 0x1(0xeaa3 bytes) 2024-09-18T00:18:39.530 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:39.531 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:39.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:39.531 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:39.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.531 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2228 3.15 3:aa2fc059:::rbd_data.112aa7ba4578.000000000000014a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:18:39.531 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:18:39.531 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.2650179, 2024-09-18T00:18:39.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.068966377999999995, 2024-09-18T00:18:39.532 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.532 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.532 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:39.532 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.532 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.532 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.532 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2228 2024-09-18T00:18:39.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.532 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:18:39.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:18:39.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263099+0000", 2024-09-18T00:18:39.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2359999999999999e-06 2024-09-18T00:18:39.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263115+0000", 2024-09-18T00:18:39.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5996999999999999e-05 2024-09-18T00:18:39.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263156+0000", 2024-09-18T00:18:39.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:18:39.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263183+0000", 2024-09-18T00:18:39.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7233000000000002e-05 2024-09-18T00:18:39.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263304+0000", 2024-09-18T00:18:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121209 2024-09-18T00:18:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263528+0000", 2024-09-18T00:18:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022401800000000001 2024-09-18T00:18:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:18:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.266458+0000", 2024-09-18T00:18:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029296980000000001 2024-09-18T00:18:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331628+0000", 2024-09-18T00:18:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.065170045999999995 2024-09-18T00:18:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331819+0000", 2024-09-18T00:18:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000190819 2024-09-18T00:18:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331971+0000", 2024-09-18T00:18:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151972 2024-09-18T00:18:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332063+0000", 2024-09-18T00:18:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2083999999999995e-05 2024-09-18T00:18:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2228 3.15 e36/32 ondisk, result = 0)", 2024-09-18T00:18:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:18:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.251964787999999, 2024-09-18T00:18:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055930148999999998, 2024-09-18T00:18:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276152+0000", 2024-09-18T00:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0200000000000001e-06 2024-09-18T00:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276162+0000", 2024-09-18T00:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0767e-05 2024-09-18T00:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276202+0000", 2024-09-18T00:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000003e-05 2024-09-18T00:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276226+0000", 2024-09-18T00:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.351e-05 2024-09-18T00:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331782+0000", 2024-09-18T00:18:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055556296999999998 2024-09-18T00:18:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331801+0000", 2024-09-18T00:18:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9357000000000001e-05 2024-09-18T00:18:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332080+0000", 2024-09-18T00:18:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027849800000000002 2024-09-18T00:18:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2854 3.f 3:f02b0bca:::rbd_data.112aa7ba4578.0000000000000135:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:18:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:18:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.65804541, 2024-09-18T00:18:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.082582802999999996, 2024-09-18T00:18:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2854 2024-09-18T00:18:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:18:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:18:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870071+0000", 2024-09-18T00:18:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.227e-06 2024-09-18T00:18:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870083+0000", 2024-09-18T00:18:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1909999999999999e-05 2024-09-18T00:18:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870114+0000", 2024-09-18T00:18:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0576e-05 2024-09-18T00:18:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870129+0000", 2024-09-18T00:18:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.485e-05 2024-09-18T00:18:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870256+0000", 2024-09-18T00:18:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012691300000000001 2024-09-18T00:18:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870493+0000", 2024-09-18T00:18:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023737399999999999 2024-09-18T00:18:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.874933+0000", 2024-09-18T00:18:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044398299999999996 2024-09-18T00:18:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952452+0000", 2024-09-18T00:18:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.077518918000000006 2024-09-18T00:18:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952486+0000", 2024-09-18T00:18:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4035999999999997e-05 2024-09-18T00:18:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952575+0000", 2024-09-18T00:18:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.954e-05 2024-09-18T00:18:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952652+0000", 2024-09-18T00:18:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6629000000000001e-05 2024-09-18T00:18:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2854 3.f e38/32 ondisk, result = 0)", 2024-09-18T00:18:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:18:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.640514594999999, 2024-09-18T00:18:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064868625999999999, 2024-09-18T00:18:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:18:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:18:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887602+0000", 2024-09-18T00:18:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.677e-06 2024-09-18T00:18:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887609+0000", 2024-09-18T00:18:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6729999999999992e-06 2024-09-18T00:18:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887637+0000", 2024-09-18T00:18:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8206000000000001e-05 2024-09-18T00:18:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887651+0000", 2024-09-18T00:18:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3387000000000001e-05 2024-09-18T00:18:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952422+0000", 2024-09-18T00:18:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064771366999999996 2024-09-18T00:18:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952441+0000", 2024-09-18T00:18:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8737e-05 2024-09-18T00:18:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952469+0000", 2024-09-18T00:18:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7579000000000001e-05 2024-09-18T00:18:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5631 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297078+0000", 2024-09-18T00:18:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.231036208000001, 2024-09-18T00:18:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046621463000000002, 2024-09-18T00:18:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5631 2024-09-18T00:18:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297078+0000", 2024-09-18T00:18:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297078+0000", 2024-09-18T00:18:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297080+0000", 2024-09-18T00:18:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2929999999999999e-06 2024-09-18T00:18:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297085+0000", 2024-09-18T00:18:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6799999999999998e-06 2024-09-18T00:18:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297103+0000", 2024-09-18T00:18:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7479e-05 2024-09-18T00:18:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297116+0000", 2024-09-18T00:18:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2894000000000001e-05 2024-09-18T00:18:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343417+0000", 2024-09-18T00:18:39.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046301006999999998 2024-09-18T00:18:39.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343551+0000", 2024-09-18T00:18:39.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013466300000000001 2024-09-18T00:18:39.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343700+0000", 2024-09-18T00:18:39.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014844700000000001 2024-09-18T00:18:39.562 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.562 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.562 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.562 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5632 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:39.562 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297135+0000", 2024-09-18T00:18:39.563 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.230979096, 2024-09-18T00:18:39.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046771002999999998, 2024-09-18T00:18:39.563 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.563 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.563 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.563 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.563 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.563 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.563 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5632 2024-09-18T00:18:39.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.563 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297135+0000", 2024-09-18T00:18:39.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297135+0000", 2024-09-18T00:18:39.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297137+0000", 2024-09-18T00:18:39.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1999999999999999e-06 2024-09-18T00:18:39.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297141+0000", 2024-09-18T00:18:39.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8239999999999999e-06 2024-09-18T00:18:39.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297158+0000", 2024-09-18T00:18:39.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6566e-05 2024-09-18T00:18:39.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297175+0000", 2024-09-18T00:18:39.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7280000000000001e-05 2024-09-18T00:18:39.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343749+0000", 2024-09-18T00:18:39.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046574084000000002 2024-09-18T00:18:39.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343817+0000", 2024-09-18T00:18:39.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7198000000000003e-05 2024-09-18T00:18:39.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343906+0000", 2024-09-18T00:18:39.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9851000000000004e-05 2024-09-18T00:18:39.568 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.569 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.569 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.569 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5633 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:39.570 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297194+0000", 2024-09-18T00:18:39.570 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.230920766000001, 2024-09-18T00:18:39.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046979053, 2024-09-18T00:18:39.570 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.570 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.570 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.571 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.571 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.571 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.571 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5633 2024-09-18T00:18:39.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.571 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297194+0000", 2024-09-18T00:18:39.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297194+0000", 2024-09-18T00:18:39.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297195+0000", 2024-09-18T00:18:39.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2440000000000001e-06 2024-09-18T00:18:39.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297201+0000", 2024-09-18T00:18:39.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5489999999999997e-06 2024-09-18T00:18:39.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297218+0000", 2024-09-18T00:18:39.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7652999999999999e-05 2024-09-18T00:18:39.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297245+0000", 2024-09-18T00:18:39.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7212999999999998e-05 2024-09-18T00:18:39.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343955+0000", 2024-09-18T00:18:39.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046709564000000002 2024-09-18T00:18:39.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344028+0000", 2024-09-18T00:18:39.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3370999999999999e-05 2024-09-18T00:18:39.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344173+0000", 2024-09-18T00:18:39.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014445899999999999 2024-09-18T00:18:39.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.577 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.577 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5634 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:39.577 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297263+0000", 2024-09-18T00:18:39.578 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.230851489999999, 2024-09-18T00:18:39.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047153893000000002, 2024-09-18T00:18:39.578 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.578 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.578 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.578 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.578 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.578 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.578 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5634 2024-09-18T00:18:39.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.578 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297263+0000", 2024-09-18T00:18:39.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297263+0000", 2024-09-18T00:18:39.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297264+0000", 2024-09-18T00:18:39.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4169999999999999e-06 2024-09-18T00:18:39.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297270+0000", 2024-09-18T00:18:39.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4829999999999999e-06 2024-09-18T00:18:39.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297287+0000", 2024-09-18T00:18:39.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6993000000000001e-05 2024-09-18T00:18:39.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297300+0000", 2024-09-18T00:18:39.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.324e-05 2024-09-18T00:18:39.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344221+0000", 2024-09-18T00:18:39.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046921030000000002 2024-09-18T00:18:39.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344310+0000", 2024-09-18T00:18:39.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8533000000000001e-05 2024-09-18T00:18:39.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344417+0000", 2024-09-18T00:18:39.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107197 2024-09-18T00:18:39.583 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.583 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.583 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.583 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5635 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:39.583 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297317+0000", 2024-09-18T00:18:39.583 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.23079774, 2024-09-18T00:18:39.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047292222000000002, 2024-09-18T00:18:39.584 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.584 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.584 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.584 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.584 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.584 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.584 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5635 2024-09-18T00:18:39.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.584 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297317+0000", 2024-09-18T00:18:39.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297317+0000", 2024-09-18T00:18:39.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297318+0000", 2024-09-18T00:18:39.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2869999999999999e-06 2024-09-18T00:18:39.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297323+0000", 2024-09-18T00:18:39.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1760000000000001e-06 2024-09-18T00:18:39.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297343+0000", 2024-09-18T00:18:39.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9473000000000001e-05 2024-09-18T00:18:39.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297356+0000", 2024-09-18T00:18:39.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3767e-05 2024-09-18T00:18:39.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344455+0000", 2024-09-18T00:18:39.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047098220000000003 2024-09-18T00:18:39.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344518+0000", 2024-09-18T00:18:39.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2856999999999998e-05 2024-09-18T00:18:39.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344609+0000", 2024-09-18T00:18:39.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1441999999999996e-05 2024-09-18T00:18:39.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.588 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.588 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5672 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 9109504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:39.588 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298045+0000", 2024-09-18T00:18:39.588 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.230069399, 2024-09-18T00:18:39.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046729202999999997, 2024-09-18T00:18:39.589 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.589 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.589 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.589 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5672 2024-09-18T00:18:39.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.589 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298045+0000", 2024-09-18T00:18:39.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298045+0000", 2024-09-18T00:18:39.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298046+0000", 2024-09-18T00:18:39.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1260000000000001e-06 2024-09-18T00:18:39.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298051+0000", 2024-09-18T00:18:39.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9300000000000002e-06 2024-09-18T00:18:39.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298068+0000", 2024-09-18T00:18:39.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7153999999999999e-05 2024-09-18T00:18:39.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298082+0000", 2024-09-18T00:18:39.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3329999999999999e-05 2024-09-18T00:18:39.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344636+0000", 2024-09-18T00:18:39.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046554460999999998 2024-09-18T00:18:39.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344696+0000", 2024-09-18T00:18:39.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0034999999999997e-05 2024-09-18T00:18:39.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344774+0000", 2024-09-18T00:18:39.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8166999999999999e-05 2024-09-18T00:18:39.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.593 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.593 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5673 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 9895936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:39.593 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298098+0000", 2024-09-18T00:18:39.594 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.230016022999999, 2024-09-18T00:18:39.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046867918000000001, 2024-09-18T00:18:39.594 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.594 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.594 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.594 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5673 2024-09-18T00:18:39.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.594 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298098+0000", 2024-09-18T00:18:39.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298098+0000", 2024-09-18T00:18:39.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298100+0000", 2024-09-18T00:18:39.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0699999999999999e-06 2024-09-18T00:18:39.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298104+0000", 2024-09-18T00:18:39.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9339999999999997e-06 2024-09-18T00:18:39.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298121+0000", 2024-09-18T00:18:39.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6656000000000001e-05 2024-09-18T00:18:39.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298149+0000", 2024-09-18T00:18:39.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8243e-05 2024-09-18T00:18:39.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344801+0000", 2024-09-18T00:18:39.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046651199999999997 2024-09-18T00:18:39.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344860+0000", 2024-09-18T00:18:39.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9537999999999998e-05 2024-09-18T00:18:39.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344966+0000", 2024-09-18T00:18:39.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106277 2024-09-18T00:18:39.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.598 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.598 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5674 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 10682368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:39.599 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298168+0000", 2024-09-18T00:18:39.599 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.229946936999999, 2024-09-18T00:18:39.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047098783999999998, 2024-09-18T00:18:39.599 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.599 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.599 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.599 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.599 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.599 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.599 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5674 2024-09-18T00:18:39.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.599 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298168+0000", 2024-09-18T00:18:39.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298168+0000", 2024-09-18T00:18:39.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298169+0000", 2024-09-18T00:18:39.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1370000000000001e-06 2024-09-18T00:18:39.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298174+0000", 2024-09-18T00:18:39.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6500000000000001e-06 2024-09-18T00:18:39.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298192+0000", 2024-09-18T00:18:39.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7620000000000001e-05 2024-09-18T00:18:39.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298205+0000", 2024-09-18T00:18:39.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3363000000000001e-05 2024-09-18T00:18:39.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345013+0000", 2024-09-18T00:18:39.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046807418000000003 2024-09-18T00:18:39.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345125+0000", 2024-09-18T00:18:39.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111901 2024-09-18T00:18:39.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345266+0000", 2024-09-18T00:18:39.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000141695 2024-09-18T00:18:39.604 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.604 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.604 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.604 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5675 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:39.604 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298223+0000", 2024-09-18T00:18:39.604 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.229891044, 2024-09-18T00:18:39.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047324643999999999, 2024-09-18T00:18:39.605 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.605 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.605 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.605 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.605 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.605 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.605 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5675 2024-09-18T00:18:39.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.605 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298223+0000", 2024-09-18T00:18:39.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298223+0000", 2024-09-18T00:18:39.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298225+0000", 2024-09-18T00:18:39.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.13e-06 2024-09-18T00:18:39.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298230+0000", 2024-09-18T00:18:39.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1599999999999997e-06 2024-09-18T00:18:39.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298247+0000", 2024-09-18T00:18:39.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6813999999999999e-05 2024-09-18T00:18:39.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298260+0000", 2024-09-18T00:18:39.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3036e-05 2024-09-18T00:18:39.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345315+0000", 2024-09-18T00:18:39.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047055420000000001 2024-09-18T00:18:39.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345384+0000", 2024-09-18T00:18:39.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9375999999999994e-05 2024-09-18T00:18:39.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345548+0000", 2024-09-18T00:18:39.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016370800000000001 2024-09-18T00:18:39.609 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.609 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.609 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.610 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5676 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 12255232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:39.610 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:18:39.610 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.229838738, 2024-09-18T00:18:39.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047571816000000003, 2024-09-18T00:18:39.610 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.610 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.610 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.610 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.610 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.610 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.610 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5676 2024-09-18T00:18:39.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.611 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:18:39.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:18:39.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298277+0000", 2024-09-18T00:18:39.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.454e-06 2024-09-18T00:18:39.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298282+0000", 2024-09-18T00:18:39.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6299999999999997e-06 2024-09-18T00:18:39.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298299+0000", 2024-09-18T00:18:39.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6959000000000001e-05 2024-09-18T00:18:39.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298313+0000", 2024-09-18T00:18:39.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.376e-05 2024-09-18T00:18:39.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345610+0000", 2024-09-18T00:18:39.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047297128000000001 2024-09-18T00:18:39.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345708+0000", 2024-09-18T00:18:39.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8245999999999994e-05 2024-09-18T00:18:39.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345848+0000", 2024-09-18T00:18:39.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000139639 2024-09-18T00:18:39.614 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.615 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.615 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.615 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5677 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 12779520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:39.615 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:18:39.615 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.229785225000001, 2024-09-18T00:18:39.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047810679000000002, 2024-09-18T00:18:39.615 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.615 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.615 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.615 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.616 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.616 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.616 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5677 2024-09-18T00:18:39.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.616 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:18:39.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:18:39.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298330+0000", 2024-09-18T00:18:39.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.217e-06 2024-09-18T00:18:39.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298354+0000", 2024-09-18T00:18:39.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3753000000000001e-05 2024-09-18T00:18:39.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298388+0000", 2024-09-18T00:18:39.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4276999999999997e-05 2024-09-18T00:18:39.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298415+0000", 2024-09-18T00:18:39.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6403000000000001e-05 2024-09-18T00:18:39.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345896+0000", 2024-09-18T00:18:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047481007999999998 2024-09-18T00:18:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345992+0000", 2024-09-18T00:18:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6141e-05 2024-09-18T00:18:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346140+0000", 2024-09-18T00:18:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014788 2024-09-18T00:18:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.620 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.620 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5678 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:39.620 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:18:39.620 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.229679045999999, 2024-09-18T00:18:39.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047985466999999997, 2024-09-18T00:18:39.620 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.620 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.620 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.620 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.620 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.620 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.620 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5678 2024-09-18T00:18:39.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.621 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:18:39.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:18:39.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298437+0000", 2024-09-18T00:18:39.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1799999999999999e-06 2024-09-18T00:18:39.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298443+0000", 2024-09-18T00:18:39.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1140000000000002e-06 2024-09-18T00:18:39.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298460+0000", 2024-09-18T00:18:39.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7615999999999999e-05 2024-09-18T00:18:39.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298474+0000", 2024-09-18T00:18:39.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4017e-05 2024-09-18T00:18:39.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346180+0000", 2024-09-18T00:18:39.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047705801999999999 2024-09-18T00:18:39.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346274+0000", 2024-09-18T00:18:39.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3814999999999995e-05 2024-09-18T00:18:39.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346421+0000", 2024-09-18T00:18:39.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000146923 2024-09-18T00:18:39.624 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.624 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.624 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.624 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5679 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:39.624 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:18:39.624 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.229620002000001, 2024-09-18T00:18:39.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048190771, 2024-09-18T00:18:39.625 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.625 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.625 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.625 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.625 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.625 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.625 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5679 2024-09-18T00:18:39.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.625 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:18:39.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:18:39.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298496+0000", 2024-09-18T00:18:39.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4699999999999999e-06 2024-09-18T00:18:39.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298501+0000", 2024-09-18T00:18:39.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:18:39.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298519+0000", 2024-09-18T00:18:39.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7329e-05 2024-09-18T00:18:39.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298532+0000", 2024-09-18T00:18:39.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3083e-05 2024-09-18T00:18:39.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346462+0000", 2024-09-18T00:18:39.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047930449999999999 2024-09-18T00:18:39.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346556+0000", 2024-09-18T00:18:39.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3222999999999998e-05 2024-09-18T00:18:39.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346685+0000", 2024-09-18T00:18:39.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000129696 2024-09-18T00:18:39.629 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.629 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.629 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.629 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5680 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:39.629 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:18:39.629 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.229562593000001, 2024-09-18T00:18:39.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048535847, 2024-09-18T00:18:39.629 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.629 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.629 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.630 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.630 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5680 2024-09-18T00:18:39.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.630 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:18:39.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:18:39.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298553+0000", 2024-09-18T00:18:39.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0759999999999999e-06 2024-09-18T00:18:39.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298575+0000", 2024-09-18T00:18:39.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2019999999999999e-05 2024-09-18T00:18:39.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298593+0000", 2024-09-18T00:18:39.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7900000000000001e-05 2024-09-18T00:18:39.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298607+0000", 2024-09-18T00:18:39.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3844e-05 2024-09-18T00:18:39.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346791+0000", 2024-09-18T00:18:39.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048184449999999997 2024-09-18T00:18:39.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346921+0000", 2024-09-18T00:18:39.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012994699999999999 2024-09-18T00:18:39.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347088+0000", 2024-09-18T00:18:39.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016661000000000001 2024-09-18T00:18:39.633 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.633 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.633 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.634 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5681 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:39.634 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:18:39.634 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.229490182999999, 2024-09-18T00:18:39.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048743337999999997, 2024-09-18T00:18:39.634 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.634 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.634 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.635 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5681 2024-09-18T00:18:39.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.635 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:18:39.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:18:39.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298626+0000", 2024-09-18T00:18:39.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4899999999999999e-06 2024-09-18T00:18:39.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298631+0000", 2024-09-18T00:18:39.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0300000000000001e-06 2024-09-18T00:18:39.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298650+0000", 2024-09-18T00:18:39.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9499000000000001e-05 2024-09-18T00:18:39.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298664+0000", 2024-09-18T00:18:39.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3767e-05 2024-09-18T00:18:39.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347139+0000", 2024-09-18T00:18:39.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048475348000000001 2024-09-18T00:18:39.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347228+0000", 2024-09-18T00:18:39.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8843000000000003e-05 2024-09-18T00:18:39.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347368+0000", 2024-09-18T00:18:39.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013936100000000001 2024-09-18T00:18:39.638 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.638 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.639 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.639 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5682 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:39.639 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:18:39.639 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.229433368, 2024-09-18T00:18:39.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048916864999999997, 2024-09-18T00:18:39.639 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:39.639 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:39.639 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:39.640 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:39.640 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:39.640 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:39.640 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5682 2024-09-18T00:18:39.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.640 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:39.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:39.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:18:39.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:39.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:18:39.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:39.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:39.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298682+0000", 2024-09-18T00:18:39.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:18:39.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:39.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298687+0000", 2024-09-18T00:18:39.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9239999999999998e-06 2024-09-18T00:18:39.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:39.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298704+0000", 2024-09-18T00:18:39.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6569999999999999e-05 2024-09-18T00:18:39.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:39.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298731+0000", 2024-09-18T00:18:39.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6752999999999999e-05 2024-09-18T00:18:39.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:39.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347416+0000", 2024-09-18T00:18:39.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048685056999999997 2024-09-18T00:18:39.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:39.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347506+0000", 2024-09-18T00:18:39.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9936000000000001e-05 2024-09-18T00:18:39.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:39.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:39.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:39.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347598+0000", 2024-09-18T00:18:39.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2335000000000003e-05 2024-09-18T00:18:39.645 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.645 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.645 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.646 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:39.646 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:39.646 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:39.646 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:18:39.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:18:39.649 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:39.642+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:39.649 INFO:teuthology.orchestra.run.smithi097.stdout:2411 trunc from 0xc448de8 to 0xdc79b9f 2024-09-18T00:18:39.659 INFO:teuthology.orchestra.run.smithi097.stdout:2412 write 0xbe09ea2 thru 0xbe1756e (0xd6cd bytes) 2024-09-18T00:18:39.671 INFO:teuthology.orchestra.run.smithi097.stdout:2414 punch from 0x451747c to 0x452691a, (0xf49e bytes) 2024-09-18T00:18:39.671 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.671 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.671 INFO:teuthology.orchestra.run.smithi097.stdout:2420 write 0x4fc52e7 thru 0x4fd192c (0xc646 bytes) 2024-09-18T00:18:39.681 INFO:teuthology.orchestra.run.smithi097.stdout:2422 read 0x42b5ac2 thru 0x42c02a7 (0xa7e6 bytes) 2024-09-18T00:18:39.681 INFO:teuthology.orchestra.run.smithi097.stdout:2423 write 0xa33ba2c thru 0xa33cc23 (0x11f8 bytes) 2024-09-18T00:18:39.686 INFO:teuthology.orchestra.run.smithi097.stdout:2424 read 0x55cd161 thru 0x55d7efa (0xad9a bytes) 2024-09-18T00:18:39.686 INFO:teuthology.orchestra.run.smithi097.stdout:2428 punch from 0x83e93c9 to 0x83eacdd, (0x1914 bytes) 2024-09-18T00:18:39.686 INFO:teuthology.orchestra.run.smithi097.stdout:2429 read 0xbbd0588 thru 0xbbd3eac (0x3925 bytes) 2024-09-18T00:18:39.686 INFO:teuthology.orchestra.run.smithi097.stdout:2430 writesame 0x72ae7b1 thru 0x72b5ca3 data_size 0xd(0x74f3 bytes) 2024-09-18T00:18:39.695 INFO:teuthology.orchestra.run.smithi097.stdout:2431 read 0x142c745 thru 0x1432564 (0x5e20 bytes) 2024-09-18T00:18:39.695 INFO:teuthology.orchestra.run.smithi097.stdout:2432 read 0x1f8d1ad thru 0x1f97c23 (0xaa77 bytes) 2024-09-18T00:18:39.695 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.695 INFO:teuthology.orchestra.run.smithi097.stdout:2434 read 0x725e497 thru 0x726aa7e (0xc5e8 bytes) 2024-09-18T00:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout:2436 punch from 0x703307 to 0x70853d, (0x5236 bytes) 2024-09-18T00:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout:2437 read 0xaaefcd2 thru 0xaafb428 (0xb757 bytes) 2024-09-18T00:18:39.696 INFO:teuthology.orchestra.run.smithi097.stdout:2438 read 0xb38edcb thru 0xb39b387 (0xc5bd bytes) 2024-09-18T00:18:39.697 INFO:teuthology.orchestra.run.smithi097.stdout:2440 write 0xe95e32 thru 0xe9890f (0x2ade bytes) 2024-09-18T00:18:39.701 INFO:teuthology.orchestra.run.smithi097.stdout:2441 read 0x55a41c1 thru 0x55a9fe9 (0x5e29 bytes) 2024-09-18T00:18:39.701 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.701 INFO:teuthology.orchestra.run.smithi097.stdout:2443 read 0xd04fa11 thru 0xd053b81 (0x4171 bytes) 2024-09-18T00:18:39.703 INFO:teuthology.orchestra.run.smithi097.stdout:2444 trunc from 0xdc79b9f to 0xebc76fa 2024-09-18T00:18:39.714 INFO:teuthology.orchestra.run.smithi097.stdout:2445 trunc from 0xebc76fa to 0xd39f9f4 2024-09-18T00:18:39.723 INFO:teuthology.orchestra.run.smithi097.stdout:2447 read 0xb953ef4 thru 0xb961e99 (0xdfa6 bytes) 2024-09-18T00:18:39.724 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.724 INFO:teuthology.orchestra.run.smithi097.stdout:2450 punch from 0xd0b04c4 to 0xd0bd058, (0xcb94 bytes) 2024-09-18T00:18:39.724 INFO:teuthology.orchestra.run.smithi097.stdout:2451 write 0x5e7c5f5 thru 0x5e8624f (0x9c5b bytes) 2024-09-18T00:18:39.731 INFO:teuthology.orchestra.run.smithi097.stdout:2454 read 0xaf4743 thru 0xb02911 (0xe1cf bytes) 2024-09-18T00:18:39.732 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.732 INFO:teuthology.orchestra.run.smithi097.stdout:2456 write 0xf20d71 thru 0xf270ae (0x633e bytes) 2024-09-18T00:18:39.748 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:39.742+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:39.754 INFO:teuthology.orchestra.run.smithi097.stdout:2458 trunc from 0xd39f9f4 to 0x2c51af4 2024-09-18T00:18:39.775 INFO:teuthology.orchestra.run.smithi097.stdout:2460 read 0x24a3a0 thru 0x24d18e (0x2def bytes) 2024-09-18T00:18:39.784 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:39.784 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:39.784 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:39.784 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:39.795 INFO:teuthology.orchestra.run.smithi097.stdout:2461 write 0x73dbe29 thru 0x73dcb3d (0xd15 bytes) 2024-09-18T00:18:39.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:18:39.809 INFO:teuthology.orchestra.run.smithi097.stdout:2462 trunc from 0x73dcb3e to 0xcce245c 2024-09-18T00:18:39.819 INFO:teuthology.orchestra.run.smithi097.stdout:2463 punch from 0x288308c to 0x288e58f, (0xb503 bytes) 2024-09-18T00:18:39.820 INFO:teuthology.orchestra.run.smithi097.stdout:2464 write 0x9b1f71d thru 0x9b2cd12 (0xd5f6 bytes) 2024-09-18T00:18:39.827 INFO:teuthology.orchestra.run.smithi097.stdout:2465 punch from 0x2bd734e to 0x2be4af1, (0xd7a3 bytes) 2024-09-18T00:18:39.827 INFO:teuthology.orchestra.run.smithi097.stdout:2466 punch from 0x3b3a3b2 to 0x3b486ad, (0xe2fb bytes) 2024-09-18T00:18:39.839 INFO:teuthology.orchestra.run.smithi097.stdout:2467 trunc from 0xcce245c to 0x462f764 2024-09-18T00:18:39.848 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:39.842+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:39.860 INFO:teuthology.orchestra.run.smithi097.stdout:2468 punch from 0xde79ae to 0xdebcc5, (0x4317 bytes) 2024-09-18T00:18:39.860 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.860 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.880 INFO:teuthology.orchestra.run.smithi097.stdout:2472 writesame 0xa9b2fa5 thru 0xa9b52ab data_size 0x15(0x2307 bytes) 2024-09-18T00:18:39.880 INFO:teuthology.orchestra.run.smithi097.stdout:2473 write 0x32c63ec thru 0x32cda5b (0x7670 bytes) 2024-09-18T00:18:39.888 INFO:teuthology.orchestra.run.smithi097.stdout:2474 read 0x13252ea thru 0x132baea (0x6801 bytes) 2024-09-18T00:18:39.898 INFO:teuthology.orchestra.run.smithi097.stdout:2476 trunc from 0xa9b52ac to 0x51b051b 2024-09-18T00:18:39.911 INFO:teuthology.orchestra.run.smithi097.stdout:2477 read 0x4a8a169 thru 0x4a981cc (0xe064 bytes) 2024-09-18T00:18:39.911 INFO:teuthology.orchestra.run.smithi097.stdout:2479 punch from 0xd02423 to 0xd04262, (0x1e3f bytes) 2024-09-18T00:18:39.932 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:39.932 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:39.932 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:39.932 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:39.932 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:39.943 INFO:teuthology.orchestra.run.smithi097.stdout:2480 write 0xd46a883 thru 0xd47a215 (0xf993 bytes) 2024-09-18T00:18:39.947 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:18:39.949 INFO:teuthology.orchestra.run.smithi097.stdout:2481 trunc from 0xd47a216 to 0xd7bafe5 2024-09-18T00:18:39.950 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:39.946+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:39.963 INFO:teuthology.orchestra.run.smithi097.stdout:2482 read 0x966e5a3 thru 0x967df65 (0xf9c3 bytes) 2024-09-18T00:18:39.964 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:39.964 INFO:teuthology.orchestra.run.smithi097.stdout:2484 writesame 0x97acde4 thru 0x97b43e6 data_size 0x1(0x7603 bytes) 2024-09-18T00:18:40.050 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:40.046+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:40.058 INFO:teuthology.orchestra.run.smithi097.stdout:2488 writesame 0x242ef9e thru 0x243d478 data_size 0x3b(0xe4db bytes) 2024-09-18T00:18:40.067 INFO:teuthology.orchestra.run.smithi097.stdout:2490 writesame 0xcf653e5 thru 0xcf71d65 data_size 0x13(0xc981 bytes) 2024-09-18T00:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2230 3.17 3:ef492888:::rbd_data.112aa7ba4578.000000000000014f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:40.079 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.810299147999999, 2024-09-18T00:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075065283999999996, 2024-09-18T00:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2230 2024-09-18T00:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266014+0000", 2024-09-18T00:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266029+0000", 2024-09-18T00:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4616e-05 2024-09-18T00:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266053+0000", 2024-09-18T00:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4369999999999999e-05 2024-09-18T00:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266067+0000", 2024-09-18T00:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4243999999999999e-05 2024-09-18T00:18:40.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266157+0000", 2024-09-18T00:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9909999999999998e-05 2024-09-18T00:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266327+0000", 2024-09-18T00:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016981200000000001 2024-09-18T00:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269180+0000", 2024-09-18T00:18:40.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028526559999999999 2024-09-18T00:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340955+0000", 2024-09-18T00:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.071774904 2024-09-18T00:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340974+0000", 2024-09-18T00:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8824000000000001e-05 2024-09-18T00:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341028+0000", 2024-09-18T00:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4456999999999997e-05 2024-09-18T00:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341076+0000", 2024-09-18T00:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8284000000000001e-05 2024-09-18T00:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2230 3.17 e36/23 ondisk, result = 0)", 2024-09-18T00:18:40.085 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.792955567, 2024-09-18T00:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057609645000000001, 2024-09-18T00:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283356+0000", 2024-09-18T00:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.44e-06 2024-09-18T00:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283362+0000", 2024-09-18T00:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2539999999999998e-06 2024-09-18T00:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283382+0000", 2024-09-18T00:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9896000000000001e-05 2024-09-18T00:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283398+0000", 2024-09-18T00:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5489999999999999e-05 2024-09-18T00:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340936+0000", 2024-09-18T00:18:40.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057538649999999997 2024-09-18T00:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340947+0000", 2024-09-18T00:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0981e-05 2024-09-18T00:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340964+0000", 2024-09-18T00:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6934000000000001e-05 2024-09-18T00:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2419 3.17 3:ed94b133:::rbd_data.112aa7ba4578.00000000000001b5:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.338834093999999, 2024-09-18T00:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.064761812000000002, 2024-09-18T00:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2419 2024-09-18T00:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.090 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737479+0000", 2024-09-18T00:18:40.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6089999999999999e-06 2024-09-18T00:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737487+0000", 2024-09-18T00:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7900000000000005e-06 2024-09-18T00:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737507+0000", 2024-09-18T00:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9936e-05 2024-09-18T00:18:40.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737518+0000", 2024-09-18T00:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1046999999999999e-05 2024-09-18T00:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737599+0000", 2024-09-18T00:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0697000000000003e-05 2024-09-18T00:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737697+0000", 2024-09-18T00:18:40.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8058000000000002e-05 2024-09-18T00:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.740445+0000", 2024-09-18T00:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027481279999999999 2024-09-18T00:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802023+0000", 2024-09-18T00:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.061577321999999997 2024-09-18T00:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802060+0000", 2024-09-18T00:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7586000000000002e-05 2024-09-18T00:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802158+0000", 2024-09-18T00:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7583000000000004e-05 2024-09-18T00:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802238+0000", 2024-09-18T00:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0055999999999994e-05 2024-09-18T00:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.095 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2419 3.17 e37/23 ondisk, result = 0)", 2024-09-18T00:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.326609147999999, 2024-09-18T00:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052342948, 2024-09-18T00:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749702+0000", 2024-09-18T00:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:18:40.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749710+0000", 2024-09-18T00:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7279999999999995e-06 2024-09-18T00:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749736+0000", 2024-09-18T00:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5797999999999999e-05 2024-09-18T00:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749751+0000", 2024-09-18T00:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5238999999999999e-05 2024-09-18T00:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801745+0000", 2024-09-18T00:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051994022000000001 2024-09-18T00:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801770+0000", 2024-09-18T00:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5103e-05 2024-09-18T00:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802044+0000", 2024-09-18T00:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027395799999999998 2024-09-18T00:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6503 3.1b 3:d9387a1f:::rbd_data.112a3a39fc45.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.511936177999999, 2024-09-18T00:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050510547000000003, 2024-09-18T00:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.100 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6503 2024-09-18T00:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:18:40.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564376+0000", 2024-09-18T00:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-06 2024-09-18T00:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564386+0000", 2024-09-18T00:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0312999999999999e-05 2024-09-18T00:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564408+0000", 2024-09-18T00:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2001000000000002e-05 2024-09-18T00:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564421+0000", 2024-09-18T00:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2271e-05 2024-09-18T00:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564491+0000", 2024-09-18T00:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0630000000000006e-05 2024-09-18T00:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564606+0000", 2024-09-18T00:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011449 2024-09-18T00:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.566743+0000", 2024-09-18T00:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021373830000000001 2024-09-18T00:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614738+0000", 2024-09-18T00:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047995359000000001 2024-09-18T00:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614759+0000", 2024-09-18T00:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1010999999999999e-05 2024-09-18T00:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614826+0000", 2024-09-18T00:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6654000000000003e-05 2024-09-18T00:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614884+0000", 2024-09-18T00:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8332999999999999e-05 2024-09-18T00:18:40.105 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6503 3.1b e42/23 ondisk, result = 0)", 2024-09-18T00:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.502676709999999, 2024-09-18T00:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041115407, 2024-09-18T00:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573635+0000", 2024-09-18T00:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5689999999999999e-06 2024-09-18T00:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573643+0000", 2024-09-18T00:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9699999999999999e-06 2024-09-18T00:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573668+0000", 2024-09-18T00:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5536000000000001e-05 2024-09-18T00:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573683+0000", 2024-09-18T00:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4025e-05 2024-09-18T00:18:40.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614717+0000", 2024-09-18T00:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041034517 2024-09-18T00:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614730+0000", 2024-09-18T00:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2945999999999999e-05 2024-09-18T00:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614749+0000", 2024-09-18T00:18:40.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8844000000000001e-05 2024-09-18T00:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6518 3.1 3:8061bf9f:::rbd_data.112a3a39fc45.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.425497904, 2024-09-18T00:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062303123000000002, 2024-09-18T00:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.110 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6518 2024-09-18T00:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650814+0000", 2024-09-18T00:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.629e-06 2024-09-18T00:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650823+0000", 2024-09-18T00:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1800000000000002e-06 2024-09-18T00:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650843+0000", 2024-09-18T00:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9836000000000001e-05 2024-09-18T00:18:40.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650854+0000", 2024-09-18T00:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1564e-05 2024-09-18T00:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650917+0000", 2024-09-18T00:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3076000000000004e-05 2024-09-18T00:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.651029+0000", 2024-09-18T00:18:40.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111154 2024-09-18T00:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.655531+0000", 2024-09-18T00:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045027629999999999 2024-09-18T00:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712924+0000", 2024-09-18T00:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057393006000000003 2024-09-18T00:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:40.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712988+0000", 2024-09-18T00:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000005e-05 2024-09-18T00:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713056+0000", 2024-09-18T00:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7904999999999994e-05 2024-09-18T00:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713115+0000", 2024-09-18T00:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9339999999999998e-05 2024-09-18T00:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.115 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6518 3.1 e42/23 ondisk, result = 0)", 2024-09-18T00:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.413072821, 2024-09-18T00:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049887304, 2024-09-18T00:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:18:40.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663239+0000", 2024-09-18T00:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663246+0000", 2024-09-18T00:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4189999999999996e-06 2024-09-18T00:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663265+0000", 2024-09-18T00:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8323000000000001e-05 2024-09-18T00:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663277+0000", 2024-09-18T00:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2065999999999999e-05 2024-09-18T00:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712965+0000", 2024-09-18T00:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049688317000000003 2024-09-18T00:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712977+0000", 2024-09-18T00:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.206e-05 2024-09-18T00:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713125+0000", 2024-09-18T00:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014749899999999999 2024-09-18T00:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.119 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:8831 3.12 3:49cac8cf:::rbd_data.112a9f4259df.0000000000000130:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T00:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.7952125719999996, 2024-09-18T00:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104125047, 2024-09-18T00:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 8831 2024-09-18T00:18:40.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281099+0000", 2024-09-18T00:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.029e-06 2024-09-18T00:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281103+0000", 2024-09-18T00:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7759999999999997e-06 2024-09-18T00:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281114+0000", 2024-09-18T00:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1090000000000001e-05 2024-09-18T00:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281120+0000", 2024-09-18T00:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.604e-06 2024-09-18T00:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281164+0000", 2024-09-18T00:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3696999999999999e-05 2024-09-18T00:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281270+0000", 2024-09-18T00:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106041 2024-09-18T00:18:40.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.285596+0000", 2024-09-18T00:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043266659999999998 2024-09-18T00:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385028+0000", 2024-09-18T00:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.099431808999999996 2024-09-18T00:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:40.124 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385094+0000", 2024-09-18T00:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5740000000000004e-05 2024-09-18T00:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385166+0000", 2024-09-18T00:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1637000000000005e-05 2024-09-18T00:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385223+0000", 2024-09-18T00:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6957999999999999e-05 2024-09-18T00:18:40.125 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:8831 3.12 e48/32 ondisk, result = 0)", 2024-09-18T00:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.7786984620000004, 2024-09-18T00:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087621558000000002, 2024-09-18T00:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297613+0000", 2024-09-18T00:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3400000000000001e-06 2024-09-18T00:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297620+0000", 2024-09-18T00:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5869999999999998e-06 2024-09-18T00:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297639+0000", 2024-09-18T00:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9888999999999999e-05 2024-09-18T00:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297661+0000", 2024-09-18T00:18:40.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1772000000000001e-05 2024-09-18T00:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385073+0000", 2024-09-18T00:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087411303999999995 2024-09-18T00:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385086+0000", 2024-09-18T00:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3040000000000001e-05 2024-09-18T00:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385233+0000", 2024-09-18T00:18:40.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014762600000000001 2024-09-18T00:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10099 3.6 3:62e7f20b:::rbd_data.112a9f4259df.0000000000000001:8 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.572938+0000", 2024-09-18T00:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.5033718220000001, 2024-09-18T00:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026138630999999999, 2024-09-18T00:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:40.130 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10099 2024-09-18T00:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572938+0000", 2024-09-18T00:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572938+0000", 2024-09-18T00:18:40.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572939+0000", 2024-09-18T00:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7400000000000003e-07 2024-09-18T00:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572942+0000", 2024-09-18T00:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7159999999999999e-06 2024-09-18T00:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572949+0000", 2024-09-18T00:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7379999999999994e-06 2024-09-18T00:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572955+0000", 2024-09-18T00:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7860000000000002e-06 2024-09-18T00:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.598855+0000", 2024-09-18T00:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025900175000000001 2024-09-18T00:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.598939+0000", 2024-09-18T00:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3244000000000005e-05 2024-09-18T00:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599077+0000", 2024-09-18T00:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138398 2024-09-18T00:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10108 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:40.134 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.572963+0000", 2024-09-18T00:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.503347379, 2024-09-18T00:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026310093999999999, 2024-09-18T00:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10108 2024-09-18T00:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572963+0000", 2024-09-18T00:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572963+0000", 2024-09-18T00:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572963+0000", 2024-09-18T00:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7299999999999996e-07 2024-09-18T00:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572966+0000", 2024-09-18T00:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4389999999999999e-06 2024-09-18T00:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572973+0000", 2024-09-18T00:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6520000000000006e-06 2024-09-18T00:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572980+0000", 2024-09-18T00:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:18:40.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599154+0000", 2024-09-18T00:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026174096000000001 2024-09-18T00:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599196+0000", 2024-09-18T00:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1993000000000003e-05 2024-09-18T00:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599273+0000", 2024-09-18T00:18:40.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7144999999999994e-05 2024-09-18T00:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10109 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.5033229019999998, 2024-09-18T00:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026513858000000001, 2024-09-18T00:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.139 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10109 2024-09-18T00:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:18:40.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572988+0000", 2024-09-18T00:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1099999999999996e-07 2024-09-18T00:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572990+0000", 2024-09-18T00:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3329999999999999e-06 2024-09-18T00:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572998+0000", 2024-09-18T00:18:40.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2039999999999997e-06 2024-09-18T00:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573012+0000", 2024-09-18T00:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3879e-05 2024-09-18T00:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599312+0000", 2024-09-18T00:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026300093 2024-09-18T00:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599380+0000", 2024-09-18T00:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7401000000000001e-05 2024-09-18T00:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599501+0000", 2024-09-18T00:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000121437 2024-09-18T00:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10110 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:18:40.143 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.5032894429999999, 2024-09-18T00:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026746066999999998, 2024-09-18T00:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10110 2024-09-18T00:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:18:40.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2799999999999996e-07 2024-09-18T00:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573030+0000", 2024-09-18T00:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5469999999999997e-06 2024-09-18T00:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573039+0000", 2024-09-18T00:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6080000000000002e-06 2024-09-18T00:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573049+0000", 2024-09-18T00:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0759999999999992e-06 2024-09-18T00:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599539+0000", 2024-09-18T00:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026490952000000002 2024-09-18T00:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599621+0000", 2024-09-18T00:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1601999999999996e-05 2024-09-18T00:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599767+0000", 2024-09-18T00:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145654 2024-09-18T00:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10111 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.5032512279999999, 2024-09-18T00:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027006651999999999, 2024-09-18T00:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10111 2024-09-18T00:18:40.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9500000000000002e-07 2024-09-18T00:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573062+0000", 2024-09-18T00:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6539999999999999e-06 2024-09-18T00:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573071+0000", 2024-09-18T00:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5380000000000002e-06 2024-09-18T00:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573077+0000", 2024-09-18T00:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999999e-06 2024-09-18T00:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599818+0000", 2024-09-18T00:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02674093 2024-09-18T00:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599923+0000", 2024-09-18T00:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010490200000000001 2024-09-18T00:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600066+0000", 2024-09-18T00:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014284500000000001 2024-09-18T00:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10112 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.503225434, 2024-09-18T00:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027148835, 2024-09-18T00:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.152 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.153 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:40.153 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.153 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.153 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.153 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10112 2024-09-18T00:18:40.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.153 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:18:40.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:18:40.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:18:40.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9500000000000003e-07 2024-09-18T00:18:40.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573088+0000", 2024-09-18T00:18:40.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9569999999999998e-06 2024-09-18T00:18:40.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573097+0000", 2024-09-18T00:18:40.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8599999999999999e-06 2024-09-18T00:18:40.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573104+0000", 2024-09-18T00:18:40.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5710000000000003e-06 2024-09-18T00:18:40.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600091+0000", 2024-09-18T00:18:40.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026987315000000001 2024-09-18T00:18:40.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600158+0000", 2024-09-18T00:18:40.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6727e-05 2024-09-18T00:18:40.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600234+0000", 2024-09-18T00:18:40.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5909999999999997e-05 2024-09-18T00:18:40.156 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.156 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.156 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.157 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10113 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:40.157 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:18:40.157 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.503181176, 2024-09-18T00:18:40.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027356020000000002, 2024-09-18T00:18:40.157 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.157 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.157 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:40.157 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.157 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.157 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.157 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10113 2024-09-18T00:18:40.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.158 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:18:40.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:18:40.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573130+0000", 2024-09-18T00:18:40.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1999999999999999e-07 2024-09-18T00:18:40.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573133+0000", 2024-09-18T00:18:40.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:18:40.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573141+0000", 2024-09-18T00:18:40.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5110000000000003e-06 2024-09-18T00:18:40.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573155+0000", 2024-09-18T00:18:40.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3730000000000001e-05 2024-09-18T00:18:40.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600275+0000", 2024-09-18T00:18:40.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027119621999999999 2024-09-18T00:18:40.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600349+0000", 2024-09-18T00:18:40.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4624999999999998e-05 2024-09-18T00:18:40.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600485+0000", 2024-09-18T00:18:40.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013570499999999999 2024-09-18T00:18:40.161 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.161 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.161 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.161 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10114 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:40.161 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:18:40.161 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.5031458129999997, 2024-09-18T00:18:40.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027471173000000002, 2024-09-18T00:18:40.162 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.162 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.162 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:40.162 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.162 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.162 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.162 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10114 2024-09-18T00:18:40.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.162 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:18:40.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:18:40.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573165+0000", 2024-09-18T00:18:40.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.99e-07 2024-09-18T00:18:40.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573168+0000", 2024-09-18T00:18:40.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8339999999999999e-06 2024-09-18T00:18:40.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573176+0000", 2024-09-18T00:18:40.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0199999999999994e-06 2024-09-18T00:18:40.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573182+0000", 2024-09-18T00:18:40.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.455e-06 2024-09-18T00:18:40.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600521+0000", 2024-09-18T00:18:40.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027338989000000001 2024-09-18T00:18:40.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600577+0000", 2024-09-18T00:18:40.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5717000000000002e-05 2024-09-18T00:18:40.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600635+0000", 2024-09-18T00:18:40.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8559e-05 2024-09-18T00:18:40.165 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.166 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.166 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.166 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10115 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:40.166 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:18:40.166 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.5030929359999998, 2024-09-18T00:18:40.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027643300999999999, 2024-09-18T00:18:40.166 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.166 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.166 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:40.166 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.166 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.167 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.167 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10115 2024-09-18T00:18:40.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.167 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:18:40.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:18:40.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573218+0000", 2024-09-18T00:18:40.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-18T00:18:40.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573221+0000", 2024-09-18T00:18:40.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3079999999999999e-06 2024-09-18T00:18:40.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573233+0000", 2024-09-18T00:18:40.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1544e-05 2024-09-18T00:18:40.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573240+0000", 2024-09-18T00:18:40.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4070000000000004e-06 2024-09-18T00:18:40.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600665+0000", 2024-09-18T00:18:40.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02742522 2024-09-18T00:18:40.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600745+0000", 2024-09-18T00:18:40.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0014999999999999e-05 2024-09-18T00:18:40.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600860+0000", 2024-09-18T00:18:40.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115127 2024-09-18T00:18:40.170 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.170 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.170 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.170 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11459 3.1b 3:db862ee8:::rbd_data.112ab1fa102a.0000000000000152:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:40.170 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:18:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.8146641250000002, 2024-09-18T00:18:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.032246431999999998, 2024-09-18T00:18:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11459 2024-09-18T00:18:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:18:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:18:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261649+0000", 2024-09-18T00:18:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1200000000000002e-06 2024-09-18T00:18:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261665+0000", 2024-09-18T00:18:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5668999999999999e-05 2024-09-18T00:18:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261700+0000", 2024-09-18T00:18:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4925000000000002e-05 2024-09-18T00:18:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261723+0000", 2024-09-18T00:18:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3189000000000001e-05 2024-09-18T00:18:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261815+0000", 2024-09-18T00:18:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1662000000000004e-05 2024-09-18T00:18:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261904+0000", 2024-09-18T00:18:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9201000000000002e-05 2024-09-18T00:18:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.262955+0000", 2024-09-18T00:18:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00105156 2024-09-18T00:18:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293760+0000", 2024-09-18T00:18:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030804292 2024-09-18T00:18:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293773+0000", 2024-09-18T00:18:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3361e-05 2024-09-18T00:18:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293833+0000", 2024-09-18T00:18:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9626000000000001e-05 2024-09-18T00:18:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293892+0000", 2024-09-18T00:18:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9827000000000002e-05 2024-09-18T00:18:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.176 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:40.177 INFO:teuthology.orchestra.run.smithi097.stdout:2492 trunc from 0xd7bafe5 to 0xb86b9b8 2024-09-18T00:18:40.178 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:18:40.179 INFO:teuthology.orchestra.run.smithi097.stdout:2494 punch from 0x30ec22d to 0x30f59fa, (0x97cd bytes) 2024-09-18T00:18:40.179 INFO:teuthology.orchestra.run.smithi097.stdout:2496 write 0x8a3d271 thru 0x8a3e1fd (0xf8d bytes) 2024-09-18T00:18:40.180 INFO:teuthology.orchestra.run.smithi097.stdout:2497 punch from 0x8f8e7b7 to 0x8f9be7f, (0xd6c8 bytes) 2024-09-18T00:18:40.180 INFO:teuthology.orchestra.run.smithi097.stdout:2498 read 0x91f0a44 thru 0x91f850c (0x7ac9 bytes) 2024-09-18T00:18:40.180 INFO:teuthology.orchestra.run.smithi097.stdout:2499 read 0x6f9fbb3 thru 0x6fa9794 (0x9be2 bytes) 2024-09-18T00:18:40.180 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:40.180 INFO:teuthology.orchestra.run.smithi097.stdout:2502 write 0x6283fa0 thru 0x6289a22 (0x5a83 bytes) 2024-09-18T00:18:40.180 INFO:teuthology.orchestra.run.smithi097.stdout:2503 writesame 0xc44b74d thru 0xc457f46 data_size 0x1a(0xc7fa bytes) 2024-09-18T00:18:40.180 INFO:teuthology.orchestra.run.smithi097.stdout:2504 write 0x760b69b thru 0x761687a (0xb1e0 bytes) 2024-09-18T00:18:40.180 INFO:teuthology.orchestra.run.smithi097.stdout:2505 read 0x5c656eb thru 0x5c68385 (0x2c9b bytes) 2024-09-18T00:18:40.180 INFO:teuthology.orchestra.run.smithi097.stdout:2507 punch from 0x7820d41 to 0x782d1d7, (0xc496 bytes) 2024-09-18T00:18:40.180 INFO:teuthology.orchestra.run.smithi097.stdout:2509 read 0xb2f85fa thru 0xb2f8934 (0x33b bytes) 2024-09-18T00:18:40.180 INFO:teuthology.orchestra.run.smithi097.stdout:2512 writesame 0xd818cd1 thru 0xd8243c4 data_size 0x12(0xb6f4 bytes) 2024-09-18T00:18:40.180 INFO:teuthology.orchestra.run.smithi097.stdout:2514 write 0x83dce47 thru 0x83ebd65 (0xef1f bytes) 2024-09-18T00:18:40.181 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:40.174+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:40.186 INFO:teuthology.orchestra.run.smithi097.stdout:2516 write 0x304512 thru 0x30545b (0xf4a bytes) 2024-09-18T00:18:40.192 INFO:teuthology.orchestra.run.smithi097.stdout:2518 trunc from 0xd8243c5 to 0xe4b1703 2024-09-18T00:18:40.213 INFO:teuthology.orchestra.run.smithi097.stdout:2519 trunc from 0xe4b1703 to 0xb546a5b 2024-09-18T00:18:40.226 INFO:teuthology.orchestra.run.smithi097.stdout:2520 read 0x3621c4e thru 0x362589e (0x3c51 bytes) 2024-09-18T00:18:40.226 INFO:teuthology.orchestra.run.smithi097.stdout:2521 read 0x107b39c thru 0x107f3b5 (0x401a bytes) 2024-09-18T00:18:40.226 INFO:teuthology.orchestra.run.smithi097.stdout:2522 writesame 0x6fea8c9 thru 0x6fecbe2 data_size 0x1(0x231a bytes) 2024-09-18T00:18:40.240 INFO:teuthology.orchestra.run.smithi097.stdout:2523 read 0x115ed8b thru 0x116a7a0 (0xba16 bytes) 2024-09-18T00:18:40.241 INFO:teuthology.orchestra.run.smithi097.stdout:2524 clone 9 order 18 su 65536 sc 3 2024-09-18T00:18:40.278 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:40.274+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:40.314 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:40.315 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:40.315 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:40.315 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:40.329 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:18:40.378 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:40.374+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:40.479 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:40.474+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:40.503 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:40.503 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:40.503 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:40.503 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:40.503 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:40.517 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:18:40.580 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:40.574+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:40.627 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is set 2024-09-18T00:18:40.680 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:40.674+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.704 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4263.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-18T00:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:55.481299+0000", 2024-09-18T00:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.219849947, 2024-09-18T00:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036304219999999999, 2024-09-18T00:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481299+0000", 2024-09-18T00:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481299+0000", 2024-09-18T00:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481300+0000", 2024-09-18T00:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3e-07 2024-09-18T00:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481307+0000", 2024-09-18T00:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4959999999999999e-06 2024-09-18T00:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481330+0000", 2024-09-18T00:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3363999999999999e-05 2024-09-18T00:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481347+0000", 2024-09-18T00:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6263e-05 2024-09-18T00:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481428+0000", 2024-09-18T00:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1612000000000004e-05 2024-09-18T00:18:40.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481495+0000", 2024-09-18T00:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6439999999999994e-05 2024-09-18T00:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.484828+0000", 2024-09-18T00:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033337140000000002 2024-09-18T00:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.484929+0000", 2024-09-18T00:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100803 2024-09-18T00:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:285 3.9 e28/23 3:94c9a3ca:::rbd_data.112a45c8e167.000000000000001c:head v 28'1, mlcod=0'0)", 2024-09-18T00:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.892623+0000", 2024-09-18T00:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.808525249999999, 2024-09-18T00:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040991170000000002, 2024-09-18T00:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.709 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892623+0000", 2024-09-18T00:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892623+0000", 2024-09-18T00:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892655+0000", 2024-09-18T00:18:40.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1377000000000001e-05 2024-09-18T00:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892819+0000", 2024-09-18T00:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000164275 2024-09-18T00:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892851+0000", 2024-09-18T00:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1696000000000003e-05 2024-09-18T00:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892873+0000", 2024-09-18T00:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.26e-05 2024-09-18T00:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892958+0000", 2024-09-18T00:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4245000000000004e-05 2024-09-18T00:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892975+0000", 2024-09-18T00:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7240000000000001e-05 2024-09-18T00:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.896666+0000", 2024-09-18T00:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003691083 2024-09-18T00:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.896723+0000", 2024-09-18T00:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6601000000000002e-05 2024-09-18T00:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:692 3.0 e29/23 3:03fab4ad:::rbd_data.112ae20746f5.0000000000000057:head v 29'1, mlcod=0'0)", 2024-09-18T00:18:40.713 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.462311761999999, 2024-09-18T00:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074841819999999998, 2024-09-18T00:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238838+0000", 2024-09-18T00:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.187e-06 2024-09-18T00:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238874+0000", 2024-09-18T00:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5380999999999999e-05 2024-09-18T00:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238893+0000", 2024-09-18T00:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999999e-05 2024-09-18T00:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238906+0000", 2024-09-18T00:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3011e-05 2024-09-18T00:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238982+0000", 2024-09-18T00:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6018000000000003e-05 2024-09-18T00:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.239006+0000", 2024-09-18T00:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3683e-05 2024-09-18T00:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246227+0000", 2024-09-18T00:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072210989999999999 2024-09-18T00:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246321+0000", 2024-09-18T00:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4263000000000002e-05 2024-09-18T00:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.717 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:746 3.0 e29/23 3:03157719:::rbd_data.112ae20746f5.0000000000000004:head v 29'2, mlcod=0'0)", 2024-09-18T00:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.392074+0000", 2024-09-18T00:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.309074289000002, 2024-09-18T00:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033355659999999999, 2024-09-18T00:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392074+0000", 2024-09-18T00:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392074+0000", 2024-09-18T00:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392075+0000", 2024-09-18T00:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392101+0000", 2024-09-18T00:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6001999999999999e-05 2024-09-18T00:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392125+0000", 2024-09-18T00:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3771e-05 2024-09-18T00:18:40.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392138+0000", 2024-09-18T00:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3675e-05 2024-09-18T00:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392216+0000", 2024-09-18T00:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8054999999999997e-05 2024-09-18T00:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392245+0000", 2024-09-18T00:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8501999999999999e-05 2024-09-18T00:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395315+0000", 2024-09-18T00:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003069975 2024-09-18T00:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395410+0000", 2024-09-18T00:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5068000000000005e-05 2024-09-18T00:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.722 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:870 3.0 e30/23 3:05335421:::rbd_data.112ae20746f5.0000000000000014:head v 30'5, mlcod=30'3)", 2024-09-18T00:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.955786+0000", 2024-09-18T00:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.745362876000002, 2024-09-18T00:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0044642019999999996, 2024-09-18T00:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955786+0000", 2024-09-18T00:18:40.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955786+0000", 2024-09-18T00:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955787+0000", 2024-09-18T00:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.271e-06 2024-09-18T00:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955824+0000", 2024-09-18T00:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6668999999999997e-05 2024-09-18T00:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955840+0000", 2024-09-18T00:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5977999999999998e-05 2024-09-18T00:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955847+0000", 2024-09-18T00:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1760000000000004e-06 2024-09-18T00:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955929+0000", 2024-09-18T00:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1568000000000002e-05 2024-09-18T00:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955961+0000", 2024-09-18T00:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2873000000000002e-05 2024-09-18T00:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960199+0000", 2024-09-18T00:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042373990000000002 2024-09-18T00:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960250+0000", 2024-09-18T00:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1267999999999998e-05 2024-09-18T00:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2344 3.4 e37/23 3:20ffa5cc:::rbd_data.112aa7ba4578.000000000000021d:head v 37'95, mlcod=36'93)", 2024-09-18T00:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.593553+0000", 2024-09-18T00:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.107595270000001, 2024-09-18T00:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033906819999999999, 2024-09-18T00:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.727 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593553+0000", 2024-09-18T00:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593553+0000", 2024-09-18T00:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593555+0000", 2024-09-18T00:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.643e-06 2024-09-18T00:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593567+0000", 2024-09-18T00:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2289999999999999e-05 2024-09-18T00:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593616+0000", 2024-09-18T00:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8406999999999999e-05 2024-09-18T00:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593629+0000", 2024-09-18T00:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2774e-05 2024-09-18T00:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593696+0000", 2024-09-18T00:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7587000000000002e-05 2024-09-18T00:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593720+0000", 2024-09-18T00:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3638e-05 2024-09-18T00:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.596897+0000", 2024-09-18T00:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003176772 2024-09-18T00:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.596944+0000", 2024-09-18T00:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7571000000000002e-05 2024-09-18T00:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2381 3.0 e37/23 3:01782307:::rbd_data.112aa7ba4578.000000000000004f:head v 37'16, mlcod=35'14)", 2024-09-18T00:18:40.731 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.662312+0000", 2024-09-18T00:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.038836610000001, 2024-09-18T00:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033464179999999999, 2024-09-18T00:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662312+0000", 2024-09-18T00:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662312+0000", 2024-09-18T00:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662313+0000", 2024-09-18T00:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1900000000000001e-07 2024-09-18T00:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662319+0000", 2024-09-18T00:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4670000000000001e-06 2024-09-18T00:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662330+0000", 2024-09-18T00:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.023e-05 2024-09-18T00:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662336+0000", 2024-09-18T00:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.691e-06 2024-09-18T00:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662427+0000", 2024-09-18T00:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0216000000000005e-05 2024-09-18T00:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662448+0000", 2024-09-18T00:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1795000000000001e-05 2024-09-18T00:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.665606+0000", 2024-09-18T00:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031574799999999998 2024-09-18T00:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.665659+0000", 2024-09-18T00:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2719999999999997e-05 2024-09-18T00:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.735 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2460 3.0 e37/23 3:016548b0:::rbd_data.112aa7ba4578.0000000000000018:head v 37'18, mlcod=37'16)", 2024-09-18T00:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.985197+0000", 2024-09-18T00:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.715951721, 2024-09-18T00:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004264894, 2024-09-18T00:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985197+0000", 2024-09-18T00:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985197+0000", 2024-09-18T00:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985198+0000", 2024-09-18T00:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.342e-06 2024-09-18T00:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985210+0000", 2024-09-18T00:18:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1783999999999999e-05 2024-09-18T00:18:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985231+0000", 2024-09-18T00:18:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0769000000000001e-05 2024-09-18T00:18:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985241+0000", 2024-09-18T00:18:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0266e-05 2024-09-18T00:18:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985398+0000", 2024-09-18T00:18:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156701 2024-09-18T00:18:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985447+0000", 2024-09-18T00:18:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9391000000000002e-05 2024-09-18T00:18:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989422+0000", 2024-09-18T00:18:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039745470000000001 2024-09-18T00:18:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989462+0000", 2024-09-18T00:18:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0093999999999999e-05 2024-09-18T00:18:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2469 3.0 e37/23 3:02fdd8b9:::rbd_data.112aa7ba4578.0000000000000044:head v 37'19, mlcod=37'16)", 2024-09-18T00:18:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.989106+0000", 2024-09-18T00:18:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.712042502999999, 2024-09-18T00:18:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004408843, 2024-09-18T00:18:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989106+0000", 2024-09-18T00:18:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989106+0000", 2024-09-18T00:18:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989107+0000", 2024-09-18T00:18:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.088e-06 2024-09-18T00:18:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989115+0000", 2024-09-18T00:18:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8590000000000005e-06 2024-09-18T00:18:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989126+0000", 2024-09-18T00:18:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1144e-05 2024-09-18T00:18:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989134+0000", 2024-09-18T00:18:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4229999999999999e-06 2024-09-18T00:18:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989215+0000", 2024-09-18T00:18:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1551000000000006e-05 2024-09-18T00:18:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989237+0000", 2024-09-18T00:18:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1832999999999999e-05 2024-09-18T00:18:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.993479+0000", 2024-09-18T00:18:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042420119999999999 2024-09-18T00:18:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.993515+0000", 2024-09-18T00:18:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5933000000000003e-05 2024-09-18T00:18:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2471 3.4 e37/23 3:26bbbe4b:::rbd_data.112aa7ba4578.000000000000004e:head v 37'98, mlcod=37'95)", 2024-09-18T00:18:40.744 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.989721+0000", 2024-09-18T00:18:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.711427604000001, 2024-09-18T00:18:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043834369999999996, 2024-09-18T00:18:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989721+0000", 2024-09-18T00:18:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989721+0000", 2024-09-18T00:18:40.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989722+0000", 2024-09-18T00:18:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3300000000000001e-07 2024-09-18T00:18:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989728+0000", 2024-09-18T00:18:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:18:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989741+0000", 2024-09-18T00:18:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3444999999999999e-05 2024-09-18T00:18:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989750+0000", 2024-09-18T00:18:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7369999999999995e-06 2024-09-18T00:18:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989807+0000", 2024-09-18T00:18:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6971000000000001e-05 2024-09-18T00:18:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989826+0000", 2024-09-18T00:18:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8918000000000001e-05 2024-09-18T00:18:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994077+0000", 2024-09-18T00:18:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042507719999999999 2024-09-18T00:18:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994105+0000", 2024-09-18T00:18:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7885000000000002e-05 2024-09-18T00:18:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.748 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2472 3.0 e37/23 3:01782307:::rbd_data.112aa7ba4578.000000000000004f:head v 37'20, mlcod=37'17)", 2024-09-18T00:18:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.990421+0000", 2024-09-18T00:18:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.710727510000002, 2024-09-18T00:18:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00366118, 2024-09-18T00:18:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990421+0000", 2024-09-18T00:18:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990421+0000", 2024-09-18T00:18:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990422+0000", 2024-09-18T00:18:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8599999999999998e-07 2024-09-18T00:18:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990428+0000", 2024-09-18T00:18:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.908e-06 2024-09-18T00:18:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990438+0000", 2024-09-18T00:18:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0244000000000001e-05 2024-09-18T00:18:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990445+0000", 2024-09-18T00:18:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7170000000000002e-06 2024-09-18T00:18:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990492+0000", 2024-09-18T00:18:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7644e-05 2024-09-18T00:18:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990513+0000", 2024-09-18T00:18:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.02e-05 2024-09-18T00:18:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994045+0000", 2024-09-18T00:18:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035321699999999998 2024-09-18T00:18:40.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994082+0000", 2024-09-18T00:18:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7611000000000003e-05 2024-09-18T00:18:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2898 3.9 e38/23 3:956131f6:::rbd_data.112aa7ba4578.000000000000019f:head v 38'23, mlcod=38'21)", 2024-09-18T00:18:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.040063+0000", 2024-09-18T00:18:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.661085442000001, 2024-09-18T00:18:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031556990000000001, 2024-09-18T00:18:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.753 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040063+0000", 2024-09-18T00:18:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040063+0000", 2024-09-18T00:18:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040065+0000", 2024-09-18T00:18:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.84e-06 2024-09-18T00:18:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040076+0000", 2024-09-18T00:18:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1267000000000001e-05 2024-09-18T00:18:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040104+0000", 2024-09-18T00:18:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7406000000000002e-05 2024-09-18T00:18:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040116+0000", 2024-09-18T00:18:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.241e-05 2024-09-18T00:18:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040244+0000", 2024-09-18T00:18:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012744199999999999 2024-09-18T00:18:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040280+0000", 2024-09-18T00:18:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6034000000000001e-05 2024-09-18T00:18:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.043136+0000", 2024-09-18T00:18:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028566170000000001 2024-09-18T00:18:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.043219+0000", 2024-09-18T00:18:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2682999999999994e-05 2024-09-18T00:18:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.757 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2921 3.0 e38/23 3:02fdd8b9:::rbd_data.112aa7ba4578.0000000000000044:head v 38'27, mlcod=38'25)", 2024-09-18T00:18:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.091200+0000", 2024-09-18T00:18:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.609948849999999, 2024-09-18T00:18:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031551589999999998, 2024-09-18T00:18:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.091200+0000", 2024-09-18T00:18:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.091200+0000", 2024-09-18T00:18:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.091202+0000", 2024-09-18T00:18:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2800000000000002e-06 2024-09-18T00:18:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.091253+0000", 2024-09-18T00:18:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0593000000000002e-05 2024-09-18T00:18:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.091287+0000", 2024-09-18T00:18:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3782999999999998e-05 2024-09-18T00:18:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.091301+0000", 2024-09-18T00:18:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4946999999999999e-05 2024-09-18T00:18:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.091443+0000", 2024-09-18T00:18:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014168199999999999 2024-09-18T00:18:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.091509+0000", 2024-09-18T00:18:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5766999999999996e-05 2024-09-18T00:18:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.094270+0000", 2024-09-18T00:18:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027610880000000001 2024-09-18T00:18:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.094355+0000", 2024-09-18T00:18:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5018999999999994e-05 2024-09-18T00:18:40.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2973 3.4 e38/23 3:23ae92e0:::rbd_data.112aa7ba4578.00000000000002c2:head v 38'99, mlcod=37'97)", 2024-09-18T00:18:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.229597+0000", 2024-09-18T00:18:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.471552173999999, 2024-09-18T00:18:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0048943249999999997, 2024-09-18T00:18:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229597+0000", 2024-09-18T00:18:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229597+0000", 2024-09-18T00:18:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229599+0000", 2024-09-18T00:18:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.396e-06 2024-09-18T00:18:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229634+0000", 2024-09-18T00:18:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5237000000000002e-05 2024-09-18T00:18:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229666+0000", 2024-09-18T00:18:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1888999999999999e-05 2024-09-18T00:18:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229681+0000", 2024-09-18T00:18:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4783000000000001e-05 2024-09-18T00:18:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229837+0000", 2024-09-18T00:18:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156193 2024-09-18T00:18:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229899+0000", 2024-09-18T00:18:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1458999999999996e-05 2024-09-18T00:18:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.234405+0000", 2024-09-18T00:18:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045061220000000004 2024-09-18T00:18:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.234491+0000", 2024-09-18T00:18:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6246000000000001e-05 2024-09-18T00:18:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2983 3.9 e38/23 3:93d2c681:::rbd_data.112aa7ba4578.000000000000003d:head v 38'24, mlcod=38'22)", 2024-09-18T00:18:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.258087+0000", 2024-09-18T00:18:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.443061927999999, 2024-09-18T00:18:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0034586859999999999, 2024-09-18T00:18:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.766 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258087+0000", 2024-09-18T00:18:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258087+0000", 2024-09-18T00:18:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258089+0000", 2024-09-18T00:18:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5639999999999999e-06 2024-09-18T00:18:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258196+0000", 2024-09-18T00:18:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106185 2024-09-18T00:18:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258235+0000", 2024-09-18T00:18:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9867000000000003e-05 2024-09-18T00:18:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258257+0000", 2024-09-18T00:18:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1166e-05 2024-09-18T00:18:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258360+0000", 2024-09-18T00:18:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103396 2024-09-18T00:18:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258388+0000", 2024-09-18T00:18:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7773e-05 2024-09-18T00:18:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.261500+0000", 2024-09-18T00:18:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00311233 2024-09-18T00:18:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.261545+0000", 2024-09-18T00:18:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5405000000000003e-05 2024-09-18T00:18:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.770 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:3013 3.4 e38/23 3:23ae92e0:::rbd_data.112aa7ba4578.00000000000002c2:head v 38'101, mlcod=38'99)", 2024-09-18T00:18:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.411123+0000", 2024-09-18T00:18:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.290025341, 2024-09-18T00:18:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003259236, 2024-09-18T00:18:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411123+0000", 2024-09-18T00:18:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411123+0000", 2024-09-18T00:18:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411124+0000", 2024-09-18T00:18:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:18:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411129+0000", 2024-09-18T00:18:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0529999999999997e-06 2024-09-18T00:18:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411138+0000", 2024-09-18T00:18:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8370000000000003e-06 2024-09-18T00:18:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411144+0000", 2024-09-18T00:18:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.852e-06 2024-09-18T00:18:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411201+0000", 2024-09-18T00:18:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6963999999999999e-05 2024-09-18T00:18:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411219+0000", 2024-09-18T00:18:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8780999999999999e-05 2024-09-18T00:18:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.414342+0000", 2024-09-18T00:18:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031231929999999998 2024-09-18T00:18:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.414383+0000", 2024-09-18T00:18:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0139000000000003e-05 2024-09-18T00:18:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:5722 3.4 e41/23 3:20381d86:::rbd_data.112a3a39fc45.000000000000001d:head v 41'107, mlcod=40'105)", 2024-09-18T00:18:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:18:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.365653092000001, 2024-09-18T00:18:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01185172, 2024-09-18T00:18:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.775 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:18:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:18:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335497+0000", 2024-09-18T00:18:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.505e-06 2024-09-18T00:18:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335945+0000", 2024-09-18T00:18:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000447687 2024-09-18T00:18:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335961+0000", 2024-09-18T00:18:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6430999999999999e-05 2024-09-18T00:18:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335969+0000", 2024-09-18T00:18:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:18:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336030+0000", 2024-09-18T00:18:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1255999999999998e-05 2024-09-18T00:18:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336061+0000", 2024-09-18T00:18:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0579e-05 2024-09-18T00:18:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347296+0000", 2024-09-18T00:18:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011235663 2024-09-18T00:18:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347347+0000", 2024-09-18T00:18:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0914000000000001e-05 2024-09-18T00:18:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11443 3.4 3:2361bafb:::rbd_data.112ab1fa102a.00000000000001a9:head [set-alloc-hint object_size 524288 write_size 524288,write 90112~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.239414+0000", 2024-09-18T00:18:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.4617351840000001, 2024-09-18T00:18:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003214681, 2024-09-18T00:18:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11443 2024-09-18T00:18:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.239414+0000", 2024-09-18T00:18:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.239414+0000", 2024-09-18T00:18:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.239417+0000", 2024-09-18T00:18:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7929999999999999e-06 2024-09-18T00:18:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.239447+0000", 2024-09-18T00:18:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9762000000000001e-05 2024-09-18T00:18:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.239490+0000", 2024-09-18T00:18:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2959e-05 2024-09-18T00:18:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.239515+0000", 2024-09-18T00:18:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5038999999999999e-05 2024-09-18T00:18:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.239649+0000", 2024-09-18T00:18:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000133559 2024-09-18T00:18:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.239796+0000", 2024-09-18T00:18:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147675 2024-09-18T00:18:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.242395+0000", 2024-09-18T00:18:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002598712 2024-09-18T00:18:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.242492+0000", 2024-09-18T00:18:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6986999999999998e-05 2024-09-18T00:18:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.242628+0000", 2024-09-18T00:18:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013619500000000001 2024-09-18T00:18:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11457 3.0 3:0214e7b3:::rbd_data.112ab1fa102a.0000000000000150:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:18:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.4412871540000003, 2024-09-18T00:18:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028499165, 2024-09-18T00:18:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11457 2024-09-18T00:18:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:18:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:18:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259865+0000", 2024-09-18T00:18:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7230000000000002e-06 2024-09-18T00:18:40.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259883+0000", 2024-09-18T00:18:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8067000000000002e-05 2024-09-18T00:18:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259924+0000", 2024-09-18T00:18:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0550999999999999e-05 2024-09-18T00:18:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259943+0000", 2024-09-18T00:18:40.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:18:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260054+0000", 2024-09-18T00:18:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011075 2024-09-18T00:18:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260138+0000", 2024-09-18T00:18:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4549000000000007e-05 2024-09-18T00:18:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:40.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.287926+0000", 2024-09-18T00:18:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027787642000000001 2024-09-18T00:18:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288090+0000", 2024-09-18T00:18:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016402499999999999 2024-09-18T00:18:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288361+0000", 2024-09-18T00:18:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027097800000000001 2024-09-18T00:18:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:12086 3.0 3:02534001:::rbd_data.112a7b1589a.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1490944~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:36.337486+0000", 2024-09-18T00:18:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.3636627949999998, 2024-09-18T00:18:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038926289999999999, 2024-09-18T00:18:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 12086 2024-09-18T00:18:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.337486+0000", 2024-09-18T00:18:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.337486+0000", 2024-09-18T00:18:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.337489+0000", 2024-09-18T00:18:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4230000000000001e-06 2024-09-18T00:18:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.337521+0000", 2024-09-18T00:18:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1553000000000001e-05 2024-09-18T00:18:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.337562+0000", 2024-09-18T00:18:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1393000000000002e-05 2024-09-18T00:18:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.337601+0000", 2024-09-18T00:18:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9042000000000003e-05 2024-09-18T00:18:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.337741+0000", 2024-09-18T00:18:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000140118 2024-09-18T00:18:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.338143+0000", 2024-09-18T00:18:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00040189499999999999 2024-09-18T00:18:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.341222+0000", 2024-09-18T00:18:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0030784240000000002 2024-09-18T00:18:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.341291+0000", 2024-09-18T00:18:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9116999999999996e-05 2024-09-18T00:18:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.341379+0000", 2024-09-18T00:18:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7664000000000006e-05 2024-09-18T00:18:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.795 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:40.796 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:18:40.797 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:40.790+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:40.897 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:40.890+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:40.905 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-18T00:18:40.923 INFO:teuthology.orchestra.run.smithi097.stdout:2525 write 0x83f3e4c thru 0x83fce4a (0x8fff bytes) 2024-09-18T00:18:40.956 INFO:teuthology.orchestra.run.smithi097.stdout:2526 trunc from 0xb546a5b to 0xa83528 2024-09-18T00:18:40.970 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:14822 3.5 e53/23 3:a0686808:::rbd_object_map.112a4712e9d2:head v 53'70, mlcod=53'68)", 2024-09-18T00:18:40.970 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:40.969906+0000", 2024-09-18T00:18:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.001090588, 2024-09-18T00:18:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011076739999999999, 2024-09-18T00:18:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.969906+0000", 2024-09-18T00:18:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:40.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.969906+0000", 2024-09-18T00:18:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.969907+0000", 2024-09-18T00:18:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9100000000000003e-07 2024-09-18T00:18:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.969913+0000", 2024-09-18T00:18:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0249999999999999e-06 2024-09-18T00:18:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.969926+0000", 2024-09-18T00:18:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3409e-05 2024-09-18T00:18:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.969933+0000", 2024-09-18T00:18:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1790000000000002e-06 2024-09-18T00:18:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.970050+0000", 2024-09-18T00:18:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000116119 2024-09-18T00:18:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:40.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.970106+0000", 2024-09-18T00:18:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6351999999999999e-05 2024-09-18T00:18:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-18T00:18:40.974 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 1 2024-09-18T00:18:40.974 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:40.987 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:18:40.998 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:40.994+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:41.016 INFO:teuthology.orchestra.run.smithi097.stdout:2529 write 0xed2e09f thru 0xed36377 (0x82d9 bytes) 2024-09-18T00:18:41.024 INFO:teuthology.orchestra.run.smithi097.stdout:2530 read 0x136740e thru 0x13732db (0xbece bytes) 2024-09-18T00:18:41.024 INFO:teuthology.orchestra.run.smithi097.stdout:2531 write 0xaf66eb0 thru 0xaf7300f (0xc160 bytes) 2024-09-18T00:18:41.043 INFO:teuthology.orchestra.run.smithi097.stdout:2532 read 0xe22d902 thru 0xe238319 (0xaa18 bytes) 2024-09-18T00:18:41.043 INFO:teuthology.orchestra.run.smithi097.stdout:2533 read 0xe16c6e thru 0xe24622 (0xd9b5 bytes) 2024-09-18T00:18:41.044 INFO:teuthology.orchestra.run.smithi097.stdout:2534 punch from 0x2832dbf to 0x2837dd1, (0x5012 bytes) 2024-09-18T00:18:41.044 INFO:teuthology.orchestra.run.smithi097.stdout:2535 write 0x92e582f thru 0x92ef791 (0x9f63 bytes) 2024-09-18T00:18:41.054 INFO:teuthology.orchestra.run.smithi097.stdout:2536 punch from 0x9c3edcf to 0x9c42839, (0x3a6a bytes) 2024-09-18T00:18:41.054 INFO:teuthology.orchestra.run.smithi097.stdout:2538 read 0x2ba4a4b thru 0x2ba5984 (0xf3a bytes) 2024-09-18T00:18:41.054 INFO:teuthology.orchestra.run.smithi097.stdout:2539 writesame 0x4e81532 thru 0x4e8e3bc data_size 0x8d(0xce8b bytes) 2024-09-18T00:18:41.061 INFO:teuthology.orchestra.run.smithi097.stdout:2540 write 0x5bbf8d6 thru 0x5bc4139 (0x4864 bytes) 2024-09-18T00:18:41.070 INFO:teuthology.orchestra.run.smithi097.stdout:2541 read 0xd0bd90a thru 0xd0c6176 (0x886d bytes) 2024-09-18T00:18:41.070 INFO:teuthology.orchestra.run.smithi097.stdout:2543 read 0x6136a4f thru 0x6137cc0 (0x1272 bytes) 2024-09-18T00:18:41.070 INFO:teuthology.orchestra.run.smithi097.stdout:2544 write 0x69d5893 thru 0x69d8070 (0x27de bytes) 2024-09-18T00:18:41.084 INFO:teuthology.orchestra.run.smithi097.stdout:2545 trunc from 0xed36378 to 0xa392f41 2024-09-18T00:18:41.098 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:41.094+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:41.103 INFO:teuthology.orchestra.run.smithi097.stdout:2546 read 0x4a115c5 thru 0x4a14176 (0x2bb2 bytes) 2024-09-18T00:18:41.104 INFO:teuthology.orchestra.run.smithi097.stdout:2547 write 0x8688cf5 thru 0x8693589 (0xa895 bytes) 2024-09-18T00:18:41.108 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:18:41.111 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:41.112 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:41.112 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:41.114 INFO:teuthology.orchestra.run.smithi097.stdout:2548 read 0x38bfbf thru 0x398541 (0xc583 bytes) 2024-09-18T00:18:41.117 INFO:teuthology.orchestra.run.smithi097.stdout:2549 punch from 0x8844ce to 0x88a855, (0x6387 bytes) 2024-09-18T00:18:41.117 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:41.127 INFO:teuthology.orchestra.run.smithi097.stdout:2552 trunc from 0xa392f41 to 0x4470d32 2024-09-18T00:18:41.130 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:18:41.157 INFO:teuthology.orchestra.run.smithi097.stdout:2554 write 0x6cd4a19 thru 0x6ce383e (0xee26 bytes) 2024-09-18T00:18:41.179 INFO:teuthology.orchestra.run.smithi097.stdout:2555 writesame 0xad979e9 thru 0xada7897 data_size 0x67(0xfeaf bytes) 2024-09-18T00:18:41.179 INFO:teuthology.orchestra.run.smithi097.stdout:2556 read 0x70c94c6 thru 0x70cc813 (0x334e bytes) 2024-09-18T00:18:41.183 INFO:teuthology.orchestra.run.smithi097.stdout:2557 trunc from 0xada7898 to 0x86ca1bb 2024-09-18T00:18:41.192 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:41.192 INFO:teuthology.orchestra.run.smithi097.stdout:2559 clone 10 order 22 su 262144 sc 6 2024-09-18T00:18:41.199 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:41.194+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:41.268 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:41.268 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:41.269 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:18:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:41.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:599 3.15 3:ac22c048:::rbd_data.112ae20746f5.0000000000000110:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.980706+0000", 2024-09-18T00:18:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.298883998000001, 2024-09-18T00:18:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017399257000000001, 2024-09-18T00:18:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 599 2024-09-18T00:18:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980706+0000", 2024-09-18T00:18:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980706+0000", 2024-09-18T00:18:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980708+0000", 2024-09-18T00:18:41.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5919999999999999e-06 2024-09-18T00:18:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980722+0000", 2024-09-18T00:18:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.364e-05 2024-09-18T00:18:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980752+0000", 2024-09-18T00:18:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9794999999999998e-05 2024-09-18T00:18:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980778+0000", 2024-09-18T00:18:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6494000000000001e-05 2024-09-18T00:18:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980885+0000", 2024-09-18T00:18:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107104 2024-09-18T00:18:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.981079+0000", 2024-09-18T00:18:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019375699999999999 2024-09-18T00:18:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.981866+0000", 2024-09-18T00:18:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00078716099999999998 2024-09-18T00:18:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.997852+0000", 2024-09-18T00:18:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015986305999999999 2024-09-18T00:18:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.997957+0000", 2024-09-18T00:18:41.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104085 2024-09-18T00:18:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.998037+0000", 2024-09-18T00:18:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0098000000000005e-05 2024-09-18T00:18:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.998105+0000", 2024-09-18T00:18:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8225000000000005e-05 2024-09-18T00:18:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:640 3.7 3:e0810bc8:::rbd_data.112ae20746f5.0000000000000019:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.104726+0000", 2024-09-18T00:18:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.174863169999998, 2024-09-18T00:18:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017067268, 2024-09-18T00:18:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 640 2024-09-18T00:18:41.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104726+0000", 2024-09-18T00:18:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104726+0000", 2024-09-18T00:18:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104728+0000", 2024-09-18T00:18:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.336e-06 2024-09-18T00:18:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104735+0000", 2024-09-18T00:18:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6149999999999999e-06 2024-09-18T00:18:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104753+0000", 2024-09-18T00:18:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7362000000000001e-05 2024-09-18T00:18:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104761+0000", 2024-09-18T00:18:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5609999999999999e-06 2024-09-18T00:18:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104842+0000", 2024-09-18T00:18:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0526999999999997e-05 2024-09-18T00:18:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104916+0000", 2024-09-18T00:18:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3724000000000001e-05 2024-09-18T00:18:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.106201+0000", 2024-09-18T00:18:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001285046 2024-09-18T00:18:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121677+0000", 2024-09-18T00:18:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01547665 2024-09-18T00:18:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121696+0000", 2024-09-18T00:18:41.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8304999999999998e-05 2024-09-18T00:18:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121744+0000", 2024-09-18T00:18:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8415000000000003e-05 2024-09-18T00:18:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121794+0000", 2024-09-18T00:18:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9727e-05 2024-09-18T00:18:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.479422928000002, 2024-09-18T00:18:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:18:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:18:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.294 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:18:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:18:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:18:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:18:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:18:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:18:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:18:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:18:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:18:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:18:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:18:41.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:18:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:18:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:18:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:18:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:18:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:41.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:18:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:18:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:18:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:18:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:18:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:18:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.299 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:18:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.463215054999999, 2024-09-18T00:18:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:18:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:41.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:18:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:18:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:18:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:18:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:18:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:18:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:18:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:18:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:18:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:18:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:18:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:18:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:18:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:18:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.303 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:5724 3.1e 3:78324145:::rbd_data.112a3a39fc45.000000000000001e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.341891+0000", 2024-09-18T00:18:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.937698592, 2024-09-18T00:18:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014843063, 2024-09-18T00:18:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.304 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 5724 2024-09-18T00:18:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.341891+0000", 2024-09-18T00:18:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.341891+0000", 2024-09-18T00:18:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.341893+0000", 2024-09-18T00:18:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1129999999999999e-06 2024-09-18T00:18:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343233+0000", 2024-09-18T00:18:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001339614 2024-09-18T00:18:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343251+0000", 2024-09-18T00:18:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8479e-05 2024-09-18T00:18:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343260+0000", 2024-09-18T00:18:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1470000000000007e-06 2024-09-18T00:18:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343333+0000", 2024-09-18T00:18:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2302e-05 2024-09-18T00:18:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343494+0000", 2024-09-18T00:18:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000161593 2024-09-18T00:18:41.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 6", 2024-09-18T00:18:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343778+0000", 2024-09-18T00:18:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028366099999999999 2024-09-18T00:18:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.351688+0000", 2024-09-18T00:18:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0079100739999999996 2024-09-18T00:18:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.356673+0000", 2024-09-18T00:18:41.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049848210000000004 2024-09-18T00:18:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.356706+0000", 2024-09-18T00:18:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3531000000000001e-05 2024-09-18T00:18:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.356734+0000", 2024-09-18T00:18:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7727999999999999e-05 2024-09-18T00:18:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6175 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 8585216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.473970+0000", 2024-09-18T00:18:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.805619219, 2024-09-18T00:18:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013962384, 2024-09-18T00:18:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6175 2024-09-18T00:18:41.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473970+0000", 2024-09-18T00:18:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473970+0000", 2024-09-18T00:18:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473972+0000", 2024-09-18T00:18:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1310000000000001e-06 2024-09-18T00:18:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473977+0000", 2024-09-18T00:18:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1769999999999997e-06 2024-09-18T00:18:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473993+0000", 2024-09-18T00:18:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5988999999999999e-05 2024-09-18T00:18:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474005+0000", 2024-09-18T00:18:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2622999999999999e-05 2024-09-18T00:18:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.485712+0000", 2024-09-18T00:18:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011706701 2024-09-18T00:18:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487289+0000", 2024-09-18T00:18:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0015770280000000001 2024-09-18T00:18:41.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487933+0000", 2024-09-18T00:18:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00064373499999999999 2024-09-18T00:18:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6176 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 9371648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474022+0000", 2024-09-18T00:18:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.805567954000001, 2024-09-18T00:18:41.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014183915, 2024-09-18T00:18:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6176 2024-09-18T00:18:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474022+0000", 2024-09-18T00:18:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474022+0000", 2024-09-18T00:18:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474023+0000", 2024-09-18T00:18:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.02e-06 2024-09-18T00:18:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474028+0000", 2024-09-18T00:18:41.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9599999999999999e-06 2024-09-18T00:18:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474044+0000", 2024-09-18T00:18:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6113999999999999e-05 2024-09-18T00:18:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474056+0000", 2024-09-18T00:18:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2349e-05 2024-09-18T00:18:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487994+0000", 2024-09-18T00:18:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013937748 2024-09-18T00:18:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488088+0000", 2024-09-18T00:18:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3888999999999995e-05 2024-09-18T00:18:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488206+0000", 2024-09-18T00:18:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117835 2024-09-18T00:18:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.318 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6177 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474074+0000", 2024-09-18T00:18:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.805515286, 2024-09-18T00:18:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01434039, 2024-09-18T00:18:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6177 2024-09-18T00:18:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.319 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474074+0000", 2024-09-18T00:18:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474074+0000", 2024-09-18T00:18:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474075+0000", 2024-09-18T00:18:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-18T00:18:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474080+0000", 2024-09-18T00:18:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9459999999999997e-06 2024-09-18T00:18:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474097+0000", 2024-09-18T00:18:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6705999999999999e-05 2024-09-18T00:18:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474110+0000", 2024-09-18T00:18:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2582000000000001e-05 2024-09-18T00:18:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488252+0000", 2024-09-18T00:18:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014142937 2024-09-18T00:18:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488327+0000", 2024-09-18T00:18:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4184000000000007e-05 2024-09-18T00:18:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488415+0000", 2024-09-18T00:18:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8075e-05 2024-09-18T00:18:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6178 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474128+0000", 2024-09-18T00:18:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.805461128999999, 2024-09-18T00:18:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014460381, 2024-09-18T00:18:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.323 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6178 2024-09-18T00:18:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474128+0000", 2024-09-18T00:18:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474128+0000", 2024-09-18T00:18:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474130+0000", 2024-09-18T00:18:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1489999999999999e-06 2024-09-18T00:18:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474135+0000", 2024-09-18T00:18:41.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0830000000000003e-06 2024-09-18T00:18:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474151+0000", 2024-09-18T00:18:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6677e-05 2024-09-18T00:18:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474166+0000", 2024-09-18T00:18:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4871e-05 2024-09-18T00:18:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488458+0000", 2024-09-18T00:18:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014292206999999999 2024-09-18T00:18:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488516+0000", 2024-09-18T00:18:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7627999999999998e-05 2024-09-18T00:18:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488589+0000", 2024-09-18T00:18:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2766000000000001e-05 2024-09-18T00:18:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6179 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 11730944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474183+0000", 2024-09-18T00:18:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.805406375, 2024-09-18T00:18:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014618816, 2024-09-18T00:18:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6179 2024-09-18T00:18:41.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474183+0000", 2024-09-18T00:18:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474183+0000", 2024-09-18T00:18:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474184+0000", 2024-09-18T00:18:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.083e-06 2024-09-18T00:18:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474190+0000", 2024-09-18T00:18:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6570000000000002e-06 2024-09-18T00:18:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474206+0000", 2024-09-18T00:18:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5914e-05 2024-09-18T00:18:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474225+0000", 2024-09-18T00:18:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9369e-05 2024-09-18T00:18:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488639+0000", 2024-09-18T00:18:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014413769 2024-09-18T00:18:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488709+0000", 2024-09-18T00:18:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9655999999999999e-05 2024-09-18T00:18:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488802+0000", 2024-09-18T00:18:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3368000000000005e-05 2024-09-18T00:18:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.332 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6180 3.8 3:16b24e91:::rbd_data.112aa7ba4578.0000000000000161:5 [read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474244+0000", 2024-09-18T00:18:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.805346077999999, 2024-09-18T00:18:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014495918999999999, 2024-09-18T00:18:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6180 2024-09-18T00:18:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474244+0000", 2024-09-18T00:18:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474244+0000", 2024-09-18T00:18:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474245+0000", 2024-09-18T00:18:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.105e-06 2024-09-18T00:18:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474318+0000", 2024-09-18T00:18:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3182999999999994e-05 2024-09-18T00:18:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474345+0000", 2024-09-18T00:18:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6730999999999999e-05 2024-09-18T00:18:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474359+0000", 2024-09-18T00:18:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4433999999999999e-05 2024-09-18T00:18:41.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488626+0000", 2024-09-18T00:18:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014266993 2024-09-18T00:18:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488680+0000", 2024-09-18T00:18:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4453999999999997e-05 2024-09-18T00:18:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488739+0000", 2024-09-18T00:18:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9018999999999999e-05 2024-09-18T00:18:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6181 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 12517376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474384+0000", 2024-09-18T00:18:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.805205142, 2024-09-18T00:18:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014686072999999999, 2024-09-18T00:18:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:41.337 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6181 2024-09-18T00:18:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474384+0000", 2024-09-18T00:18:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474384+0000", 2024-09-18T00:18:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474386+0000", 2024-09-18T00:18:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.688e-06 2024-09-18T00:18:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474392+0000", 2024-09-18T00:18:41.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.575e-06 2024-09-18T00:18:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474409+0000", 2024-09-18T00:18:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7638999999999998e-05 2024-09-18T00:18:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474423+0000", 2024-09-18T00:18:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3237e-05 2024-09-18T00:18:41.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488844+0000", 2024-09-18T00:18:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014421079 2024-09-18T00:18:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488943+0000", 2024-09-18T00:18:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9764000000000002e-05 2024-09-18T00:18:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.489071+0000", 2024-09-18T00:18:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012709099999999999 2024-09-18T00:18:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6280 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 11075584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.478893+0000", 2024-09-18T00:18:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.800696890999999, 2024-09-18T00:18:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012517118000000001, 2024-09-18T00:18:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:41.342 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6280 2024-09-18T00:18:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478893+0000", 2024-09-18T00:18:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478893+0000", 2024-09-18T00:18:41.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478894+0000", 2024-09-18T00:18:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-18T00:18:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478905+0000", 2024-09-18T00:18:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1045e-05 2024-09-18T00:18:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478921+0000", 2024-09-18T00:18:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6469000000000001e-05 2024-09-18T00:18:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478934+0000", 2024-09-18T00:18:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2525e-05 2024-09-18T00:18:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491258+0000", 2024-09-18T00:18:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012324422999999999 2024-09-18T00:18:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491327+0000", 2024-09-18T00:18:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8923000000000004e-05 2024-09-18T00:18:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491410+0000", 2024-09-18T00:18:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2730000000000002e-05 2024-09-18T00:18:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6281 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 11862016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.478952+0000", 2024-09-18T00:18:41.346 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.800637492, 2024-09-18T00:18:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013203609, 2024-09-18T00:18:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6281 2024-09-18T00:18:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478952+0000", 2024-09-18T00:18:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478952+0000", 2024-09-18T00:18:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478953+0000", 2024-09-18T00:18:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0720000000000001e-06 2024-09-18T00:18:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478958+0000", 2024-09-18T00:18:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7369999999999997e-06 2024-09-18T00:18:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.482143+0000", 2024-09-18T00:18:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031850870000000001 2024-09-18T00:18:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.482179+0000", 2024-09-18T00:18:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6381000000000003e-05 2024-09-18T00:18:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.492006+0000", 2024-09-18T00:18:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0098267319999999995 2024-09-18T00:18:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.492069+0000", 2024-09-18T00:18:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2564000000000006e-05 2024-09-18T00:18:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.492156+0000", 2024-09-18T00:18:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7035999999999998e-05 2024-09-18T00:18:41.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6572 3.7 3:e54d9f3a:::rbd_data.112a3a39fc45.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 458184~568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.725733+0000", 2024-09-18T00:18:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.553856331, 2024-09-18T00:18:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016678148, 2024-09-18T00:18:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.351 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6572 2024-09-18T00:18:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.725733+0000", 2024-09-18T00:18:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.725733+0000", 2024-09-18T00:18:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.725736+0000", 2024-09-18T00:18:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2919999999999998e-06 2024-09-18T00:18:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726273+0000", 2024-09-18T00:18:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00053760199999999998 2024-09-18T00:18:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726295+0000", 2024-09-18T00:18:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1855999999999999e-05 2024-09-18T00:18:41.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726305+0000", 2024-09-18T00:18:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0098000000000001e-05 2024-09-18T00:18:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726384+0000", 2024-09-18T00:18:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8731000000000002e-05 2024-09-18T00:18:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726582+0000", 2024-09-18T00:18:41.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019781399999999999 2024-09-18T00:18:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726857+0000", 2024-09-18T00:18:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027557299999999999 2024-09-18T00:18:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.735232+0000", 2024-09-18T00:18:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083744660000000005 2024-09-18T00:18:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:41.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.742317+0000", 2024-09-18T00:18:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0070851789999999996 2024-09-18T00:18:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.742357+0000", 2024-09-18T00:18:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9975000000000003e-05 2024-09-18T00:18:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.742411+0000", 2024-09-18T00:18:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4561999999999997e-05 2024-09-18T00:18:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.356 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.436173735000001, 2024-09-18T00:18:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:18:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:18:41.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:18:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:18:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:18:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:18:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:18:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:18:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:18:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:18:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:18:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:18:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:18:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:18:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:18:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:18:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:18:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:18:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:18:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:18:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:18:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:18:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:18:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:18:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.362 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:18:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.414897183000001, 2024-09-18T00:18:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:18:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:18:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:18:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:18:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:18:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:18:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:18:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:18:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:18:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:18:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:18:41.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:18:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:18:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:18:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:18:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11460 3.19 3:9e578b6c:::rbd_data.112ab1fa102a.0000000000000153:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:18:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.0125257310000002, 2024-09-18T00:18:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041989011999999999, 2024-09-18T00:18:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11460 2024-09-18T00:18:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:18:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:18:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267067+0000", 2024-09-18T00:18:41.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:18:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267088+0000", 2024-09-18T00:18:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0908999999999999e-05 2024-09-18T00:18:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267133+0000", 2024-09-18T00:18:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5232000000000003e-05 2024-09-18T00:18:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267157+0000", 2024-09-18T00:18:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3833e-05 2024-09-18T00:18:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267300+0000", 2024-09-18T00:18:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142282 2024-09-18T00:18:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267447+0000", 2024-09-18T00:18:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147851 2024-09-18T00:18:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308889+0000", 2024-09-18T00:18:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041441779999999998 2024-09-18T00:18:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308956+0000", 2024-09-18T00:18:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7136000000000002e-05 2024-09-18T00:18:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.309053+0000", 2024-09-18T00:18:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6552999999999995e-05 2024-09-18T00:18:41.371 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11464 3.b 3:d331f8ac:::rbd_data.112ab1fa102a.0000000000000157:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:18:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.0075352449999997, 2024-09-18T00:18:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034709218, 2024-09-18T00:18:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.372 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11464 2024-09-18T00:18:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:18:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:18:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272058+0000", 2024-09-18T00:18:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4070000000000002e-06 2024-09-18T00:18:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272074+0000", 2024-09-18T00:18:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5872999999999999e-05 2024-09-18T00:18:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272113+0000", 2024-09-18T00:18:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9224999999999998e-05 2024-09-18T00:18:41.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272135+0000", 2024-09-18T00:18:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2093e-05 2024-09-18T00:18:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272238+0000", 2024-09-18T00:18:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102939 2024-09-18T00:18:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272327+0000", 2024-09-18T00:18:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8746999999999996e-05 2024-09-18T00:18:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.273486+0000", 2024-09-18T00:18:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011588779999999999 2024-09-18T00:18:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306439+0000", 2024-09-18T00:18:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.032953142999999997 2024-09-18T00:18:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306491+0000", 2024-09-18T00:18:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2475000000000002e-05 2024-09-18T00:18:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306648+0000", 2024-09-18T00:18:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015725099999999999 2024-09-18T00:18:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306764+0000", 2024-09-18T00:18:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011518699999999999 2024-09-18T00:18:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.377 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:14346 3.2 3:44bbdafa:::rbd_data.112a13ec2d6c.0000000000000033:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-18T00:18:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:39.282847+0000", 2024-09-18T00:18:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.996742402, 2024-09-18T00:18:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016717600999999999, 2024-09-18T00:18:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:41.378 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 14346 2024-09-18T00:18:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282847+0000", 2024-09-18T00:18:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282847+0000", 2024-09-18T00:18:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282850+0000", 2024-09-18T00:18:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1159999999999999e-06 2024-09-18T00:18:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282867+0000", 2024-09-18T00:18:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6310000000000001e-05 2024-09-18T00:18:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282904+0000", 2024-09-18T00:18:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6952000000000001e-05 2024-09-18T00:18:41.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282921+0000", 2024-09-18T00:18:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7765999999999999e-05 2024-09-18T00:18:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.283061+0000", 2024-09-18T00:18:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013918599999999999 2024-09-18T00:18:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.283385+0000", 2024-09-18T00:18:41.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032432700000000001 2024-09-18T00:18:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.284421+0000", 2024-09-18T00:18:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0010358889999999999 2024-09-18T00:18:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.299281+0000", 2024-09-18T00:18:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014860329 2024-09-18T00:18:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.299336+0000", 2024-09-18T00:18:41.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4926000000000002e-05 2024-09-18T00:18:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.299458+0000", 2024-09-18T00:18:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000122111 2024-09-18T00:18:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.299565+0000", 2024-09-18T00:18:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010668899999999999 2024-09-18T00:18:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.383 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:41.384 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:41.384 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:41.384 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:41.385 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:41.386 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:18:41.386 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:41.386 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:41.387 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:41.387 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:41.387 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:18:41.387 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:18:41.388 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:41.388 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:41.388 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:41.382+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:41.388 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:41.389 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:41.393 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:18:41.393 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:18:41.486 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:41.478+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:41.520 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:41.520 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:41.534 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:18:41.626 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:41.622+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:41.687 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:41.682+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:41.707 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:41.708 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:41.723 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:18:41.772 INFO:teuthology.orchestra.run.smithi097.stdout:truncating image image_client.0-clone9 from 0x86ca1bb (overlap 0xa83528) to 0x541a94 2024-09-18T00:18:41.815 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:41.810+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:41.890 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:41.886+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:41.897 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:41.897 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:41.912 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:18:41.930 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-18T00:18:41.998 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:41.994+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:42.076 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:42.077 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:42.091 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:18:42.091 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:42.086+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:42.192 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:42.186+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:42.259 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:42.259 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:42.272 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:18:42.295 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:42.290+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:42.375 INFO:teuthology.orchestra.run.smithi097.stdout:2560 writesame 0xb11d1ec thru 0xb122697 data_size 0x1(0x54ac bytes) 2024-09-18T00:18:42.376 INFO:teuthology.orchestra.run.smithi097.stdout:2561 read 0x7ed7413 thru 0x7ee6b50 (0xf73e bytes) 2024-09-18T00:18:42.376 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:42.376 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:42.377 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:42.385 INFO:teuthology.orchestra.run.smithi097.stdout:2567 trunc from 0xb122698 to 0x3f01cb8 2024-09-18T00:18:42.394 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:42.390+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:42.405 INFO:teuthology.orchestra.run.smithi097.stdout:2568 read 0x23c2425 thru 0x23c4fd4 (0x2bb0 bytes) 2024-09-18T00:18:42.406 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:42.413 INFO:teuthology.orchestra.run.smithi097.stdout:2570 write 0x44656db thru 0x4472ce8 (0xd60e bytes) 2024-09-18T00:18:42.424 INFO:teuthology.orchestra.run.smithi097.stdout:2571 trunc from 0x4472ce9 to 0x8704d94 2024-09-18T00:18:42.435 INFO:teuthology.orchestra.run.smithi097.stdout:2572 read 0x40bac thru 0x4cd79 (0xc1ce bytes) 2024-09-18T00:18:42.436 INFO:teuthology.orchestra.run.smithi097.stdout:2573 write 0x78194ba thru 0x782949d (0xffe4 bytes) 2024-09-18T00:18:42.437 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:42.437 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:42.452 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:18:42.466 INFO:teuthology.orchestra.run.smithi097.stdout:2574 write 0xb637dea thru 0xb6445a8 (0xc7bf bytes) 2024-09-18T00:18:42.475 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:42.475 INFO:teuthology.orchestra.run.smithi097.stdout:2577 trunc from 0xb6445a9 to 0xb537df0 2024-09-18T00:18:42.488 INFO:teuthology.orchestra.run.smithi097.stdout:2578 write 0x30b1cf9 thru 0x30b459d (0x28a5 bytes) 2024-09-18T00:18:42.491 INFO:teuthology.orchestra.run.smithi097.stdout:2579 write 0x422340a thru 0x423019f (0xcd96 bytes) 2024-09-18T00:18:42.495 INFO:teuthology.orchestra.run.smithi097.stdout:2580 read 0x6e50c5f thru 0x6e5411e (0x34c0 bytes) 2024-09-18T00:18:42.495 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:42.490+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:42.513 INFO:teuthology.orchestra.run.smithi097.stdout:2581 write 0xe521f4e thru 0xe5317f3 (0xf8a6 bytes) 2024-09-18T00:18:42.521 INFO:teuthology.orchestra.run.smithi097.stdout:2582 write 0xbb2cfee thru 0xbb334e7 (0x64fa bytes) 2024-09-18T00:18:42.529 INFO:teuthology.orchestra.run.smithi097.stdout:2583 trunc from 0xe5317f4 to 0xddf69fb 2024-09-18T00:18:42.545 INFO:teuthology.orchestra.run.smithi097.stdout:2584 punch from 0xb7f8861 to 0xb7fb25b, (0x29fa bytes) 2024-09-18T00:18:42.545 INFO:teuthology.orchestra.run.smithi097.stdout:2586 punch from 0x842c2f6 to 0x842ea7f, (0x2789 bytes) 2024-09-18T00:18:42.546 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:42.569 INFO:teuthology.orchestra.run.smithi097.stdout:2588 trunc from 0xddf69fb to 0x4b4b26e 2024-09-18T00:18:42.581 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:42.581 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:42.592 INFO:teuthology.orchestra.run.smithi097.stdout:2589 read 0x14424eb thru 0x144250d (0x23 bytes) 2024-09-18T00:18:42.592 INFO:teuthology.orchestra.run.smithi097.stdout:2590 read 0x4efe6f thru 0x4f496d (0x4aff bytes) 2024-09-18T00:18:42.593 INFO:teuthology.orchestra.run.smithi097.stdout:2591 read 0x2114070 thru 0x2121e96 (0xde27 bytes) 2024-09-18T00:18:42.593 INFO:teuthology.orchestra.run.smithi097.stdout:2592 read 0x54d3af thru 0x553b04 (0x6756 bytes) 2024-09-18T00:18:42.596 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:18:42.596 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:42.590+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:42.611 INFO:teuthology.orchestra.run.smithi097.stdout:2593 trunc from 0x4b4b26e to 0xb44546a 2024-09-18T00:18:42.639 INFO:teuthology.orchestra.run.smithi097.stdout:2594 trunc from 0xb44546a to 0x59747d5 2024-09-18T00:18:42.651 INFO:teuthology.orchestra.run.smithi097.stdout:2595 write 0x295b09b thru 0x295bac3 (0xa29 bytes) 2024-09-18T00:18:42.654 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:18:42.657 INFO:teuthology.orchestra.run.smithi097.stdout:2596 write 0x48c55fb thru 0x48c6ce3 (0x16e9 bytes) 2024-09-18T00:18:42.663 INFO:teuthology.orchestra.run.smithi097.stdout:2597 punch from 0x1435ec7 to 0x1435f2e, (0x67 bytes) 2024-09-18T00:18:42.672 INFO:teuthology.orchestra.run.smithi097.stdout:2598 writesame 0x5f07dae thru 0x5f116ff data_size 0x7d(0x9952 bytes) 2024-09-18T00:18:42.694 INFO:teuthology.orchestra.run.smithi097.stdout:2599 writesame 0xcf1cb4e thru 0xcf25b8a data_size 0x23(0x903d bytes) 2024-09-18T00:18:42.697 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:42.690+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:42.705 INFO:teuthology.orchestra.run.smithi097.stdout:2601 trunc from 0xcf25b8b to 0x6455eb7 2024-09-18T00:18:42.714 INFO:teuthology.orchestra.run.smithi097.stdout:2602 write 0x5083c4d thru 0x508622b (0x25df bytes) 2024-09-18T00:18:42.721 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:42.721 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:42.726 INFO:teuthology.orchestra.run.smithi097.stdout:2603 trunc from 0x6455eb7 to 0x14788d4 2024-09-18T00:18:42.738 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:18:42.797 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:42.794+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:42.803 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:42.806 INFO:teuthology.orchestra.run.smithi097.stdout:2605 trunc from 0x14788d4 to 0x37fc77d 2024-09-18T00:18:42.813 INFO:teuthology.orchestra.run.smithi097.stdout:2607 write 0x117cf97 thru 0x11822a2 (0x530c bytes) 2024-09-18T00:18:42.815 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:42.834 INFO:teuthology.orchestra.run.smithi097.stdout:2610 writesame 0xcb254e8 thru 0xcb2e6c5 data_size 0x1(0x91de bytes) 2024-09-18T00:18:42.843 INFO:teuthology.orchestra.run.smithi097.stdout:2611 writesame 0xea2bf8e thru 0xea300e1 data_size 0x25(0x4154 bytes) 2024-09-18T00:18:42.843 INFO:teuthology.orchestra.run.smithi097.stdout:2612 writesame 0x1dd57b7 thru 0x1de1aae data_size 0x44(0xc2f8 bytes) 2024-09-18T00:18:42.847 INFO:teuthology.orchestra.run.smithi097.stdout:2613 read 0x2bb9352 thru 0x2bc2aba (0x9769 bytes) 2024-09-18T00:18:42.847 INFO:teuthology.orchestra.run.smithi097.stdout:2614 read 0xb240341 thru 0xb2449a5 (0x4665 bytes) 2024-09-18T00:18:42.847 INFO:teuthology.orchestra.run.smithi097.stdout:2616 clone 11 order 20 su 131072 sc 2 2024-09-18T00:18:42.898 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:42.894+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:42.900 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:42.901 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:42.914 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:18:43.002 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:42.998+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:43.084 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:43.084 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:43.098 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:18:43.100 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:43.094+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:43.201 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:43.194+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:43.265 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:43.265 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:43.279 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:18:43.322 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:43.318+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:43.394 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:18:43.403 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:18:43.398+0000 7f94543e7640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26528) UID: 0 2024-09-18T00:18:43.445 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:43.445 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:43.459 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:18:43.505 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:43.502+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:43.547 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:43.548 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:43.548 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:18:43.565 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:43.565 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:43.582 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:43.582 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:43.593 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:43.593 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:43.593 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:43.593 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:43.602 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:43.602 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:43.604 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:43.602+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:43.607 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:18:43.623 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:18:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:18:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is set 2024-09-18T00:18:43.646 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:43.646 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:43.683 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:43.683 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:43.686 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:18:43.687 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:18:43.733 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:43.730+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:43.756 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:43.756 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:43.756 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:43.757 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:43.757 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:43.757 INFO:teuthology.orchestra.run.smithi097.stdout:leaving image image_client.0-clone10 intact 2024-09-18T00:18:43.773 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:18:43.821 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:43.818+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:43.907 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:43.902+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:43.943 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:43.944 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:43.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:43.944 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:43.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.944 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2230 3.17 3:ef492888:::rbd_data.112aa7ba4578.000000000000014f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:18:43.944 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:43.944 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.676778472999999, 2024-09-18T00:18:43.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075065283999999996, 2024-09-18T00:18:43.944 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:43.944 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:43.944 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:43.944 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:43.945 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:43.945 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:43.945 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2230 2024-09-18T00:18:43.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.945 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:43.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:43.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:43.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:43.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:43.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:43.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266014+0000", 2024-09-18T00:18:43.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:18:43.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:43.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266029+0000", 2024-09-18T00:18:43.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4616e-05 2024-09-18T00:18:43.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:43.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266053+0000", 2024-09-18T00:18:43.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4369999999999999e-05 2024-09-18T00:18:43.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:43.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266067+0000", 2024-09-18T00:18:43.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4243999999999999e-05 2024-09-18T00:18:43.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:43.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266157+0000", 2024-09-18T00:18:43.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9909999999999998e-05 2024-09-18T00:18:43.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:43.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266327+0000", 2024-09-18T00:18:43.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016981200000000001 2024-09-18T00:18:43.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:43.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269180+0000", 2024-09-18T00:18:43.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028526559999999999 2024-09-18T00:18:43.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:43.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340955+0000", 2024-09-18T00:18:43.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.071774904 2024-09-18T00:18:43.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:43.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340974+0000", 2024-09-18T00:18:43.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8824000000000001e-05 2024-09-18T00:18:43.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:43.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341028+0000", 2024-09-18T00:18:43.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4456999999999997e-05 2024-09-18T00:18:43.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:43.950 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341076+0000", 2024-09-18T00:18:43.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8284000000000001e-05 2024-09-18T00:18:43.950 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.950 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:43.950 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.950 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2230 3.17 e36/23 ondisk, result = 0)", 2024-09-18T00:18:43.950 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:43.950 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.659434892, 2024-09-18T00:18:43.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057609645000000001, 2024-09-18T00:18:43.950 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:43.950 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:43.950 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:43.950 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:43.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:43.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:43.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:43.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:43.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:43.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283356+0000", 2024-09-18T00:18:43.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.44e-06 2024-09-18T00:18:43.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:43.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283362+0000", 2024-09-18T00:18:43.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2539999999999998e-06 2024-09-18T00:18:43.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:43.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283382+0000", 2024-09-18T00:18:43.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9896000000000001e-05 2024-09-18T00:18:43.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:43.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283398+0000", 2024-09-18T00:18:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5489999999999999e-05 2024-09-18T00:18:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340936+0000", 2024-09-18T00:18:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057538649999999997 2024-09-18T00:18:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340947+0000", 2024-09-18T00:18:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0981e-05 2024-09-18T00:18:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:43.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340964+0000", 2024-09-18T00:18:43.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6934000000000001e-05 2024-09-18T00:18:43.954 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.954 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:43.954 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.954 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2419 3.17 3:ed94b133:::rbd_data.112aa7ba4578.00000000000001b5:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:43.954 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:18:43.954 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.205313418999999, 2024-09-18T00:18:43.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.064761812000000002, 2024-09-18T00:18:43.954 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:43.955 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:43.955 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:43.955 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:43.955 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:43.955 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:43.955 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2419 2024-09-18T00:18:43.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.955 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:43.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:43.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:18:43.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:43.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:18:43.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:43.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737479+0000", 2024-09-18T00:18:43.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6089999999999999e-06 2024-09-18T00:18:43.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:43.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737487+0000", 2024-09-18T00:18:43.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7900000000000005e-06 2024-09-18T00:18:43.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:43.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737507+0000", 2024-09-18T00:18:43.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9936e-05 2024-09-18T00:18:43.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:43.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737518+0000", 2024-09-18T00:18:43.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1046999999999999e-05 2024-09-18T00:18:43.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:43.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737599+0000", 2024-09-18T00:18:43.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0697000000000003e-05 2024-09-18T00:18:43.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:43.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737697+0000", 2024-09-18T00:18:43.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8058000000000002e-05 2024-09-18T00:18:43.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:43.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.740445+0000", 2024-09-18T00:18:43.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027481279999999999 2024-09-18T00:18:43.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:43.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802023+0000", 2024-09-18T00:18:43.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.061577321999999997 2024-09-18T00:18:43.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:43.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802060+0000", 2024-09-18T00:18:43.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7586000000000002e-05 2024-09-18T00:18:43.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:43.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802158+0000", 2024-09-18T00:18:43.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7583000000000004e-05 2024-09-18T00:18:43.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:43.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802238+0000", 2024-09-18T00:18:43.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0055999999999994e-05 2024-09-18T00:18:43.960 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2419 3.17 e37/23 ondisk, result = 0)", 2024-09-18T00:18:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:18:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.193088473, 2024-09-18T00:18:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052342948, 2024-09-18T00:18:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:43.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:43.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:18:43.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:43.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:18:43.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:43.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749702+0000", 2024-09-18T00:18:43.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:18:43.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:43.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749710+0000", 2024-09-18T00:18:43.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7279999999999995e-06 2024-09-18T00:18:43.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:43.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749736+0000", 2024-09-18T00:18:43.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5797999999999999e-05 2024-09-18T00:18:43.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:43.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749751+0000", 2024-09-18T00:18:43.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5238999999999999e-05 2024-09-18T00:18:43.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:43.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801745+0000", 2024-09-18T00:18:43.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051994022000000001 2024-09-18T00:18:43.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:43.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801770+0000", 2024-09-18T00:18:43.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5103e-05 2024-09-18T00:18:43.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:43.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802044+0000", 2024-09-18T00:18:43.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027395799999999998 2024-09-18T00:18:43.965 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.965 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:43.965 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.965 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6503 3.1b 3:d9387a1f:::rbd_data.112a3a39fc45.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:43.965 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:18:43.965 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.378415502999999, 2024-09-18T00:18:43.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050510547000000003, 2024-09-18T00:18:43.965 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:43.965 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:43.965 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:43.965 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:43.966 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:43.966 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:43.966 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6503 2024-09-18T00:18:43.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.966 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:43.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:43.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:18:43.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:43.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:18:43.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:43.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564376+0000", 2024-09-18T00:18:43.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-06 2024-09-18T00:18:43.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:43.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564386+0000", 2024-09-18T00:18:43.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0312999999999999e-05 2024-09-18T00:18:43.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:43.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564408+0000", 2024-09-18T00:18:43.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2001000000000002e-05 2024-09-18T00:18:43.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:43.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564421+0000", 2024-09-18T00:18:43.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2271e-05 2024-09-18T00:18:43.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:43.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564491+0000", 2024-09-18T00:18:43.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0630000000000006e-05 2024-09-18T00:18:43.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:43.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564606+0000", 2024-09-18T00:18:43.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011449 2024-09-18T00:18:43.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:43.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.566743+0000", 2024-09-18T00:18:43.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021373830000000001 2024-09-18T00:18:43.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:43.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614738+0000", 2024-09-18T00:18:43.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047995359000000001 2024-09-18T00:18:43.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:43.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614759+0000", 2024-09-18T00:18:43.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1010999999999999e-05 2024-09-18T00:18:43.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:43.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614826+0000", 2024-09-18T00:18:43.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6654000000000003e-05 2024-09-18T00:18:43.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:43.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614884+0000", 2024-09-18T00:18:43.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8332999999999999e-05 2024-09-18T00:18:43.971 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.971 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:43.971 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.971 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6503 3.1b e42/23 ondisk, result = 0)", 2024-09-18T00:18:43.971 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:18:43.971 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.369156035, 2024-09-18T00:18:43.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041115407, 2024-09-18T00:18:43.971 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:43.971 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:43.972 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:43.972 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:43.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:43.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:18:43.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:43.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:18:43.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:43.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573635+0000", 2024-09-18T00:18:43.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5689999999999999e-06 2024-09-18T00:18:43.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:43.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573643+0000", 2024-09-18T00:18:43.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9699999999999999e-06 2024-09-18T00:18:43.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:43.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573668+0000", 2024-09-18T00:18:43.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5536000000000001e-05 2024-09-18T00:18:43.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:43.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573683+0000", 2024-09-18T00:18:43.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4025e-05 2024-09-18T00:18:43.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:43.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614717+0000", 2024-09-18T00:18:43.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041034517 2024-09-18T00:18:43.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:43.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614730+0000", 2024-09-18T00:18:43.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2945999999999999e-05 2024-09-18T00:18:43.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:43.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614749+0000", 2024-09-18T00:18:43.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8844000000000001e-05 2024-09-18T00:18:43.975 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.975 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:43.975 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.975 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6518 3.1 3:8061bf9f:::rbd_data.112a3a39fc45.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:43.975 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:18:43.975 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.291977229, 2024-09-18T00:18:43.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062303123000000002, 2024-09-18T00:18:43.976 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:43.976 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:43.976 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:43.976 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:43.976 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:43.976 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:43.976 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6518 2024-09-18T00:18:43.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.976 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:43.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:43.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:18:43.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:43.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:18:43.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:43.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650814+0000", 2024-09-18T00:18:43.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.629e-06 2024-09-18T00:18:43.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:43.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650823+0000", 2024-09-18T00:18:43.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1800000000000002e-06 2024-09-18T00:18:43.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:43.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650843+0000", 2024-09-18T00:18:43.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9836000000000001e-05 2024-09-18T00:18:43.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:43.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650854+0000", 2024-09-18T00:18:43.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1564e-05 2024-09-18T00:18:43.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:43.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650917+0000", 2024-09-18T00:18:43.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3076000000000004e-05 2024-09-18T00:18:43.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:43.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.651029+0000", 2024-09-18T00:18:43.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111154 2024-09-18T00:18:43.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:43.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.655531+0000", 2024-09-18T00:18:43.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045027629999999999 2024-09-18T00:18:43.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:43.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712924+0000", 2024-09-18T00:18:43.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057393006000000003 2024-09-18T00:18:43.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:43.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712988+0000", 2024-09-18T00:18:43.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000005e-05 2024-09-18T00:18:43.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:43.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713056+0000", 2024-09-18T00:18:43.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7904999999999994e-05 2024-09-18T00:18:43.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:43.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713115+0000", 2024-09-18T00:18:43.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9339999999999998e-05 2024-09-18T00:18:43.981 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.981 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:43.981 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.981 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6518 3.1 e42/23 ondisk, result = 0)", 2024-09-18T00:18:43.981 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:18:43.981 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.279552146, 2024-09-18T00:18:43.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049887304, 2024-09-18T00:18:43.982 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:43.982 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:43.982 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:43.982 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:43.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:43.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:18:43.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:43.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:18:43.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:43.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663239+0000", 2024-09-18T00:18:43.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:18:43.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:43.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663246+0000", 2024-09-18T00:18:43.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4189999999999996e-06 2024-09-18T00:18:43.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:43.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663265+0000", 2024-09-18T00:18:43.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8323000000000001e-05 2024-09-18T00:18:43.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:43.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663277+0000", 2024-09-18T00:18:43.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2065999999999999e-05 2024-09-18T00:18:43.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:43.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712965+0000", 2024-09-18T00:18:43.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049688317000000003 2024-09-18T00:18:43.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:43.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712977+0000", 2024-09-18T00:18:43.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.206e-05 2024-09-18T00:18:43.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:43.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713125+0000", 2024-09-18T00:18:43.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014749899999999999 2024-09-18T00:18:43.985 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.985 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:43.985 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.986 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:8831 3.12 3:49cac8cf:::rbd_data.112a9f4259df.0000000000000130:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T00:18:43.986 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:18:43.986 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 11.661691897000001, 2024-09-18T00:18:43.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104125047, 2024-09-18T00:18:43.986 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:43.986 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:43.986 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:43.986 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:43.986 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:43.986 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:43.986 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 8831 2024-09-18T00:18:43.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.987 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:43.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:43.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:18:43.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:43.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:18:43.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:43.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281099+0000", 2024-09-18T00:18:43.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.029e-06 2024-09-18T00:18:43.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:43.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281103+0000", 2024-09-18T00:18:43.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7759999999999997e-06 2024-09-18T00:18:43.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:43.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281114+0000", 2024-09-18T00:18:43.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1090000000000001e-05 2024-09-18T00:18:43.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:43.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281120+0000", 2024-09-18T00:18:43.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.604e-06 2024-09-18T00:18:43.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:43.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281164+0000", 2024-09-18T00:18:43.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3696999999999999e-05 2024-09-18T00:18:43.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:43.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281270+0000", 2024-09-18T00:18:43.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106041 2024-09-18T00:18:43.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:43.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.285596+0000", 2024-09-18T00:18:43.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043266659999999998 2024-09-18T00:18:43.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:43.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385028+0000", 2024-09-18T00:18:43.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.099431808999999996 2024-09-18T00:18:43.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:43.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385094+0000", 2024-09-18T00:18:43.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5740000000000004e-05 2024-09-18T00:18:43.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:43.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385166+0000", 2024-09-18T00:18:43.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1637000000000005e-05 2024-09-18T00:18:43.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:43.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385223+0000", 2024-09-18T00:18:43.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6957999999999999e-05 2024-09-18T00:18:43.992 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.992 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:43.992 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.992 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:8831 3.12 e48/32 ondisk, result = 0)", 2024-09-18T00:18:43.992 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:18:43.992 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 11.645177787, 2024-09-18T00:18:43.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087621558000000002, 2024-09-18T00:18:43.992 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:43.992 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:43.992 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:43.992 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:43.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:43.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:18:43.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:43.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:18:43.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:43.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297613+0000", 2024-09-18T00:18:43.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3400000000000001e-06 2024-09-18T00:18:43.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:43.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297620+0000", 2024-09-18T00:18:43.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5869999999999998e-06 2024-09-18T00:18:43.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:43.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297639+0000", 2024-09-18T00:18:43.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9888999999999999e-05 2024-09-18T00:18:43.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:43.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297661+0000", 2024-09-18T00:18:43.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1772000000000001e-05 2024-09-18T00:18:43.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:43.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385073+0000", 2024-09-18T00:18:43.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087411303999999995 2024-09-18T00:18:43.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:43.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385086+0000", 2024-09-18T00:18:43.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3040000000000001e-05 2024-09-18T00:18:43.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:43.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385233+0000", 2024-09-18T00:18:43.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014762600000000001 2024-09-18T00:18:43.996 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.996 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:43.996 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:43.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.996 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10099 3.6 3:62e7f20b:::rbd_data.112a9f4259df.0000000000000001:8 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:43.996 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.572938+0000", 2024-09-18T00:18:43.996 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.369851147, 2024-09-18T00:18:43.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026138630999999999, 2024-09-18T00:18:43.997 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:43.997 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:43.997 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:43.997 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:43.997 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:43.997 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:43.997 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10099 2024-09-18T00:18:43.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.997 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:43.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:43.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572938+0000", 2024-09-18T00:18:43.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:43.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572938+0000", 2024-09-18T00:18:43.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:43.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:43.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572939+0000", 2024-09-18T00:18:43.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7400000000000003e-07 2024-09-18T00:18:43.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:43.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572942+0000", 2024-09-18T00:18:43.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7159999999999999e-06 2024-09-18T00:18:43.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:43.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572949+0000", 2024-09-18T00:18:43.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7379999999999994e-06 2024-09-18T00:18:43.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:43.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:43.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:43.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572955+0000", 2024-09-18T00:18:43.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7860000000000002e-06 2024-09-18T00:18:44.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.598855+0000", 2024-09-18T00:18:44.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025900175000000001 2024-09-18T00:18:44.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.598939+0000", 2024-09-18T00:18:44.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3244000000000005e-05 2024-09-18T00:18:44.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599077+0000", 2024-09-18T00:18:44.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138398 2024-09-18T00:18:44.001 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.001 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.001 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.001 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10108 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:44.001 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.572963+0000", 2024-09-18T00:18:44.001 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.369826703999999, 2024-09-18T00:18:44.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026310093999999999, 2024-09-18T00:18:44.001 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.001 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.001 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:44.002 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:44.002 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:44.002 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:44.002 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10108 2024-09-18T00:18:44.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.002 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572963+0000", 2024-09-18T00:18:44.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572963+0000", 2024-09-18T00:18:44.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572963+0000", 2024-09-18T00:18:44.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7299999999999996e-07 2024-09-18T00:18:44.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572966+0000", 2024-09-18T00:18:44.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4389999999999999e-06 2024-09-18T00:18:44.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572973+0000", 2024-09-18T00:18:44.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6520000000000006e-06 2024-09-18T00:18:44.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572980+0000", 2024-09-18T00:18:44.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:18:44.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599154+0000", 2024-09-18T00:18:44.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026174096000000001 2024-09-18T00:18:44.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599196+0000", 2024-09-18T00:18:44.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1993000000000003e-05 2024-09-18T00:18:44.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599273+0000", 2024-09-18T00:18:44.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7144999999999994e-05 2024-09-18T00:18:44.005 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.006 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.006 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.006 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10109 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:44.006 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:18:44.006 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.369802226999999, 2024-09-18T00:18:44.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026513858000000001, 2024-09-18T00:18:44.006 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.006 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.006 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:44.006 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:44.006 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:44.007 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:44.007 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10109 2024-09-18T00:18:44.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.007 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:18:44.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:18:44.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572988+0000", 2024-09-18T00:18:44.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1099999999999996e-07 2024-09-18T00:18:44.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572990+0000", 2024-09-18T00:18:44.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3329999999999999e-06 2024-09-18T00:18:44.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572998+0000", 2024-09-18T00:18:44.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2039999999999997e-06 2024-09-18T00:18:44.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573012+0000", 2024-09-18T00:18:44.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3879e-05 2024-09-18T00:18:44.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599312+0000", 2024-09-18T00:18:44.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026300093 2024-09-18T00:18:44.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599380+0000", 2024-09-18T00:18:44.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7401000000000001e-05 2024-09-18T00:18:44.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599501+0000", 2024-09-18T00:18:44.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000121437 2024-09-18T00:18:44.010 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.010 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.010 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.010 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10110 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:44.010 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:18:44.011 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.369768768, 2024-09-18T00:18:44.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026746066999999998, 2024-09-18T00:18:44.011 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.011 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.011 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:44.011 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:44.011 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:44.011 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:44.011 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10110 2024-09-18T00:18:44.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.011 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:18:44.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:18:44.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:18:44.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2799999999999996e-07 2024-09-18T00:18:44.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573030+0000", 2024-09-18T00:18:44.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5469999999999997e-06 2024-09-18T00:18:44.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573039+0000", 2024-09-18T00:18:44.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6080000000000002e-06 2024-09-18T00:18:44.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573049+0000", 2024-09-18T00:18:44.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0759999999999992e-06 2024-09-18T00:18:44.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599539+0000", 2024-09-18T00:18:44.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026490952000000002 2024-09-18T00:18:44.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599621+0000", 2024-09-18T00:18:44.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1601999999999996e-05 2024-09-18T00:18:44.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599767+0000", 2024-09-18T00:18:44.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145654 2024-09-18T00:18:44.015 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.015 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.015 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.015 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10111 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:44.015 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:18:44.015 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.369730553, 2024-09-18T00:18:44.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027006651999999999, 2024-09-18T00:18:44.016 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.016 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.016 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:44.016 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:44.016 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:44.016 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:44.016 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10111 2024-09-18T00:18:44.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.016 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:18:44.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:18:44.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:18:44.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9500000000000002e-07 2024-09-18T00:18:44.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573062+0000", 2024-09-18T00:18:44.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6539999999999999e-06 2024-09-18T00:18:44.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573071+0000", 2024-09-18T00:18:44.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5380000000000002e-06 2024-09-18T00:18:44.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573077+0000", 2024-09-18T00:18:44.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999999e-06 2024-09-18T00:18:44.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599818+0000", 2024-09-18T00:18:44.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02674093 2024-09-18T00:18:44.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599923+0000", 2024-09-18T00:18:44.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010490200000000001 2024-09-18T00:18:44.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600066+0000", 2024-09-18T00:18:44.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014284500000000001 2024-09-18T00:18:44.019 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.019 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.019 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.020 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10112 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:44.020 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:18:44.020 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.369704758999999, 2024-09-18T00:18:44.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027148835, 2024-09-18T00:18:44.020 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.020 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.020 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:44.020 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:44.020 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:44.020 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:44.020 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10112 2024-09-18T00:18:44.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.021 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:18:44.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:18:44.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:18:44.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9500000000000003e-07 2024-09-18T00:18:44.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573088+0000", 2024-09-18T00:18:44.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9569999999999998e-06 2024-09-18T00:18:44.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573097+0000", 2024-09-18T00:18:44.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8599999999999999e-06 2024-09-18T00:18:44.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573104+0000", 2024-09-18T00:18:44.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5710000000000003e-06 2024-09-18T00:18:44.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600091+0000", 2024-09-18T00:18:44.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026987315000000001 2024-09-18T00:18:44.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600158+0000", 2024-09-18T00:18:44.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6727e-05 2024-09-18T00:18:44.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600234+0000", 2024-09-18T00:18:44.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5909999999999997e-05 2024-09-18T00:18:44.024 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.024 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.024 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.024 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10113 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:44.024 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:18:44.024 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.369660501, 2024-09-18T00:18:44.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027356020000000002, 2024-09-18T00:18:44.025 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.025 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.025 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:44.025 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:44.025 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:44.025 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:44.025 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10113 2024-09-18T00:18:44.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.025 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:18:44.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:18:44.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573130+0000", 2024-09-18T00:18:44.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1999999999999999e-07 2024-09-18T00:18:44.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573133+0000", 2024-09-18T00:18:44.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:18:44.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573141+0000", 2024-09-18T00:18:44.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5110000000000003e-06 2024-09-18T00:18:44.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573155+0000", 2024-09-18T00:18:44.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3730000000000001e-05 2024-09-18T00:18:44.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600275+0000", 2024-09-18T00:18:44.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027119621999999999 2024-09-18T00:18:44.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600349+0000", 2024-09-18T00:18:44.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4624999999999998e-05 2024-09-18T00:18:44.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600485+0000", 2024-09-18T00:18:44.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013570499999999999 2024-09-18T00:18:44.028 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.028 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.028 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.029 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10114 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:44.029 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:18:44.029 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.369625138, 2024-09-18T00:18:44.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027471173000000002, 2024-09-18T00:18:44.029 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.029 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.029 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:44.029 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:44.029 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:44.029 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:44.029 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10114 2024-09-18T00:18:44.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.029 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:18:44.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:18:44.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573165+0000", 2024-09-18T00:18:44.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.99e-07 2024-09-18T00:18:44.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573168+0000", 2024-09-18T00:18:44.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8339999999999999e-06 2024-09-18T00:18:44.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573176+0000", 2024-09-18T00:18:44.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0199999999999994e-06 2024-09-18T00:18:44.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573182+0000", 2024-09-18T00:18:44.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.455e-06 2024-09-18T00:18:44.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600521+0000", 2024-09-18T00:18:44.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027338989000000001 2024-09-18T00:18:44.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600577+0000", 2024-09-18T00:18:44.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5717000000000002e-05 2024-09-18T00:18:44.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600635+0000", 2024-09-18T00:18:44.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8559e-05 2024-09-18T00:18:44.033 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.033 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.033 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.033 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10115 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:44.033 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:18:44.033 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.369572261, 2024-09-18T00:18:44.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027643300999999999, 2024-09-18T00:18:44.033 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.033 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.033 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:44.034 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:44.034 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:44.034 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:44.034 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10115 2024-09-18T00:18:44.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.034 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:18:44.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:18:44.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573218+0000", 2024-09-18T00:18:44.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-18T00:18:44.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573221+0000", 2024-09-18T00:18:44.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3079999999999999e-06 2024-09-18T00:18:44.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573233+0000", 2024-09-18T00:18:44.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1544e-05 2024-09-18T00:18:44.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573240+0000", 2024-09-18T00:18:44.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4070000000000004e-06 2024-09-18T00:18:44.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600665+0000", 2024-09-18T00:18:44.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02742522 2024-09-18T00:18:44.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600745+0000", 2024-09-18T00:18:44.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0014999999999999e-05 2024-09-18T00:18:44.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600860+0000", 2024-09-18T00:18:44.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115127 2024-09-18T00:18:44.037 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.037 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.037 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.037 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11459 3.1b 3:db862ee8:::rbd_data.112ab1fa102a.0000000000000152:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:44.037 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:18:44.038 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.6811434500000004, 2024-09-18T00:18:44.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.032246431999999998, 2024-09-18T00:18:44.038 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.038 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.038 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.038 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:44.038 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:44.038 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:44.038 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11459 2024-09-18T00:18:44.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.038 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:18:44.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:18:44.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261649+0000", 2024-09-18T00:18:44.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1200000000000002e-06 2024-09-18T00:18:44.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261665+0000", 2024-09-18T00:18:44.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5668999999999999e-05 2024-09-18T00:18:44.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261700+0000", 2024-09-18T00:18:44.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4925000000000002e-05 2024-09-18T00:18:44.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261723+0000", 2024-09-18T00:18:44.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3189000000000001e-05 2024-09-18T00:18:44.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261815+0000", 2024-09-18T00:18:44.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1662000000000004e-05 2024-09-18T00:18:44.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261904+0000", 2024-09-18T00:18:44.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9201000000000002e-05 2024-09-18T00:18:44.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:44.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.262955+0000", 2024-09-18T00:18:44.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00105156 2024-09-18T00:18:44.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:44.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293760+0000", 2024-09-18T00:18:44.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030804292 2024-09-18T00:18:44.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:44.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293773+0000", 2024-09-18T00:18:44.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3361e-05 2024-09-18T00:18:44.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293833+0000", 2024-09-18T00:18:44.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9626000000000001e-05 2024-09-18T00:18:44.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293892+0000", 2024-09-18T00:18:44.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9827000000000002e-05 2024-09-18T00:18:44.043 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.043 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.043 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.043 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.043 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.043 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:44.044 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2024-09-18T00:18:44.044 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:18:44.044 INFO:tasks.thrashosds.thrasher:Killing osd 5, live_osds are [0, 2, 3, 4, 5, 6, 7] 2024-09-18T00:18:44.044 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-18T00:18:44.044 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T00:18:44.046 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:18:44.046 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:44.042+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:44.100 INFO:tasks.ceph.osd.5:Stopped 2024-09-18T00:18:44.100 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.5 2024-09-18T00:18:44.100 INFO:tasks.thrashosds.thrasher:remote for osd 5 is ubuntu@smithi114.front.sepia.ceph.com 2024-09-18T00:18:44.100 INFO:tasks.thrashosds.thrasher:remote for exp osd 1 is ubuntu@smithi097.front.sepia.ceph.com 2024-09-18T00:18:44.101 DEBUG:teuthology.orchestra.run.smithi097:> type ceph-objectstore-tool 2024-09-18T00:18:44.145 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T00:18:44.145 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op list-pgs 2024-09-18T00:18:44.189 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:44.186+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:44.224 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:44.224 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:44.224 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:44.238 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:18:44.281 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:44.278+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:44.349 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:44.346+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:44.401 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:44.402 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:44.402 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:44.402 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:44.402 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:44.416 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:18:44.461 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:44.458+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:44.550 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:44.546+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:44.588 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:44.588 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:44.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:44.589 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:44.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.589 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4263.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-18T00:18:44.589 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:17:55.481299+0000", 2024-09-18T00:18:44.589 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.106244066999999, 2024-09-18T00:18:44.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036304219999999999, 2024-09-18T00:18:44.589 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.589 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.589 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.590 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481299+0000", 2024-09-18T00:18:44.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481299+0000", 2024-09-18T00:18:44.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481300+0000", 2024-09-18T00:18:44.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3e-07 2024-09-18T00:18:44.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481307+0000", 2024-09-18T00:18:44.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4959999999999999e-06 2024-09-18T00:18:44.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481330+0000", 2024-09-18T00:18:44.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3363999999999999e-05 2024-09-18T00:18:44.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481347+0000", 2024-09-18T00:18:44.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6263e-05 2024-09-18T00:18:44.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481428+0000", 2024-09-18T00:18:44.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1612000000000004e-05 2024-09-18T00:18:44.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.481495+0000", 2024-09-18T00:18:44.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6439999999999994e-05 2024-09-18T00:18:44.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.484828+0000", 2024-09-18T00:18:44.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033337140000000002 2024-09-18T00:18:44.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:17:55.484929+0000", 2024-09-18T00:18:44.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000100803 2024-09-18T00:18:44.593 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.593 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.593 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.593 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:285 3.9 e28/23 3:94c9a3ca:::rbd_data.112a45c8e167.000000000000001c:head v 28'1, mlcod=0'0)", 2024-09-18T00:18:44.594 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:10.892623+0000", 2024-09-18T00:18:44.594 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 33.694919370000001, 2024-09-18T00:18:44.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040991170000000002, 2024-09-18T00:18:44.594 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.594 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.594 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.594 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892623+0000", 2024-09-18T00:18:44.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892623+0000", 2024-09-18T00:18:44.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892655+0000", 2024-09-18T00:18:44.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1377000000000001e-05 2024-09-18T00:18:44.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892819+0000", 2024-09-18T00:18:44.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000164275 2024-09-18T00:18:44.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892851+0000", 2024-09-18T00:18:44.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1696000000000003e-05 2024-09-18T00:18:44.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892873+0000", 2024-09-18T00:18:44.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.26e-05 2024-09-18T00:18:44.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892958+0000", 2024-09-18T00:18:44.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4245000000000004e-05 2024-09-18T00:18:44.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.892975+0000", 2024-09-18T00:18:44.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7240000000000001e-05 2024-09-18T00:18:44.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.896666+0000", 2024-09-18T00:18:44.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003691083 2024-09-18T00:18:44.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:10.896723+0000", 2024-09-18T00:18:44.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6601000000000002e-05 2024-09-18T00:18:44.597 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.598 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.598 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.598 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:692 3.0 e29/23 3:03fab4ad:::rbd_data.112ae20746f5.0000000000000057:head v 29'1, mlcod=0'0)", 2024-09-18T00:18:44.598 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:44.598 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.348705881999997, 2024-09-18T00:18:44.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074841819999999998, 2024-09-18T00:18:44.598 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.598 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.598 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.598 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:44.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:44.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238838+0000", 2024-09-18T00:18:44.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.187e-06 2024-09-18T00:18:44.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238874+0000", 2024-09-18T00:18:44.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5380999999999999e-05 2024-09-18T00:18:44.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238893+0000", 2024-09-18T00:18:44.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999999e-05 2024-09-18T00:18:44.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238906+0000", 2024-09-18T00:18:44.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3011e-05 2024-09-18T00:18:44.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238982+0000", 2024-09-18T00:18:44.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6018000000000003e-05 2024-09-18T00:18:44.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.239006+0000", 2024-09-18T00:18:44.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3683e-05 2024-09-18T00:18:44.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246227+0000", 2024-09-18T00:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072210989999999999 2024-09-18T00:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246321+0000", 2024-09-18T00:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4263000000000002e-05 2024-09-18T00:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:746 3.0 e29/23 3:03157719:::rbd_data.112ae20746f5.0000000000000004:head v 29'2, mlcod=0'0)", 2024-09-18T00:18:44.602 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.392074+0000", 2024-09-18T00:18:44.603 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.195468409, 2024-09-18T00:18:44.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033355659999999999, 2024-09-18T00:18:44.603 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.603 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.603 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.603 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392074+0000", 2024-09-18T00:18:44.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392074+0000", 2024-09-18T00:18:44.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392075+0000", 2024-09-18T00:18:44.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1799999999999995e-07 2024-09-18T00:18:44.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392101+0000", 2024-09-18T00:18:44.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6001999999999999e-05 2024-09-18T00:18:44.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392125+0000", 2024-09-18T00:18:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3771e-05 2024-09-18T00:18:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392138+0000", 2024-09-18T00:18:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3675e-05 2024-09-18T00:18:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392216+0000", 2024-09-18T00:18:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8054999999999997e-05 2024-09-18T00:18:44.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.392245+0000", 2024-09-18T00:18:44.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8501999999999999e-05 2024-09-18T00:18:44.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395315+0000", 2024-09-18T00:18:44.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003069975 2024-09-18T00:18:44.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.395410+0000", 2024-09-18T00:18:44.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5068000000000005e-05 2024-09-18T00:18:44.606 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.607 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.607 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.607 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:870 3.0 e30/23 3:05335421:::rbd_data.112ae20746f5.0000000000000014:head v 30'5, mlcod=30'3)", 2024-09-18T00:18:44.607 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.955786+0000", 2024-09-18T00:18:44.607 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.631756996, 2024-09-18T00:18:44.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0044642019999999996, 2024-09-18T00:18:44.607 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.607 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.607 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.607 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955786+0000", 2024-09-18T00:18:44.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955786+0000", 2024-09-18T00:18:44.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955787+0000", 2024-09-18T00:18:44.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.271e-06 2024-09-18T00:18:44.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955824+0000", 2024-09-18T00:18:44.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6668999999999997e-05 2024-09-18T00:18:44.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955840+0000", 2024-09-18T00:18:44.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5977999999999998e-05 2024-09-18T00:18:44.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955847+0000", 2024-09-18T00:18:44.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1760000000000004e-06 2024-09-18T00:18:44.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955929+0000", 2024-09-18T00:18:44.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1568000000000002e-05 2024-09-18T00:18:44.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.955961+0000", 2024-09-18T00:18:44.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2873000000000002e-05 2024-09-18T00:18:44.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960199+0000", 2024-09-18T00:18:44.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042373990000000002 2024-09-18T00:18:44.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.960250+0000", 2024-09-18T00:18:44.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1267999999999998e-05 2024-09-18T00:18:44.611 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.611 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.611 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.611 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2344 3.4 e37/23 3:20ffa5cc:::rbd_data.112aa7ba4578.000000000000021d:head v 37'95, mlcod=36'93)", 2024-09-18T00:18:44.611 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.593553+0000", 2024-09-18T00:18:44.611 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.993989389999999, 2024-09-18T00:18:44.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033906819999999999, 2024-09-18T00:18:44.611 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.611 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.612 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.612 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593553+0000", 2024-09-18T00:18:44.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593553+0000", 2024-09-18T00:18:44.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593555+0000", 2024-09-18T00:18:44.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.643e-06 2024-09-18T00:18:44.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593567+0000", 2024-09-18T00:18:44.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2289999999999999e-05 2024-09-18T00:18:44.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593616+0000", 2024-09-18T00:18:44.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8406999999999999e-05 2024-09-18T00:18:44.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593629+0000", 2024-09-18T00:18:44.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2774e-05 2024-09-18T00:18:44.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593696+0000", 2024-09-18T00:18:44.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7587000000000002e-05 2024-09-18T00:18:44.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.593720+0000", 2024-09-18T00:18:44.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3638e-05 2024-09-18T00:18:44.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.596897+0000", 2024-09-18T00:18:44.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003176772 2024-09-18T00:18:44.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.596944+0000", 2024-09-18T00:18:44.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7571000000000002e-05 2024-09-18T00:18:44.615 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.615 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.615 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.615 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2381 3.0 e37/23 3:01782307:::rbd_data.112aa7ba4578.000000000000004f:head v 37'16, mlcod=35'14)", 2024-09-18T00:18:44.615 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.662312+0000", 2024-09-18T00:18:44.616 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.925230729999999, 2024-09-18T00:18:44.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0033464179999999999, 2024-09-18T00:18:44.616 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.616 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.616 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.616 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662312+0000", 2024-09-18T00:18:44.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662312+0000", 2024-09-18T00:18:44.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662313+0000", 2024-09-18T00:18:44.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1900000000000001e-07 2024-09-18T00:18:44.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662319+0000", 2024-09-18T00:18:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4670000000000001e-06 2024-09-18T00:18:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662330+0000", 2024-09-18T00:18:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.023e-05 2024-09-18T00:18:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662336+0000", 2024-09-18T00:18:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.691e-06 2024-09-18T00:18:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662427+0000", 2024-09-18T00:18:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0216000000000005e-05 2024-09-18T00:18:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.662448+0000", 2024-09-18T00:18:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1795000000000001e-05 2024-09-18T00:18:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.665606+0000", 2024-09-18T00:18:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031574799999999998 2024-09-18T00:18:44.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.665659+0000", 2024-09-18T00:18:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2719999999999997e-05 2024-09-18T00:18:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2460 3.0 e37/23 3:016548b0:::rbd_data.112aa7ba4578.0000000000000018:head v 37'18, mlcod=37'16)", 2024-09-18T00:18:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.985197+0000", 2024-09-18T00:18:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.602345841000002, 2024-09-18T00:18:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004264894, 2024-09-18T00:18:44.620 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.621 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.621 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.621 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985197+0000", 2024-09-18T00:18:44.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985197+0000", 2024-09-18T00:18:44.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985198+0000", 2024-09-18T00:18:44.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.342e-06 2024-09-18T00:18:44.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985210+0000", 2024-09-18T00:18:44.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1783999999999999e-05 2024-09-18T00:18:44.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985231+0000", 2024-09-18T00:18:44.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0769000000000001e-05 2024-09-18T00:18:44.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985241+0000", 2024-09-18T00:18:44.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0266e-05 2024-09-18T00:18:44.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985398+0000", 2024-09-18T00:18:44.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156701 2024-09-18T00:18:44.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.985447+0000", 2024-09-18T00:18:44.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9391000000000002e-05 2024-09-18T00:18:44.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989422+0000", 2024-09-18T00:18:44.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0039745470000000001 2024-09-18T00:18:44.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989462+0000", 2024-09-18T00:18:44.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0093999999999999e-05 2024-09-18T00:18:44.624 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.624 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.624 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.625 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2469 3.0 e37/23 3:02fdd8b9:::rbd_data.112aa7ba4578.0000000000000044:head v 37'19, mlcod=37'16)", 2024-09-18T00:18:44.625 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.989106+0000", 2024-09-18T00:18:44.625 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.598436623000001, 2024-09-18T00:18:44.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.004408843, 2024-09-18T00:18:44.625 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.625 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.625 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.625 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989106+0000", 2024-09-18T00:18:44.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989106+0000", 2024-09-18T00:18:44.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989107+0000", 2024-09-18T00:18:44.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.088e-06 2024-09-18T00:18:44.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989115+0000", 2024-09-18T00:18:44.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8590000000000005e-06 2024-09-18T00:18:44.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989126+0000", 2024-09-18T00:18:44.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1144e-05 2024-09-18T00:18:44.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989134+0000", 2024-09-18T00:18:44.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4229999999999999e-06 2024-09-18T00:18:44.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989215+0000", 2024-09-18T00:18:44.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1551000000000006e-05 2024-09-18T00:18:44.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989237+0000", 2024-09-18T00:18:44.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1832999999999999e-05 2024-09-18T00:18:44.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.993479+0000", 2024-09-18T00:18:44.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042420119999999999 2024-09-18T00:18:44.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.993515+0000", 2024-09-18T00:18:44.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5933000000000003e-05 2024-09-18T00:18:44.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.629 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.629 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2471 3.4 e37/23 3:26bbbe4b:::rbd_data.112aa7ba4578.000000000000004e:head v 37'98, mlcod=37'95)", 2024-09-18T00:18:44.629 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.989721+0000", 2024-09-18T00:18:44.629 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.597821723999999, 2024-09-18T00:18:44.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043834369999999996, 2024-09-18T00:18:44.629 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.629 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.629 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.629 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989721+0000", 2024-09-18T00:18:44.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989721+0000", 2024-09-18T00:18:44.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989722+0000", 2024-09-18T00:18:44.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3300000000000001e-07 2024-09-18T00:18:44.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989728+0000", 2024-09-18T00:18:44.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:18:44.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989741+0000", 2024-09-18T00:18:44.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3444999999999999e-05 2024-09-18T00:18:44.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989750+0000", 2024-09-18T00:18:44.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7369999999999995e-06 2024-09-18T00:18:44.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989807+0000", 2024-09-18T00:18:44.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6971000000000001e-05 2024-09-18T00:18:44.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.989826+0000", 2024-09-18T00:18:44.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8918000000000001e-05 2024-09-18T00:18:44.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994077+0000", 2024-09-18T00:18:44.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042507719999999999 2024-09-18T00:18:44.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994105+0000", 2024-09-18T00:18:44.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7885000000000002e-05 2024-09-18T00:18:44.633 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.633 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.633 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.633 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2472 3.0 e37/23 3:01782307:::rbd_data.112aa7ba4578.000000000000004f:head v 37'20, mlcod=37'17)", 2024-09-18T00:18:44.633 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.990421+0000", 2024-09-18T00:18:44.634 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.59712163, 2024-09-18T00:18:44.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00366118, 2024-09-18T00:18:44.634 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.634 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.634 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.634 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990421+0000", 2024-09-18T00:18:44.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990421+0000", 2024-09-18T00:18:44.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990422+0000", 2024-09-18T00:18:44.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8599999999999998e-07 2024-09-18T00:18:44.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990428+0000", 2024-09-18T00:18:44.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.908e-06 2024-09-18T00:18:44.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990438+0000", 2024-09-18T00:18:44.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0244000000000001e-05 2024-09-18T00:18:44.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990445+0000", 2024-09-18T00:18:44.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7170000000000002e-06 2024-09-18T00:18:44.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990492+0000", 2024-09-18T00:18:44.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7644e-05 2024-09-18T00:18:44.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.990513+0000", 2024-09-18T00:18:44.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.02e-05 2024-09-18T00:18:44.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994045+0000", 2024-09-18T00:18:44.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0035321699999999998 2024-09-18T00:18:44.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.994082+0000", 2024-09-18T00:18:44.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7611000000000003e-05 2024-09-18T00:18:44.638 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.638 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.638 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.638 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2898 3.9 e38/23 3:956131f6:::rbd_data.112aa7ba4578.000000000000019f:head v 38'23, mlcod=38'21)", 2024-09-18T00:18:44.638 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.040063+0000", 2024-09-18T00:18:44.638 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.547479561999999, 2024-09-18T00:18:44.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031556990000000001, 2024-09-18T00:18:44.638 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.638 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.638 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.638 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040063+0000", 2024-09-18T00:18:44.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040063+0000", 2024-09-18T00:18:44.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040065+0000", 2024-09-18T00:18:44.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.84e-06 2024-09-18T00:18:44.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040076+0000", 2024-09-18T00:18:44.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1267000000000001e-05 2024-09-18T00:18:44.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040104+0000", 2024-09-18T00:18:44.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7406000000000002e-05 2024-09-18T00:18:44.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040116+0000", 2024-09-18T00:18:44.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.241e-05 2024-09-18T00:18:44.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040244+0000", 2024-09-18T00:18:44.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012744199999999999 2024-09-18T00:18:44.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.040280+0000", 2024-09-18T00:18:44.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6034000000000001e-05 2024-09-18T00:18:44.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.043136+0000", 2024-09-18T00:18:44.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028566170000000001 2024-09-18T00:18:44.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.043219+0000", 2024-09-18T00:18:44.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2682999999999994e-05 2024-09-18T00:18:44.642 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.642 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.642 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.642 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2973 3.4 e38/23 3:23ae92e0:::rbd_data.112aa7ba4578.00000000000002c2:head v 38'99, mlcod=37'97)", 2024-09-18T00:18:44.642 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.229597+0000", 2024-09-18T00:18:44.642 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.357946294000001, 2024-09-18T00:18:44.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0048943249999999997, 2024-09-18T00:18:44.643 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.643 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.643 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.643 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229597+0000", 2024-09-18T00:18:44.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229597+0000", 2024-09-18T00:18:44.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229599+0000", 2024-09-18T00:18:44.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.396e-06 2024-09-18T00:18:44.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229634+0000", 2024-09-18T00:18:44.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5237000000000002e-05 2024-09-18T00:18:44.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229666+0000", 2024-09-18T00:18:44.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1888999999999999e-05 2024-09-18T00:18:44.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229681+0000", 2024-09-18T00:18:44.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4783000000000001e-05 2024-09-18T00:18:44.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229837+0000", 2024-09-18T00:18:44.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156193 2024-09-18T00:18:44.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.229899+0000", 2024-09-18T00:18:44.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1458999999999996e-05 2024-09-18T00:18:44.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.234405+0000", 2024-09-18T00:18:44.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045061220000000004 2024-09-18T00:18:44.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.234491+0000", 2024-09-18T00:18:44.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6246000000000001e-05 2024-09-18T00:18:44.646 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.646 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.647 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.647 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:2983 3.9 e38/23 3:93d2c681:::rbd_data.112aa7ba4578.000000000000003d:head v 38'24, mlcod=38'22)", 2024-09-18T00:18:44.647 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.258087+0000", 2024-09-18T00:18:44.647 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.329456048000001, 2024-09-18T00:18:44.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0034586859999999999, 2024-09-18T00:18:44.647 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.647 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.647 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.647 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258087+0000", 2024-09-18T00:18:44.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258087+0000", 2024-09-18T00:18:44.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258089+0000", 2024-09-18T00:18:44.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5639999999999999e-06 2024-09-18T00:18:44.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258196+0000", 2024-09-18T00:18:44.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106185 2024-09-18T00:18:44.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258235+0000", 2024-09-18T00:18:44.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9867000000000003e-05 2024-09-18T00:18:44.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258257+0000", 2024-09-18T00:18:44.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1166e-05 2024-09-18T00:18:44.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258360+0000", 2024-09-18T00:18:44.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103396 2024-09-18T00:18:44.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.258388+0000", 2024-09-18T00:18:44.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7773e-05 2024-09-18T00:18:44.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.261500+0000", 2024-09-18T00:18:44.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00311233 2024-09-18T00:18:44.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.261545+0000", 2024-09-18T00:18:44.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5405000000000003e-05 2024-09-18T00:18:44.651 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.651 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.651 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.651 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:3013 3.4 e38/23 3:23ae92e0:::rbd_data.112aa7ba4578.00000000000002c2:head v 38'101, mlcod=38'99)", 2024-09-18T00:18:44.651 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:22.411123+0000", 2024-09-18T00:18:44.651 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.176419460999998, 2024-09-18T00:18:44.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003259236, 2024-09-18T00:18:44.651 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.651 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.652 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411123+0000", 2024-09-18T00:18:44.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411123+0000", 2024-09-18T00:18:44.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411124+0000", 2024-09-18T00:18:44.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:18:44.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411129+0000", 2024-09-18T00:18:44.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0529999999999997e-06 2024-09-18T00:18:44.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411138+0000", 2024-09-18T00:18:44.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8370000000000003e-06 2024-09-18T00:18:44.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411144+0000", 2024-09-18T00:18:44.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.852e-06 2024-09-18T00:18:44.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411201+0000", 2024-09-18T00:18:44.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6963999999999999e-05 2024-09-18T00:18:44.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.411219+0000", 2024-09-18T00:18:44.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8780999999999999e-05 2024-09-18T00:18:44.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.414342+0000", 2024-09-18T00:18:44.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031231929999999998 2024-09-18T00:18:44.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:22.414383+0000", 2024-09-18T00:18:44.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0139000000000003e-05 2024-09-18T00:18:44.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.655 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.655 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:5722 3.4 e41/23 3:20381d86:::rbd_data.112a3a39fc45.000000000000001d:head v 41'107, mlcod=40'105)", 2024-09-18T00:18:44.655 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:18:44.656 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.252047212000001, 2024-09-18T00:18:44.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01185172, 2024-09-18T00:18:44.656 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.656 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.656 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.656 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:18:44.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:18:44.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335497+0000", 2024-09-18T00:18:44.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.505e-06 2024-09-18T00:18:44.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335945+0000", 2024-09-18T00:18:44.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000447687 2024-09-18T00:18:44.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335961+0000", 2024-09-18T00:18:44.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6430999999999999e-05 2024-09-18T00:18:44.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335969+0000", 2024-09-18T00:18:44.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:18:44.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336030+0000", 2024-09-18T00:18:44.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1255999999999998e-05 2024-09-18T00:18:44.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336061+0000", 2024-09-18T00:18:44.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0579e-05 2024-09-18T00:18:44.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347296+0000", 2024-09-18T00:18:44.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011235663 2024-09-18T00:18:44.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347347+0000", 2024-09-18T00:18:44.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0914000000000001e-05 2024-09-18T00:18:44.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.660 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.660 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.660 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11443 3.4 3:2361bafb:::rbd_data.112ab1fa102a.00000000000001a9:head [set-alloc-hint object_size 524288 write_size 524288,write 90112~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:44.660 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.239414+0000", 2024-09-18T00:18:44.660 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.3481293040000004, 2024-09-18T00:18:44.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003214681, 2024-09-18T00:18:44.660 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.660 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.660 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:44.660 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:44.661 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:44.661 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11443 2024-09-18T00:18:44.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.661 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.239414+0000", 2024-09-18T00:18:44.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.239414+0000", 2024-09-18T00:18:44.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.239417+0000", 2024-09-18T00:18:44.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7929999999999999e-06 2024-09-18T00:18:44.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.239447+0000", 2024-09-18T00:18:44.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9762000000000001e-05 2024-09-18T00:18:44.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.239490+0000", 2024-09-18T00:18:44.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2959e-05 2024-09-18T00:18:44.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.239515+0000", 2024-09-18T00:18:44.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5038999999999999e-05 2024-09-18T00:18:44.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.239649+0000", 2024-09-18T00:18:44.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000133559 2024-09-18T00:18:44.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.239796+0000", 2024-09-18T00:18:44.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147675 2024-09-18T00:18:44.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:44.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.242395+0000", 2024-09-18T00:18:44.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002598712 2024-09-18T00:18:44.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.242492+0000", 2024-09-18T00:18:44.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6986999999999998e-05 2024-09-18T00:18:44.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.242628+0000", 2024-09-18T00:18:44.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013619500000000001 2024-09-18T00:18:44.665 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.665 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.665 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.665 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11457 3.0 3:0214e7b3:::rbd_data.112ab1fa102a.0000000000000150:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:44.666 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:18:44.666 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.3276812739999997, 2024-09-18T00:18:44.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028499165, 2024-09-18T00:18:44.666 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.666 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.666 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.666 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:44.666 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:44.666 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:44.666 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11457 2024-09-18T00:18:44.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.666 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:18:44.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:18:44.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259865+0000", 2024-09-18T00:18:44.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7230000000000002e-06 2024-09-18T00:18:44.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259883+0000", 2024-09-18T00:18:44.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8067000000000002e-05 2024-09-18T00:18:44.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259924+0000", 2024-09-18T00:18:44.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0550999999999999e-05 2024-09-18T00:18:44.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259943+0000", 2024-09-18T00:18:44.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:18:44.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260054+0000", 2024-09-18T00:18:44.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011075 2024-09-18T00:18:44.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260138+0000", 2024-09-18T00:18:44.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4549000000000007e-05 2024-09-18T00:18:44.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:44.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.287926+0000", 2024-09-18T00:18:44.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027787642000000001 2024-09-18T00:18:44.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288090+0000", 2024-09-18T00:18:44.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016402499999999999 2024-09-18T00:18:44.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288361+0000", 2024-09-18T00:18:44.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027097800000000001 2024-09-18T00:18:44.670 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.670 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.670 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.671 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:12086 3.0 3:02534001:::rbd_data.112a7b1589a.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1490944~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:44.671 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:36.337486+0000", 2024-09-18T00:18:44.671 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.250056915, 2024-09-18T00:18:44.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038926289999999999, 2024-09-18T00:18:44.671 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.671 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.671 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.671 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:44.671 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:44.671 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:44.671 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 12086 2024-09-18T00:18:44.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.671 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.337486+0000", 2024-09-18T00:18:44.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.337486+0000", 2024-09-18T00:18:44.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.337489+0000", 2024-09-18T00:18:44.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4230000000000001e-06 2024-09-18T00:18:44.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.337521+0000", 2024-09-18T00:18:44.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1553000000000001e-05 2024-09-18T00:18:44.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.337562+0000", 2024-09-18T00:18:44.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1393000000000002e-05 2024-09-18T00:18:44.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.337601+0000", 2024-09-18T00:18:44.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9042000000000003e-05 2024-09-18T00:18:44.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.337741+0000", 2024-09-18T00:18:44.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000140118 2024-09-18T00:18:44.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.338143+0000", 2024-09-18T00:18:44.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00040189499999999999 2024-09-18T00:18:44.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:44.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.341222+0000", 2024-09-18T00:18:44.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0030784240000000002 2024-09-18T00:18:44.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.341291+0000", 2024-09-18T00:18:44.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9116999999999996e-05 2024-09-18T00:18:44.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:36.341379+0000", 2024-09-18T00:18:44.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7664000000000006e-05 2024-09-18T00:18:44.676 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.676 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.676 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.676 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:14706 3.0 3:03995c04:::rbd_id.image_client.0-clone9:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-18T00:18:44.676 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:40.679335+0000", 2024-09-18T00:18:44.676 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.9082078980000001, 2024-09-18T00:18:44.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0052651340000000003, 2024-09-18T00:18:44.676 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:44.676 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:44.676 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:44.676 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:44.677 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:44.677 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:44.677 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 14706 2024-09-18T00:18:44.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.677 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:44.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:44.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.679335+0000", 2024-09-18T00:18:44.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:44.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.679335+0000", 2024-09-18T00:18:44.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:44.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:44.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.679337+0000", 2024-09-18T00:18:44.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3599999999999999e-06 2024-09-18T00:18:44.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:44.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.679350+0000", 2024-09-18T00:18:44.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2278e-05 2024-09-18T00:18:44.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:44.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.679373+0000", 2024-09-18T00:18:44.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3799e-05 2024-09-18T00:18:44.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:44.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.679385+0000", 2024-09-18T00:18:44.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1277000000000001e-05 2024-09-18T00:18:44.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.679534+0000", 2024-09-18T00:18:44.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014949700000000001 2024-09-18T00:18:44.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:18:44.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.679556+0000", 2024-09-18T00:18:44.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1902000000000001e-05 2024-09-18T00:18:44.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:44.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.682283+0000", 2024-09-18T00:18:44.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027267559999999999 2024-09-18T00:18:44.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:44.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.682638+0000", 2024-09-18T00:18:44.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035490700000000001 2024-09-18T00:18:44.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:44.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.684461+0000", 2024-09-18T00:18:44.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0018228109999999999 2024-09-18T00:18:44.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:44.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.684520+0000", 2024-09-18T00:18:44.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9295000000000001e-05 2024-09-18T00:18:44.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:44.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:44.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:44.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:40.684600+0000", 2024-09-18T00:18:44.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0252000000000003e-05 2024-09-18T00:18:44.681 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.681 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.681 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.681 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:44.682 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:44.682 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:44.683 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:18:44.683 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:44.678+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:44.784 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:44.782+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:44.851 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:44.851 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:44.851 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:44.851 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:44.865 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:18:44.884 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:44.882+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:44.985 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:44.978+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:44.995 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:44.995 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:44.995 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:44.995 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:44.995 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:45.009 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:18:45.086 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:45.082+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:45.128 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:599 3.15 3:ac22c048:::rbd_data.112ae20746f5.0000000000000110:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.980706+0000", 2024-09-18T00:18:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 33.145367383999996, 2024-09-18T00:18:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017399257000000001, 2024-09-18T00:18:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:45.129 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 599 2024-09-18T00:18:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980706+0000", 2024-09-18T00:18:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980706+0000", 2024-09-18T00:18:45.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980708+0000", 2024-09-18T00:18:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5919999999999999e-06 2024-09-18T00:18:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980722+0000", 2024-09-18T00:18:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.364e-05 2024-09-18T00:18:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980752+0000", 2024-09-18T00:18:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9794999999999998e-05 2024-09-18T00:18:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980778+0000", 2024-09-18T00:18:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6494000000000001e-05 2024-09-18T00:18:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980885+0000", 2024-09-18T00:18:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107104 2024-09-18T00:18:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.981079+0000", 2024-09-18T00:18:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019375699999999999 2024-09-18T00:18:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.981866+0000", 2024-09-18T00:18:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00078716099999999998 2024-09-18T00:18:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.997852+0000", 2024-09-18T00:18:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015986305999999999 2024-09-18T00:18:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.997957+0000", 2024-09-18T00:18:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104085 2024-09-18T00:18:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.998037+0000", 2024-09-18T00:18:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0098000000000005e-05 2024-09-18T00:18:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.998105+0000", 2024-09-18T00:18:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8225000000000005e-05 2024-09-18T00:18:45.134 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:640 3.7 3:e0810bc8:::rbd_data.112ae20746f5.0000000000000019:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.104726+0000", 2024-09-18T00:18:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 33.021346555999997, 2024-09-18T00:18:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017067268, 2024-09-18T00:18:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.135 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 640 2024-09-18T00:18:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104726+0000", 2024-09-18T00:18:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104726+0000", 2024-09-18T00:18:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104728+0000", 2024-09-18T00:18:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.336e-06 2024-09-18T00:18:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104735+0000", 2024-09-18T00:18:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6149999999999999e-06 2024-09-18T00:18:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104753+0000", 2024-09-18T00:18:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7362000000000001e-05 2024-09-18T00:18:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104761+0000", 2024-09-18T00:18:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5609999999999999e-06 2024-09-18T00:18:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104842+0000", 2024-09-18T00:18:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0526999999999997e-05 2024-09-18T00:18:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104916+0000", 2024-09-18T00:18:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3724000000000001e-05 2024-09-18T00:18:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.106201+0000", 2024-09-18T00:18:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001285046 2024-09-18T00:18:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121677+0000", 2024-09-18T00:18:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01547665 2024-09-18T00:18:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121696+0000", 2024-09-18T00:18:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8304999999999998e-05 2024-09-18T00:18:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121744+0000", 2024-09-18T00:18:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8415000000000003e-05 2024-09-18T00:18:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121794+0000", 2024-09-18T00:18:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9727e-05 2024-09-18T00:18:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.141 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.325906314000001, 2024-09-18T00:18:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:18:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.142 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:18:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.143 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:18:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:18:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:18:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:18:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:18:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:18:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:18:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:18:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:18:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:18:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:18:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:18:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:18:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:18:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:18:45.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:18:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:18:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:18:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:18:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:18:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.147 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:18:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:18:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:18:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.309698441000002, 2024-09-18T00:18:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:18:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.148 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:18:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:18:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:18:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:18:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:18:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:18:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:18:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:18:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:18:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:18:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:18:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:18:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:18:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:18:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:5724 3.1e 3:78324145:::rbd_data.112a3a39fc45.000000000000001e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:45.152 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.341891+0000", 2024-09-18T00:18:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.784181977999999, 2024-09-18T00:18:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014843063, 2024-09-18T00:18:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 5724 2024-09-18T00:18:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.341891+0000", 2024-09-18T00:18:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.341891+0000", 2024-09-18T00:18:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.341893+0000", 2024-09-18T00:18:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1129999999999999e-06 2024-09-18T00:18:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343233+0000", 2024-09-18T00:18:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001339614 2024-09-18T00:18:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343251+0000", 2024-09-18T00:18:45.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8479e-05 2024-09-18T00:18:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343260+0000", 2024-09-18T00:18:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1470000000000007e-06 2024-09-18T00:18:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343333+0000", 2024-09-18T00:18:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2302e-05 2024-09-18T00:18:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343494+0000", 2024-09-18T00:18:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000161593 2024-09-18T00:18:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 6", 2024-09-18T00:18:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343778+0000", 2024-09-18T00:18:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028366099999999999 2024-09-18T00:18:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.351688+0000", 2024-09-18T00:18:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0079100739999999996 2024-09-18T00:18:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.356673+0000", 2024-09-18T00:18:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049848210000000004 2024-09-18T00:18:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.356706+0000", 2024-09-18T00:18:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3531000000000001e-05 2024-09-18T00:18:45.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.356734+0000", 2024-09-18T00:18:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7727999999999999e-05 2024-09-18T00:18:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6175 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 8585216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.473970+0000", 2024-09-18T00:18:45.159 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.652102605, 2024-09-18T00:18:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013962384, 2024-09-18T00:18:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6175 2024-09-18T00:18:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473970+0000", 2024-09-18T00:18:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473970+0000", 2024-09-18T00:18:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473972+0000", 2024-09-18T00:18:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1310000000000001e-06 2024-09-18T00:18:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473977+0000", 2024-09-18T00:18:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1769999999999997e-06 2024-09-18T00:18:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473993+0000", 2024-09-18T00:18:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5988999999999999e-05 2024-09-18T00:18:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474005+0000", 2024-09-18T00:18:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2622999999999999e-05 2024-09-18T00:18:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.485712+0000", 2024-09-18T00:18:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011706701 2024-09-18T00:18:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487289+0000", 2024-09-18T00:18:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0015770280000000001 2024-09-18T00:18:45.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487933+0000", 2024-09-18T00:18:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00064373499999999999 2024-09-18T00:18:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6176 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 9371648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474022+0000", 2024-09-18T00:18:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.65205134, 2024-09-18T00:18:45.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014183915, 2024-09-18T00:18:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6176 2024-09-18T00:18:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474022+0000", 2024-09-18T00:18:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474022+0000", 2024-09-18T00:18:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474023+0000", 2024-09-18T00:18:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.02e-06 2024-09-18T00:18:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474028+0000", 2024-09-18T00:18:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9599999999999999e-06 2024-09-18T00:18:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474044+0000", 2024-09-18T00:18:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6113999999999999e-05 2024-09-18T00:18:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474056+0000", 2024-09-18T00:18:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2349e-05 2024-09-18T00:18:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487994+0000", 2024-09-18T00:18:45.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013937748 2024-09-18T00:18:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488088+0000", 2024-09-18T00:18:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3888999999999995e-05 2024-09-18T00:18:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488206+0000", 2024-09-18T00:18:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117835 2024-09-18T00:18:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6177 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474074+0000", 2024-09-18T00:18:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.651998672000001, 2024-09-18T00:18:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01434039, 2024-09-18T00:18:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6177 2024-09-18T00:18:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.169 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474074+0000", 2024-09-18T00:18:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474074+0000", 2024-09-18T00:18:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474075+0000", 2024-09-18T00:18:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-18T00:18:45.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474080+0000", 2024-09-18T00:18:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9459999999999997e-06 2024-09-18T00:18:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474097+0000", 2024-09-18T00:18:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6705999999999999e-05 2024-09-18T00:18:45.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474110+0000", 2024-09-18T00:18:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2582000000000001e-05 2024-09-18T00:18:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488252+0000", 2024-09-18T00:18:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014142937 2024-09-18T00:18:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488327+0000", 2024-09-18T00:18:45.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4184000000000007e-05 2024-09-18T00:18:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488415+0000", 2024-09-18T00:18:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8075e-05 2024-09-18T00:18:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6178 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474128+0000", 2024-09-18T00:18:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.651944515, 2024-09-18T00:18:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014460381, 2024-09-18T00:18:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6178 2024-09-18T00:18:45.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474128+0000", 2024-09-18T00:18:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474128+0000", 2024-09-18T00:18:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474130+0000", 2024-09-18T00:18:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1489999999999999e-06 2024-09-18T00:18:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474135+0000", 2024-09-18T00:18:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0830000000000003e-06 2024-09-18T00:18:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474151+0000", 2024-09-18T00:18:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6677e-05 2024-09-18T00:18:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474166+0000", 2024-09-18T00:18:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4871e-05 2024-09-18T00:18:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488458+0000", 2024-09-18T00:18:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014292206999999999 2024-09-18T00:18:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488516+0000", 2024-09-18T00:18:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7627999999999998e-05 2024-09-18T00:18:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488589+0000", 2024-09-18T00:18:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2766000000000001e-05 2024-09-18T00:18:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.178 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6179 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 11730944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474183+0000", 2024-09-18T00:18:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.651889761, 2024-09-18T00:18:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014618816, 2024-09-18T00:18:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:45.179 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6179 2024-09-18T00:18:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474183+0000", 2024-09-18T00:18:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474183+0000", 2024-09-18T00:18:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474184+0000", 2024-09-18T00:18:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.083e-06 2024-09-18T00:18:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474190+0000", 2024-09-18T00:18:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6570000000000002e-06 2024-09-18T00:18:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474206+0000", 2024-09-18T00:18:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5914e-05 2024-09-18T00:18:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474225+0000", 2024-09-18T00:18:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9369e-05 2024-09-18T00:18:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488639+0000", 2024-09-18T00:18:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014413769 2024-09-18T00:18:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488709+0000", 2024-09-18T00:18:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9655999999999999e-05 2024-09-18T00:18:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488802+0000", 2024-09-18T00:18:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3368000000000005e-05 2024-09-18T00:18:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.183 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6180 3.8 3:16b24e91:::rbd_data.112aa7ba4578.0000000000000161:5 [read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474244+0000", 2024-09-18T00:18:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.651829463999999, 2024-09-18T00:18:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014495918999999999, 2024-09-18T00:18:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6180 2024-09-18T00:18:45.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474244+0000", 2024-09-18T00:18:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474244+0000", 2024-09-18T00:18:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474245+0000", 2024-09-18T00:18:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.105e-06 2024-09-18T00:18:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474318+0000", 2024-09-18T00:18:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3182999999999994e-05 2024-09-18T00:18:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474345+0000", 2024-09-18T00:18:45.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6730999999999999e-05 2024-09-18T00:18:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474359+0000", 2024-09-18T00:18:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4433999999999999e-05 2024-09-18T00:18:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488626+0000", 2024-09-18T00:18:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014266993 2024-09-18T00:18:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488680+0000", 2024-09-18T00:18:45.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4453999999999997e-05 2024-09-18T00:18:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488739+0000", 2024-09-18T00:18:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9018999999999999e-05 2024-09-18T00:18:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.188 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6181 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 12517376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474384+0000", 2024-09-18T00:18:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.651688528000001, 2024-09-18T00:18:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014686072999999999, 2024-09-18T00:18:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6181 2024-09-18T00:18:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.189 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474384+0000", 2024-09-18T00:18:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474384+0000", 2024-09-18T00:18:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474386+0000", 2024-09-18T00:18:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.688e-06 2024-09-18T00:18:45.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474392+0000", 2024-09-18T00:18:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.575e-06 2024-09-18T00:18:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474409+0000", 2024-09-18T00:18:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7638999999999998e-05 2024-09-18T00:18:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474423+0000", 2024-09-18T00:18:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3237e-05 2024-09-18T00:18:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488844+0000", 2024-09-18T00:18:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014421079 2024-09-18T00:18:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488943+0000", 2024-09-18T00:18:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9764000000000002e-05 2024-09-18T00:18:45.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.489071+0000", 2024-09-18T00:18:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012709099999999999 2024-09-18T00:18:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6280 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 11075584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.478893+0000", 2024-09-18T00:18:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.647180277, 2024-09-18T00:18:45.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012517118000000001, 2024-09-18T00:18:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6280 2024-09-18T00:18:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478893+0000", 2024-09-18T00:18:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478893+0000", 2024-09-18T00:18:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478894+0000", 2024-09-18T00:18:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-18T00:18:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478905+0000", 2024-09-18T00:18:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1045e-05 2024-09-18T00:18:45.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478921+0000", 2024-09-18T00:18:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6469000000000001e-05 2024-09-18T00:18:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478934+0000", 2024-09-18T00:18:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2525e-05 2024-09-18T00:18:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491258+0000", 2024-09-18T00:18:45.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012324422999999999 2024-09-18T00:18:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491327+0000", 2024-09-18T00:18:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8923000000000004e-05 2024-09-18T00:18:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491410+0000", 2024-09-18T00:18:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2730000000000002e-05 2024-09-18T00:18:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6281 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 11862016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.478952+0000", 2024-09-18T00:18:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.647120877999999, 2024-09-18T00:18:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013203609, 2024-09-18T00:18:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6281 2024-09-18T00:18:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478952+0000", 2024-09-18T00:18:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478952+0000", 2024-09-18T00:18:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478953+0000", 2024-09-18T00:18:45.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0720000000000001e-06 2024-09-18T00:18:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478958+0000", 2024-09-18T00:18:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7369999999999997e-06 2024-09-18T00:18:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.482143+0000", 2024-09-18T00:18:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031850870000000001 2024-09-18T00:18:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.482179+0000", 2024-09-18T00:18:45.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6381000000000003e-05 2024-09-18T00:18:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.492006+0000", 2024-09-18T00:18:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0098267319999999995 2024-09-18T00:18:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.492069+0000", 2024-09-18T00:18:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2564000000000006e-05 2024-09-18T00:18:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.492156+0000", 2024-09-18T00:18:45.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7035999999999998e-05 2024-09-18T00:18:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6572 3.7 3:e54d9f3a:::rbd_data.112a3a39fc45.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 458184~568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.725733+0000", 2024-09-18T00:18:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.400339717000001, 2024-09-18T00:18:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016678148, 2024-09-18T00:18:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.202 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6572 2024-09-18T00:18:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.725733+0000", 2024-09-18T00:18:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.725733+0000", 2024-09-18T00:18:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.725736+0000", 2024-09-18T00:18:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2919999999999998e-06 2024-09-18T00:18:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726273+0000", 2024-09-18T00:18:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00053760199999999998 2024-09-18T00:18:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726295+0000", 2024-09-18T00:18:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1855999999999999e-05 2024-09-18T00:18:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726305+0000", 2024-09-18T00:18:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0098000000000001e-05 2024-09-18T00:18:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726384+0000", 2024-09-18T00:18:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8731000000000002e-05 2024-09-18T00:18:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726582+0000", 2024-09-18T00:18:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019781399999999999 2024-09-18T00:18:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726857+0000", 2024-09-18T00:18:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027557299999999999 2024-09-18T00:18:45.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.735232+0000", 2024-09-18T00:18:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083744660000000005 2024-09-18T00:18:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.742317+0000", 2024-09-18T00:18:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0070851789999999996 2024-09-18T00:18:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:45.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.742357+0000", 2024-09-18T00:18:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9975000000000003e-05 2024-09-18T00:18:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.742411+0000", 2024-09-18T00:18:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4561999999999997e-05 2024-09-18T00:18:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:45.208 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.282657121, 2024-09-18T00:18:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:18:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:18:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.209 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:18:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:18:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:18:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:18:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:18:45.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:18:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:18:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:18:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:18:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:18:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:18:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:18:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:45.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:18:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:18:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:18:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:18:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:18:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:18:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:18:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:18:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:18:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:18:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.215 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:18:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.261380569, 2024-09-18T00:18:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:18:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:18:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:18:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:18:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:18:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:18:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:18:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:18:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:18:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:18:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:18:45.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:18:45.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:18:45.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:18:45.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:18:45.219 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.220 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.220 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.220 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11460 3.19 3:9e578b6c:::rbd_data.112ab1fa102a.0000000000000153:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:45.220 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:18:45.220 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.8590091169999994, 2024-09-18T00:18:45.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041989011999999999, 2024-09-18T00:18:45.220 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.220 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.220 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:45.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.221 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11460 2024-09-18T00:18:45.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.221 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:18:45.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:18:45.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267067+0000", 2024-09-18T00:18:45.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:18:45.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267088+0000", 2024-09-18T00:18:45.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0908999999999999e-05 2024-09-18T00:18:45.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267133+0000", 2024-09-18T00:18:45.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5232000000000003e-05 2024-09-18T00:18:45.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267157+0000", 2024-09-18T00:18:45.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3833e-05 2024-09-18T00:18:45.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267300+0000", 2024-09-18T00:18:45.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142282 2024-09-18T00:18:45.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267447+0000", 2024-09-18T00:18:45.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147851 2024-09-18T00:18:45.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:45.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308889+0000", 2024-09-18T00:18:45.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041441779999999998 2024-09-18T00:18:45.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:45.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308956+0000", 2024-09-18T00:18:45.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7136000000000002e-05 2024-09-18T00:18:45.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.309053+0000", 2024-09-18T00:18:45.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6552999999999995e-05 2024-09-18T00:18:45.225 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.225 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.226 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.226 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11464 3.b 3:d331f8ac:::rbd_data.112ab1fa102a.0000000000000157:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:45.226 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:18:45.226 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.8540186310000006, 2024-09-18T00:18:45.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034709218, 2024-09-18T00:18:45.226 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.226 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.226 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:45.226 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.226 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.226 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.227 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11464 2024-09-18T00:18:45.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.227 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:18:45.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:18:45.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272058+0000", 2024-09-18T00:18:45.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4070000000000002e-06 2024-09-18T00:18:45.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272074+0000", 2024-09-18T00:18:45.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5872999999999999e-05 2024-09-18T00:18:45.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272113+0000", 2024-09-18T00:18:45.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9224999999999998e-05 2024-09-18T00:18:45.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272135+0000", 2024-09-18T00:18:45.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2093e-05 2024-09-18T00:18:45.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272238+0000", 2024-09-18T00:18:45.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102939 2024-09-18T00:18:45.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272327+0000", 2024-09-18T00:18:45.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8746999999999996e-05 2024-09-18T00:18:45.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:45.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.273486+0000", 2024-09-18T00:18:45.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011588779999999999 2024-09-18T00:18:45.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:45.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306439+0000", 2024-09-18T00:18:45.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.032953142999999997 2024-09-18T00:18:45.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:45.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306491+0000", 2024-09-18T00:18:45.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2475000000000002e-05 2024-09-18T00:18:45.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:45.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306648+0000", 2024-09-18T00:18:45.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015725099999999999 2024-09-18T00:18:45.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306764+0000", 2024-09-18T00:18:45.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011518699999999999 2024-09-18T00:18:45.231 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.231 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.231 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.232 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:14346 3.2 3:44bbdafa:::rbd_data.112a13ec2d6c.0000000000000033:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-18T00:18:45.232 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:39.282847+0000", 2024-09-18T00:18:45.232 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.8432257879999998, 2024-09-18T00:18:45.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016717600999999999, 2024-09-18T00:18:45.232 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:45.232 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:45.232 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:45.232 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:45.232 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:45.232 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.232 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 14346 2024-09-18T00:18:45.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.233 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:45.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:45.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282847+0000", 2024-09-18T00:18:45.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:45.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282847+0000", 2024-09-18T00:18:45.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:45.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:45.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282850+0000", 2024-09-18T00:18:45.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1159999999999999e-06 2024-09-18T00:18:45.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:45.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282867+0000", 2024-09-18T00:18:45.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6310000000000001e-05 2024-09-18T00:18:45.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:45.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282904+0000", 2024-09-18T00:18:45.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6952000000000001e-05 2024-09-18T00:18:45.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282921+0000", 2024-09-18T00:18:45.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7765999999999999e-05 2024-09-18T00:18:45.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:45.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.283061+0000", 2024-09-18T00:18:45.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013918599999999999 2024-09-18T00:18:45.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:45.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.283385+0000", 2024-09-18T00:18:45.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032432700000000001 2024-09-18T00:18:45.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:45.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.284421+0000", 2024-09-18T00:18:45.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0010358889999999999 2024-09-18T00:18:45.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:45.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.299281+0000", 2024-09-18T00:18:45.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014860329 2024-09-18T00:18:45.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:45.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.299336+0000", 2024-09-18T00:18:45.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4926000000000002e-05 2024-09-18T00:18:45.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:45.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.299458+0000", 2024-09-18T00:18:45.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000122111 2024-09-18T00:18:45.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:45.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:45.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:45.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.299565+0000", 2024-09-18T00:18:45.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010668899999999999 2024-09-18T00:18:45.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.237 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:45.237 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:45.237 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:45.239 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:18:45.239 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:45.238+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:45.340 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:45.334+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:45.369 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:45.369 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:45.369 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:18:45.369 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:45.383 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:18:45.441 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:45.438+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:45.542 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:45.534+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:45.550 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:45.550 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:45.551 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:45.551 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:45.551 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:45.566 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:18:45.654 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:45.650+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:45.668 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:18:45.674 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-18T00:18:45.687 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:18:45.693 INFO:teuthology.orchestra.run.smithi097.stdout:2617 read 0x1ebc3ed thru 0x1ecb9be (0xf5d2 bytes) 2024-09-18T00:18:45.713 INFO:teuthology.orchestra.run.smithi097.stdout:2618 trunc from 0xea300e2 to 0x3b3e581 2024-09-18T00:18:45.734 INFO:teuthology.orchestra.run.smithi097.stdout:2619 read 0x29f1fd6 thru 0x29ffed6 (0xdf01 bytes) 2024-09-18T00:18:45.735 INFO:teuthology.orchestra.run.smithi097.stdout:2621 read 0x166622a thru 0x1670fe7 (0xadbe bytes) 2024-09-18T00:18:45.735 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:45.736 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:45.736 INFO:teuthology.orchestra.run.smithi097.stdout:2624 read 0x7b54c4 thru 0x7c3a7b (0xe5b8 bytes) 2024-09-18T00:18:45.743 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2418 3.1a 3:59a8b40d:::rbd_data.112aa7ba4578.00000000000001b0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:18:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 25.003583904999999, 2024-09-18T00:18:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10488562999999999, 2024-09-18T00:18:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2418 2024-09-18T00:18:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:18:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:18:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735592+0000", 2024-09-18T00:18:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5229999999999998e-06 2024-09-18T00:18:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735604+0000", 2024-09-18T00:18:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2143e-05 2024-09-18T00:18:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735629+0000", 2024-09-18T00:18:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.497e-05 2024-09-18T00:18:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735648+0000", 2024-09-18T00:18:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8896999999999999e-05 2024-09-18T00:18:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735775+0000", 2024-09-18T00:18:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126935 2024-09-18T00:18:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.736118+0000", 2024-09-18T00:18:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034330800000000001 2024-09-18T00:18:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-18T00:18:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.741633+0000", 2024-09-18T00:18:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055144139999999996 2024-09-18T00:18:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840134+0000", 2024-09-18T00:18:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.098501618999999999 2024-09-18T00:18:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840184+0000", 2024-09-18T00:18:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9953000000000001e-05 2024-09-18T00:18:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840336+0000", 2024-09-18T00:18:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151476 2024-09-18T00:18:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840475+0000", 2024-09-18T00:18:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013939200000000001 2024-09-18T00:18:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2418 3.1a e37/23 ondisk, result = 0)", 2024-09-18T00:18:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:18:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.984994061999998, 2024-09-18T00:18:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.085979978999999998, 2024-09-18T00:18:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:18:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:18:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754181+0000", 2024-09-18T00:18:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8530000000000001e-06 2024-09-18T00:18:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754194+0000", 2024-09-18T00:18:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.292e-05 2024-09-18T00:18:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754230+0000", 2024-09-18T00:18:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5602999999999998e-05 2024-09-18T00:18:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754249+0000", 2024-09-18T00:18:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8850000000000001e-05 2024-09-18T00:18:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840070+0000", 2024-09-18T00:18:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.085821123999999999 2024-09-18T00:18:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840098+0000", 2024-09-18T00:18:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8546000000000001e-05 2024-09-18T00:18:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840159+0000", 2024-09-18T00:18:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1082999999999998e-05 2024-09-18T00:18:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12538 3.3 3:c6f35e4c:::rbd_data.112ab1fa102a.0000000000000000:9 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.394809+0000", 2024-09-18T00:18:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3443640699999992, 2024-09-18T00:18:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043048292000000002, 2024-09-18T00:18:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12538 2024-09-18T00:18:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394809+0000", 2024-09-18T00:18:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394809+0000", 2024-09-18T00:18:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394811+0000", 2024-09-18T00:18:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4500000000000001e-06 2024-09-18T00:18:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394817+0000", 2024-09-18T00:18:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0530000000000003e-06 2024-09-18T00:18:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394838+0000", 2024-09-18T00:18:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0823e-05 2024-09-18T00:18:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394852+0000", 2024-09-18T00:18:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4129e-05 2024-09-18T00:18:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.437681+0000", 2024-09-18T00:18:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042829205000000002 2024-09-18T00:18:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.437776+0000", 2024-09-18T00:18:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5315000000000005e-05 2024-09-18T00:18:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.437858+0000", 2024-09-18T00:18:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1316999999999994e-05 2024-09-18T00:18:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12539 3.3 3:c6f35e4c:::rbd_data.112ab1fa102a.0000000000000000:9 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.394870+0000", 2024-09-18T00:18:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3443037849999993, 2024-09-18T00:18:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043365935000000001, 2024-09-18T00:18:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12539 2024-09-18T00:18:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394870+0000", 2024-09-18T00:18:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394870+0000", 2024-09-18T00:18:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394871+0000", 2024-09-18T00:18:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2729999999999999e-06 2024-09-18T00:18:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394876+0000", 2024-09-18T00:18:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-18T00:18:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394894+0000", 2024-09-18T00:18:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7915999999999999e-05 2024-09-18T00:18:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394924+0000", 2024-09-18T00:18:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0001999999999998e-05 2024-09-18T00:18:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.437932+0000", 2024-09-18T00:18:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043008192000000001 2024-09-18T00:18:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.438071+0000", 2024-09-18T00:18:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013871099999999999 2024-09-18T00:18:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.438236+0000", 2024-09-18T00:18:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000164351 2024-09-18T00:18:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12540 3.3 3:c6f35e4c:::rbd_data.112ab1fa102a.0000000000000000:9 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.394944+0000", 2024-09-18T00:18:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3442298350000002, 2024-09-18T00:18:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043684374999999998, 2024-09-18T00:18:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12540 2024-09-18T00:18:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394944+0000", 2024-09-18T00:18:45.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394944+0000", 2024-09-18T00:18:45.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394945+0000", 2024-09-18T00:18:45.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2640000000000001e-06 2024-09-18T00:18:45.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394951+0000", 2024-09-18T00:18:45.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9830000000000001e-06 2024-09-18T00:18:45.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394969+0000", 2024-09-18T00:18:45.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8266000000000001e-05 2024-09-18T00:18:45.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394984+0000", 2024-09-18T00:18:45.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.466e-05 2024-09-18T00:18:45.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.438321+0000", 2024-09-18T00:18:45.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043337734000000003 2024-09-18T00:18:45.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.438481+0000", 2024-09-18T00:18:45.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015959500000000001 2024-09-18T00:18:45.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.438628+0000", 2024-09-18T00:18:45.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000146873 2024-09-18T00:18:45.770 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.770 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.770 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.771 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12541 3.3 3:c6f35e4c:::rbd_data.112ab1fa102a.0000000000000000:9 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.771 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395001+0000", 2024-09-18T00:18:45.771 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3441726260000006, 2024-09-18T00:18:45.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043946967000000003, 2024-09-18T00:18:45.771 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.771 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.771 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.771 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.771 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.771 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.771 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12541 2024-09-18T00:18:45.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.772 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395001+0000", 2024-09-18T00:18:45.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395001+0000", 2024-09-18T00:18:45.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395002+0000", 2024-09-18T00:18:45.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.096e-06 2024-09-18T00:18:45.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395007+0000", 2024-09-18T00:18:45.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0900000000000004e-06 2024-09-18T00:18:45.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395024+0000", 2024-09-18T00:18:45.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6736999999999999e-05 2024-09-18T00:18:45.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395037+0000", 2024-09-18T00:18:45.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3589e-05 2024-09-18T00:18:45.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.438689+0000", 2024-09-18T00:18:45.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043651769 2024-09-18T00:18:45.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.438776+0000", 2024-09-18T00:18:45.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6680000000000004e-05 2024-09-18T00:18:45.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.438948+0000", 2024-09-18T00:18:45.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017200599999999999 2024-09-18T00:18:45.775 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.775 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.775 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.775 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12542 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.775 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395055+0000", 2024-09-18T00:18:45.775 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3441184209999992, 2024-09-18T00:18:45.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044315872999999999, 2024-09-18T00:18:45.775 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.775 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.776 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.776 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.776 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.776 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.776 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12542 2024-09-18T00:18:45.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.776 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395055+0000", 2024-09-18T00:18:45.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395055+0000", 2024-09-18T00:18:45.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395056+0000", 2024-09-18T00:18:45.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0699999999999999e-06 2024-09-18T00:18:45.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395061+0000", 2024-09-18T00:18:45.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:18:45.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395088+0000", 2024-09-18T00:18:45.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6939000000000001e-05 2024-09-18T00:18:45.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395104+0000", 2024-09-18T00:18:45.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6066000000000002e-05 2024-09-18T00:18:45.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.439018+0000", 2024-09-18T00:18:45.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043913212 2024-09-18T00:18:45.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.439147+0000", 2024-09-18T00:18:45.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012977099999999999 2024-09-18T00:18:45.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.439371+0000", 2024-09-18T00:18:45.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000223525 2024-09-18T00:18:45.779 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.779 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.779 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.779 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12543 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.779 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395123+0000", 2024-09-18T00:18:45.779 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3440500459999996, 2024-09-18T00:18:45.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044696828000000001, 2024-09-18T00:18:45.780 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.780 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.780 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.780 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.780 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.780 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.780 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12543 2024-09-18T00:18:45.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.780 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395123+0000", 2024-09-18T00:18:45.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395123+0000", 2024-09-18T00:18:45.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395125+0000", 2024-09-18T00:18:45.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.203e-06 2024-09-18T00:18:45.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395130+0000", 2024-09-18T00:18:45.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3589999999999999e-06 2024-09-18T00:18:45.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395147+0000", 2024-09-18T00:18:45.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7107000000000001e-05 2024-09-18T00:18:45.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395160+0000", 2024-09-18T00:18:45.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3346000000000001e-05 2024-09-18T00:18:45.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.439454+0000", 2024-09-18T00:18:45.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044293375000000003 2024-09-18T00:18:45.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.439611+0000", 2024-09-18T00:18:45.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000157477 2024-09-18T00:18:45.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.439820+0000", 2024-09-18T00:18:45.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020896099999999999 2024-09-18T00:18:45.783 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.783 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.783 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.784 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12544 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.784 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395178+0000", 2024-09-18T00:18:45.784 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3439949579999997, 2024-09-18T00:18:45.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045040958999999998, 2024-09-18T00:18:45.784 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.784 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.784 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.784 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.784 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.784 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.784 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12544 2024-09-18T00:18:45.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.784 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395178+0000", 2024-09-18T00:18:45.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395178+0000", 2024-09-18T00:18:45.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395180+0000", 2024-09-18T00:18:45.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.28e-06 2024-09-18T00:18:45.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395186+0000", 2024-09-18T00:18:45.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3829999999999997e-06 2024-09-18T00:18:45.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395207+0000", 2024-09-18T00:18:45.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0696e-05 2024-09-18T00:18:45.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395221+0000", 2024-09-18T00:18:45.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3876000000000001e-05 2024-09-18T00:18:45.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.439891+0000", 2024-09-18T00:18:45.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044670407000000002 2024-09-18T00:18:45.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440037+0000", 2024-09-18T00:18:45.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014583700000000001 2024-09-18T00:18:45.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440219+0000", 2024-09-18T00:18:45.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018248 2024-09-18T00:18:45.788 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.788 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.788 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.788 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12545 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.788 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:18:45.788 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3439312999999995, 2024-09-18T00:18:45.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045427642999999997, 2024-09-18T00:18:45.788 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.788 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.788 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.788 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.789 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.789 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.789 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12545 2024-09-18T00:18:45.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.789 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:18:45.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:18:45.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395243+0000", 2024-09-18T00:18:45.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.206e-06 2024-09-18T00:18:45.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395249+0000", 2024-09-18T00:18:45.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:18:45.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395270+0000", 2024-09-18T00:18:45.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1223e-05 2024-09-18T00:18:45.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395285+0000", 2024-09-18T00:18:45.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4725999999999999e-05 2024-09-18T00:18:45.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440296+0000", 2024-09-18T00:18:45.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045011402999999998 2024-09-18T00:18:45.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440453+0000", 2024-09-18T00:18:45.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015742299999999999 2024-09-18T00:18:45.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440670+0000", 2024-09-18T00:18:45.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000216302 2024-09-18T00:18:45.792 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.792 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.792 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.792 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12546 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.792 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:18:45.793 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3438690920000003, 2024-09-18T00:18:45.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045915171999999997, 2024-09-18T00:18:45.793 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.793 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.793 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.793 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.793 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.793 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.793 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12546 2024-09-18T00:18:45.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.793 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:18:45.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:18:45.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395306+0000", 2024-09-18T00:18:45.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.206e-06 2024-09-18T00:18:45.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395330+0000", 2024-09-18T00:18:45.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4686e-05 2024-09-18T00:18:45.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395356+0000", 2024-09-18T00:18:45.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5418999999999998e-05 2024-09-18T00:18:45.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395387+0000", 2024-09-18T00:18:45.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0942000000000003e-05 2024-09-18T00:18:45.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440743+0000", 2024-09-18T00:18:45.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045356592000000001 2024-09-18T00:18:45.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440880+0000", 2024-09-18T00:18:45.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013641700000000001 2024-09-18T00:18:45.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441219+0000", 2024-09-18T00:18:45.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033991000000000002 2024-09-18T00:18:45.796 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.797 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.797 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.797 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12547 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 212992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.797 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:18:45.797 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3437597589999992, 2024-09-18T00:18:45.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046163691999999999, 2024-09-18T00:18:45.797 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.797 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.797 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.798 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12547 2024-09-18T00:18:45.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.798 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:18:45.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:18:45.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395415+0000", 2024-09-18T00:18:45.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.553e-06 2024-09-18T00:18:45.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395426+0000", 2024-09-18T00:18:45.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0682999999999999e-05 2024-09-18T00:18:45.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395449+0000", 2024-09-18T00:18:45.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3082999999999999e-05 2024-09-18T00:18:45.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395469+0000", 2024-09-18T00:18:45.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0173000000000002e-05 2024-09-18T00:18:45.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441289+0000", 2024-09-18T00:18:45.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045820195000000001 2024-09-18T00:18:45.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441426+0000", 2024-09-18T00:18:45.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013712699999999999 2024-09-18T00:18:45.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441577+0000", 2024-09-18T00:18:45.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015087799999999999 2024-09-18T00:18:45.801 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.801 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.801 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.801 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12548 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.801 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:18:45.801 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3436859979999998, 2024-09-18T00:18:45.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046293932000000003, 2024-09-18T00:18:45.801 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.802 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.802 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.802 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.802 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.802 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.802 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12548 2024-09-18T00:18:45.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.802 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:18:45.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:18:45.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395489+0000", 2024-09-18T00:18:45.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.22e-06 2024-09-18T00:18:45.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395512+0000", 2024-09-18T00:18:45.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3312e-05 2024-09-18T00:18:45.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395621+0000", 2024-09-18T00:18:45.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010859 2024-09-18T00:18:45.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395689+0000", 2024-09-18T00:18:45.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8232000000000007e-05 2024-09-18T00:18:45.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441610+0000", 2024-09-18T00:18:45.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045920751000000003 2024-09-18T00:18:45.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441695+0000", 2024-09-18T00:18:45.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5379999999999999e-05 2024-09-18T00:18:45.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441781+0000", 2024-09-18T00:18:45.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6446999999999994e-05 2024-09-18T00:18:45.805 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.805 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.806 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.806 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12549 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 294912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.806 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:18:45.806 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3434253460000001, 2024-09-18T00:18:45.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046410264999999999, 2024-09-18T00:18:45.806 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.806 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.806 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.806 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.806 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.806 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.806 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12549 2024-09-18T00:18:45.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.807 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:18:45.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:18:45.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395750+0000", 2024-09-18T00:18:45.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:18:45.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395758+0000", 2024-09-18T00:18:45.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5259999999999993e-06 2024-09-18T00:18:45.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395793+0000", 2024-09-18T00:18:45.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4459000000000003e-05 2024-09-18T00:18:45.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395809+0000", 2024-09-18T00:18:45.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6433e-05 2024-09-18T00:18:45.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441851+0000", 2024-09-18T00:18:45.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046041493000000003 2024-09-18T00:18:45.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442002+0000", 2024-09-18T00:18:45.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151433 2024-09-18T00:18:45.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442158+0000", 2024-09-18T00:18:45.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000156358 2024-09-18T00:18:45.810 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.810 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.810 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.810 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12550 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.810 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:18:45.810 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3433138570000001, 2024-09-18T00:18:45.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046498398000000003, 2024-09-18T00:18:45.811 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.811 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.811 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.811 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.811 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.811 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.811 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12550 2024-09-18T00:18:45.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.811 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:18:45.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:18:45.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395861+0000", 2024-09-18T00:18:45.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.234e-06 2024-09-18T00:18:45.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395872+0000", 2024-09-18T00:18:45.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1409e-05 2024-09-18T00:18:45.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395892+0000", 2024-09-18T00:18:45.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9680000000000001e-05 2024-09-18T00:18:45.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395911+0000", 2024-09-18T00:18:45.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9283000000000002e-05 2024-09-18T00:18:45.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442210+0000", 2024-09-18T00:18:45.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046298372999999997 2024-09-18T00:18:45.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442279+0000", 2024-09-18T00:18:45.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9055999999999997e-05 2024-09-18T00:18:45.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442358+0000", 2024-09-18T00:18:45.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9363000000000006e-05 2024-09-18T00:18:45.814 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.814 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.814 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.815 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12551 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.815 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:18:45.815 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3426414050000002, 2024-09-18T00:18:45.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046066902999999999, 2024-09-18T00:18:45.815 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.815 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.815 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.815 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.815 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.815 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.815 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12551 2024-09-18T00:18:45.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.816 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:18:45.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:18:45.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396534+0000", 2024-09-18T00:18:45.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6869999999999999e-06 2024-09-18T00:18:45.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396543+0000", 2024-09-18T00:18:45.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3430000000000002e-06 2024-09-18T00:18:45.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396563+0000", 2024-09-18T00:18:45.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0063000000000001e-05 2024-09-18T00:18:45.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396579+0000", 2024-09-18T00:18:45.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5610000000000001e-05 2024-09-18T00:18:45.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442434+0000", 2024-09-18T00:18:45.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045855094999999998 2024-09-18T00:18:45.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442515+0000", 2024-09-18T00:18:45.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0988000000000005e-05 2024-09-18T00:18:45.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442599+0000", 2024-09-18T00:18:45.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4116999999999994e-05 2024-09-18T00:18:45.819 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.819 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.819 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.819 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12552 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.819 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:18:45.819 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3425666029999999, 2024-09-18T00:18:45.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046210804000000001, 2024-09-18T00:18:45.820 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.820 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.820 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.820 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.820 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.820 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.820 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12552 2024-09-18T00:18:45.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.820 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:18:45.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:18:45.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396608+0000", 2024-09-18T00:18:45.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4729999999999999e-06 2024-09-18T00:18:45.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396614+0000", 2024-09-18T00:18:45.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1e-06 2024-09-18T00:18:45.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396633+0000", 2024-09-18T00:18:45.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9009e-05 2024-09-18T00:18:45.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396650+0000", 2024-09-18T00:18:45.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7045999999999999e-05 2024-09-18T00:18:45.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442662+0000", 2024-09-18T00:18:45.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046011170999999997 2024-09-18T00:18:45.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442737+0000", 2024-09-18T00:18:45.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5759000000000004e-05 2024-09-18T00:18:45.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442818+0000", 2024-09-18T00:18:45.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0246000000000004e-05 2024-09-18T00:18:45.823 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.823 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.824 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.824 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12553 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.824 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:18:45.824 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3425038990000004, 2024-09-18T00:18:45.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046339454000000002, 2024-09-18T00:18:45.824 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.824 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.824 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.824 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.824 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.824 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.825 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12553 2024-09-18T00:18:45.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.825 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:18:45.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:18:45.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396671+0000", 2024-09-18T00:18:45.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.314e-06 2024-09-18T00:18:45.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396676+0000", 2024-09-18T00:18:45.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6659999999999996e-06 2024-09-18T00:18:45.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396695+0000", 2024-09-18T00:18:45.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8383e-05 2024-09-18T00:18:45.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396709+0000", 2024-09-18T00:18:45.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4309e-05 2024-09-18T00:18:45.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442871+0000", 2024-09-18T00:18:45.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046162295999999998 2024-09-18T00:18:45.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442939+0000", 2024-09-18T00:18:45.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7607999999999994e-05 2024-09-18T00:18:45.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443009+0000", 2024-09-18T00:18:45.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9877999999999997e-05 2024-09-18T00:18:45.828 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.828 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.828 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.828 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12554 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.828 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:18:45.828 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3424459770000006, 2024-09-18T00:18:45.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046504050999999998, 2024-09-18T00:18:45.828 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.829 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.829 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:45.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.829 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12554 2024-09-18T00:18:45.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.829 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:18:45.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:18:45.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396729+0000", 2024-09-18T00:18:45.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2559999999999999e-06 2024-09-18T00:18:45.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396736+0000", 2024-09-18T00:18:45.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6140000000000003e-06 2024-09-18T00:18:45.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396755+0000", 2024-09-18T00:18:45.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8221999999999999e-05 2024-09-18T00:18:45.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396769+0000", 2024-09-18T00:18:45.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4372999999999999e-05 2024-09-18T00:18:45.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443078+0000", 2024-09-18T00:18:45.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046308850999999998 2024-09-18T00:18:45.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443152+0000", 2024-09-18T00:18:45.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4083000000000002e-05 2024-09-18T00:18:45.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443231+0000", 2024-09-18T00:18:45.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9652000000000002e-05 2024-09-18T00:18:45.832 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.832 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.832 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.833 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12632 3.3 3:c21fac02:::rbd_data.112a7b1589a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1490944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:45.833 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:18:45.833 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.335887692, 2024-09-18T00:18:45.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045843916999999998, 2024-09-18T00:18:45.833 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:45.833 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:45.833 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:45.833 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:45.833 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:45.833 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:45.833 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12632 2024-09-18T00:18:45.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.834 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:45.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:45.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:18:45.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:45.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:18:45.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:45.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:45.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403288+0000", 2024-09-18T00:18:45.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3630000000000001e-06 2024-09-18T00:18:45.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:45.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403886+0000", 2024-09-18T00:18:45.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00059840100000000003 2024-09-18T00:18:45.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:45.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403913+0000", 2024-09-18T00:18:45.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6376e-05 2024-09-18T00:18:45.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:45.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403927+0000", 2024-09-18T00:18:45.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4188e-05 2024-09-18T00:18:45.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:45.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443268+0000", 2024-09-18T00:18:45.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.039340591000000001 2024-09-18T00:18:45.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:45.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443488+0000", 2024-09-18T00:18:45.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000220545 2024-09-18T00:18:45.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:45.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443918+0000", 2024-09-18T00:18:45.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043021800000000001 2024-09-18T00:18:45.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:45.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.448558+0000", 2024-09-18T00:18:45.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046391449999999999 2024-09-18T00:18:45.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:45.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449015+0000", 2024-09-18T00:18:45.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045766600000000002 2024-09-18T00:18:45.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:45.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449085+0000", 2024-09-18T00:18:45.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0191999999999995e-05 2024-09-18T00:18:45.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:45.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:45.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:45.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449130+0000", 2024-09-18T00:18:45.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4231999999999999e-05 2024-09-18T00:18:45.838 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.838 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.838 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.838 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:45.838 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:45.838 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:45.839 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:45.738+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:45.840 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:18:45.840 INFO:teuthology.orchestra.run.smithi097.stdout:2626 write 0xece52f4 thru 0xece5621 (0x32e bytes) 2024-09-18T00:18:45.841 INFO:teuthology.orchestra.run.smithi097.stdout:2627 writesame 0xaa3a03f thru 0xaa42e34 data_size 0x1b(0x8df6 bytes) 2024-09-18T00:18:45.841 INFO:teuthology.orchestra.run.smithi097.stdout:2629 trunc from 0xece5622 to 0x405a151 2024-09-18T00:18:45.841 INFO:teuthology.orchestra.run.smithi097.stdout:3.6 2024-09-18T00:18:45.841 INFO:teuthology.orchestra.run.smithi097.stdout:3.8 2024-09-18T00:18:45.841 INFO:teuthology.orchestra.run.smithi097.stdout:3.4 2024-09-18T00:18:45.841 INFO:teuthology.orchestra.run.smithi097.stdout:3.1a 2024-09-18T00:18:45.841 INFO:teuthology.orchestra.run.smithi097.stdout:3.19 2024-09-18T00:18:45.841 INFO:teuthology.orchestra.run.smithi097.stdout:3.0 2024-09-18T00:18:45.841 INFO:teuthology.orchestra.run.smithi097.stdout:2.6 2024-09-18T00:18:45.841 INFO:teuthology.orchestra.run.smithi097.stdout:2.4 2024-09-18T00:18:45.842 INFO:teuthology.orchestra.run.smithi097.stdout:2.2 2024-09-18T00:18:45.842 INFO:teuthology.orchestra.run.smithi097.stdout:2.0 2024-09-18T00:18:45.842 INFO:teuthology.orchestra.run.smithi097.stdout:2631 read 0x2ebc989 thru 0x2ec4845 (0x7ebd bytes) 2024-09-18T00:18:45.842 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:45.842 INFO:teuthology.orchestra.run.smithi097.stdout:2633 punch from 0xcb834d to 0xcc4771, (0xc424 bytes) 2024-09-18T00:18:45.842 INFO:teuthology.orchestra.run.smithi097.stdout:2634 read 0xf90239 thru 0xf95fb3 (0x5d7b bytes) 2024-09-18T00:18:45.843 INFO:teuthology.orchestra.run.smithi097.stdout:2635 writesame 0x69509cc thru 0x695ed42 data_size 0x1(0xe377 bytes) 2024-09-18T00:18:45.843 INFO:teuthology.orchestra.run.smithi097.stdout:2636 write 0x86a87b5 thru 0x86b1af9 (0x9345 bytes) 2024-09-18T00:18:45.843 INFO:teuthology.orchestra.run.smithi097.stdout:2637 trunc from 0x86b1afa to 0x77837b3 2024-09-18T00:18:45.843 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:45.844 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:45.844 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:45.838+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:45.844 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:18:45.854 INFO:teuthology.orchestra.run.smithi097.stdout:2638 write 0x8997685 thru 0x8997a86 (0x402 bytes) 2024-09-18T00:18:45.859 INFO:teuthology.orchestra.run.smithi097.stdout:2639 punch from 0x72cea17 to 0x72d18b7, (0x2ea0 bytes) 2024-09-18T00:18:45.860 INFO:teuthology.orchestra.run.smithi097.stdout:2640 punch from 0x12fea6 to 0x13e702, (0xe85c bytes) 2024-09-18T00:18:45.860 INFO:teuthology.orchestra.run.smithi097.stdout:2641 write 0x4c1e057 thru 0x4c1ef30 (0xeda bytes) 2024-09-18T00:18:45.861 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:45.861 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:45.868 INFO:teuthology.orchestra.run.smithi097.stdout:2646 trunc from 0x8997a87 to 0xb0b4e21 2024-09-18T00:18:45.878 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:45.879 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:45.890 INFO:teuthology.orchestra.run.smithi097.stdout:2648 writesame 0xe44708d thru 0xe456131 data_size 0x2d(0xf0a5 bytes) 2024-09-18T00:18:45.890 INFO:teuthology.orchestra.run.smithi097.stdout:2649 writesame 0x11f29eb thru 0x11f34c2 data_size 0x4(0xad8 bytes) 2024-09-18T00:18:45.898 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:45.898 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:45.898 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-18T00:18:45.899 INFO:teuthology.orchestra.run.smithi097.stdout:2650 write 0x538245 thru 0x539217 (0xfd3 bytes) 2024-09-18T00:18:45.915 INFO:teuthology.orchestra.run.smithi097.stdout:2652 read 0x157ffb7 thru 0x158277a (0x27c4 bytes) 2024-09-18T00:18:45.915 INFO:teuthology.orchestra.run.smithi097.stdout:2653 write 0xc1d7264 thru 0xc1d9459 (0x21f6 bytes) 2024-09-18T00:18:45.917 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:45.920 INFO:teuthology.orchestra.run.smithi097.stdout:2654 read 0x371f5eb thru 0x372e34a (0xed60 bytes) 2024-09-18T00:18:45.936 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:45.938 INFO:teuthology.orchestra.run.smithi097.stdout:2655 trunc from 0xe456132 to 0x19407fc 2024-09-18T00:18:45.939 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:18:45.940 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:18:45.945 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:45.938+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:45.961 INFO:teuthology.orchestra.run.smithi097.stdout:2656 read 0x1827de thru 0x18c99c (0xa1bf bytes) 2024-09-18T00:18:45.961 INFO:teuthology.orchestra.run.smithi097.stdout:2657 read 0x14f527e thru 0x14f7474 (0x21f7 bytes) 2024-09-18T00:18:45.974 INFO:teuthology.orchestra.run.smithi097.stdout:2660 write 0x4859dbd thru 0x4867266 (0xd4aa bytes) 2024-09-18T00:18:45.981 INFO:teuthology.orchestra.run.smithi097.stdout:2661 punch from 0x1d9c60a to 0x1da059b, (0x3f91 bytes) 2024-09-18T00:18:45.981 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:46.007 INFO:teuthology.orchestra.run.smithi097.stdout:2663 write 0xd4b314d thru 0xd4bfa50 (0xc904 bytes) 2024-09-18T00:18:46.014 INFO:teuthology.orchestra.run.smithi097.stdout:2665 write 0x57ea1ff thru 0x57f2249 (0x804b bytes) 2024-09-18T00:18:46.029 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:46.030 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:46.038 INFO:teuthology.orchestra.run.smithi097.stdout:2666 trunc from 0xd4bfa51 to 0x243b2cc 2024-09-18T00:18:46.046 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:18:46.046 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:46.042+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:46.067 INFO:teuthology.orchestra.run.smithi097.stdout:2668 writesame 0x43c2ae3 thru 0x43d14be data_size 0x4(0xe9dc bytes) 2024-09-18T00:18:46.067 INFO:teuthology.orchestra.run.smithi097.stdout:2670 punch from 0x295fea7 to 0x2969635, (0x978e bytes) 2024-09-18T00:18:46.084 INFO:teuthology.orchestra.run.smithi097.stdout:2671 write 0x91bb58e thru 0x91c81d6 (0xcc49 bytes) 2024-09-18T00:18:46.094 INFO:teuthology.orchestra.run.smithi097.stdout:2672 trunc from 0x91c81d7 to 0x7bc1064 2024-09-18T00:18:46.112 INFO:teuthology.orchestra.run.smithi097.stdout:2673 write 0x253e840 thru 0x2545222 (0x69e3 bytes) 2024-09-18T00:18:46.120 INFO:teuthology.orchestra.run.smithi097.stdout:2677 read 0x106647f thru 0x1074a81 (0xe603 bytes) 2024-09-18T00:18:46.121 INFO:teuthology.orchestra.run.smithi097.stdout:2678 punch from 0x5887ff9 to 0x5894220, (0xc227 bytes) 2024-09-18T00:18:46.121 INFO:teuthology.orchestra.run.smithi097.stdout:2679 read 0x2a6c95e thru 0x2a7a458 (0xdafb bytes) 2024-09-18T00:18:46.121 INFO:teuthology.orchestra.run.smithi097.stdout:2681 punch from 0x6d5376b to 0x6d5f490, (0xbd25 bytes) 2024-09-18T00:18:46.121 INFO:teuthology.orchestra.run.smithi097.stdout:2682 punch from 0x502db5e to 0x503b15d, (0xd5ff bytes) 2024-09-18T00:18:46.121 INFO:teuthology.orchestra.run.smithi097.stdout:2683 read 0x4ed0f30 thru 0x4edd8c0 (0xc991 bytes) 2024-09-18T00:18:46.122 INFO:teuthology.orchestra.run.smithi097.stdout:2687 trunc from 0x7bc1064 to 0x838b0bc 2024-09-18T00:18:46.133 INFO:teuthology.orchestra.run.smithi097.stdout:2688 write 0x65dd6b4 thru 0x65e7788 (0xa0d5 bytes) 2024-09-18T00:18:46.147 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:46.142+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:46.160 INFO:teuthology.orchestra.run.smithi097.stdout:2690 write 0xab76912 thru 0xab7a75e (0x3e4d bytes) 2024-09-18T00:18:46.180 INFO:teuthology.orchestra.run.smithi097.stdout:2692 writesame 0xacc7903 thru 0xacd46ed data_size 0xd(0xcdeb bytes) 2024-09-18T00:18:46.180 INFO:teuthology.orchestra.run.smithi097.stdout:2694 write 0x50cf36e thru 0x50de7d4 (0xf467 bytes) 2024-09-18T00:18:46.195 INFO:teuthology.orchestra.run.smithi097.stdout:2695 punch from 0x45f4439 to 0x45fb4a2, (0x7069 bytes) 2024-09-18T00:18:46.214 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:46.215 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:46.220 INFO:teuthology.orchestra.run.smithi097.stdout:2696 trunc from 0xacd46ee to 0xda7405 2024-09-18T00:18:46.230 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:18:46.248 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:46.242+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:46.249 INFO:teuthology.orchestra.run.smithi097.stdout:2700 punch from 0x27396e to 0x275dcc, (0x245e bytes) 2024-09-18T00:18:46.262 INFO:teuthology.orchestra.run.smithi097.stdout:2701 write 0x2f8a8bd thru 0x2f97eaf (0xd5f3 bytes) 2024-09-18T00:18:46.287 INFO:teuthology.orchestra.run.smithi097.stdout:2702 write 0x606e1e8 thru 0x6071dc9 (0x3be2 bytes) 2024-09-18T00:18:46.293 INFO:teuthology.orchestra.run.smithi097.stdout:2704 read 0x95768c thru 0x9645f6 (0xcf6b bytes) 2024-09-18T00:18:46.294 INFO:teuthology.orchestra.run.smithi097.stdout:2705 punch from 0xf14f03 to 0xf15009, (0x106 bytes) 2024-09-18T00:18:46.321 INFO:teuthology.orchestra.run.smithi097.stdout:2706 write 0xac5cb8a thru 0xac63709 (0x6b80 bytes) 2024-09-18T00:18:46.328 INFO:teuthology.orchestra.run.smithi097.stdout:2707 write 0x3c0c73d thru 0x3c125c0 (0x5e84 bytes) 2024-09-18T00:18:46.336 INFO:teuthology.orchestra.run.smithi097.stdout:2708 punch from 0x5935435 to 0x5938ff7, (0x3bc2 bytes) 2024-09-18T00:18:46.336 INFO:teuthology.orchestra.run.smithi097.stdout:2709 read 0x470447a thru 0x470c8f0 (0x8477 bytes) 2024-09-18T00:18:46.348 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:46.342+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:46.351 INFO:teuthology.orchestra.run.smithi097.stdout:2710 write 0xc5e1cfc thru 0xc5e319c (0x14a1 bytes) 2024-09-18T00:18:46.352 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op export --pgid 2.4 --file /var/log/ceph/exp.2.4.1 2024-09-18T00:18:46.354 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:46.355 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:46.357 INFO:teuthology.orchestra.run.smithi097.stdout:2711 punch from 0x84c9a59 to 0x84cfb9a, (0x6141 bytes) 2024-09-18T00:18:46.359 INFO:teuthology.orchestra.run.smithi097.stdout:2712 trunc from 0xc5e319d to 0xc060025 2024-09-18T00:18:46.370 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:18:46.377 INFO:teuthology.orchestra.run.smithi097.stdout:2713 punch from 0x2e3eaef to 0x2e47485, (0x8996 bytes) 2024-09-18T00:18:46.377 INFO:teuthology.orchestra.run.smithi097.stdout:2715 punch from 0x4705c69 to 0x470dbf0, (0x7f87 bytes) 2024-09-18T00:18:46.377 INFO:teuthology.orchestra.run.smithi097.stdout:2716 punch from 0x8b8f75e to 0x8b9940f, (0x9cb1 bytes) 2024-09-18T00:18:46.382 INFO:teuthology.orchestra.run.smithi097.stdout:2717 trunc from 0xc060025 to 0xa515713 2024-09-18T00:18:46.393 INFO:teuthology.orchestra.run.smithi097.stdout:2718 read 0x23fed99 thru 0x2401743 (0x29ab bytes) 2024-09-18T00:18:46.410 INFO:teuthology.orchestra.run.smithi097.stdout:2720 trunc from 0xa515713 to 0x2e2875c 2024-09-18T00:18:46.425 INFO:teuthology.orchestra.run.smithi097.stdout:2723 read 0x1824b2b thru 0x182a36d (0x5843 bytes) 2024-09-18T00:18:46.425 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:46.425 INFO:teuthology.orchestra.run.smithi097.stdout:2726 read 0x83f5ca thru 0x84ae81 (0xb8b8 bytes) 2024-09-18T00:18:46.426 INFO:teuthology.orchestra.run.smithi097.stdout:2727 write 0x1e27e9a thru 0x1e32a7a (0xabe1 bytes) 2024-09-18T00:18:46.433 INFO:teuthology.orchestra.run.smithi097.stdout:2728 read 0x29e6999 thru 0x29ecd09 (0x6371 bytes) 2024-09-18T00:18:46.447 INFO:teuthology.orchestra.run.smithi097.stdout:2729 write 0x55902c9 thru 0x5598c16 (0x894e bytes) 2024-09-18T00:18:46.449 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:46.446+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:46.452 INFO:teuthology.orchestra.run.smithi097.stdout:2730 writesame 0x41f2f03 thru 0x420244e data_size 0x6(0xf54c bytes) 2024-09-18T00:18:46.463 INFO:teuthology.orchestra.run.smithi097.stdout:2731 punch from 0x2405dd9 to 0x2410bbc, (0xade3 bytes) 2024-09-18T00:18:46.464 INFO:teuthology.orchestra.run.smithi097.stdout:2732 read 0x1dc382a thru 0x1dc912c (0x5903 bytes) 2024-09-18T00:18:46.464 INFO:teuthology.orchestra.run.smithi097.stdout:2733 write 0x5c2349 thru 0x5c86d6 (0x638e bytes) 2024-09-18T00:18:46.473 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:46.499 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:46.499 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:46.504 INFO:teuthology.orchestra.run.smithi097.stdout:2735 write 0xdb72951 thru 0xdb78e96 (0x6546 bytes) 2024-09-18T00:18:46.510 INFO:teuthology.orchestra.run.smithi097.stdout:2736 punch from 0x3c652e8 to 0x3c6dfe5, (0x8cfd bytes) 2024-09-18T00:18:46.510 INFO:teuthology.orchestra.run.smithi097.stdout:truncating to largest ever: 0xed663e9 2024-09-18T00:18:46.513 INFO:teuthology.orchestra.run.smithi097.stdout:2737 trunc from 0xdb78e97 to 0xed663e9 2024-09-18T00:18:46.516 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:18:46.522 INFO:teuthology.orchestra.run.smithi097.stdout:2739 writesame 0x3ac4125 thru 0x3ac7a4b data_size 0x1(0x3927 bytes) 2024-09-18T00:18:46.562 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:46.558+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:46.566 INFO:teuthology.orchestra.run.smithi097.stdout:2740 write 0x33d9ce1 thru 0x33e4aa3 (0xadc3 bytes) 2024-09-18T00:18:46.573 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:46.573 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:46.574 INFO:teuthology.orchestra.run.smithi097.stdout:2747 read 0x2cb2ae thru 0x2ce23a (0x2f8d bytes) 2024-09-18T00:18:46.575 INFO:teuthology.orchestra.run.smithi097.stdout:2751 write 0x3d232b1 thru 0x3d293a4 (0x60f4 bytes) 2024-09-18T00:18:46.579 INFO:teuthology.orchestra.run.smithi097.stdout:2752 write 0xe26b5cb thru 0xe26c098 (0xace bytes) 2024-09-18T00:18:46.584 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:46.584 INFO:teuthology.orchestra.run.smithi097.stdout:2754 write 0x89d3212 thru 0x89de065 (0xae54 bytes) 2024-09-18T00:18:46.593 INFO:teuthology.orchestra.run.smithi097.stdout:2755 read 0xb227e6c thru 0xb235af2 (0xdc87 bytes) 2024-09-18T00:18:46.605 INFO:teuthology.orchestra.run.smithi097.stdout:2756 trunc from 0xed663e9 to 0xaa2766e 2024-09-18T00:18:46.623 INFO:teuthology.orchestra.run.smithi097.stdout:2757 read 0x951044c thru 0x9514e4f (0x4a04 bytes) 2024-09-18T00:18:46.623 INFO:teuthology.orchestra.run.smithi097.stdout:2758 read 0x2049a80 thru 0x204b318 (0x1899 bytes) 2024-09-18T00:18:46.623 INFO:teuthology.orchestra.run.smithi097.stdout:2760 punch from 0x866b5cf to 0x8673f37, (0x8968 bytes) 2024-09-18T00:18:46.643 INFO:teuthology.orchestra.run.smithi097.stdout:2761 trunc from 0xaa2766e to 0x3071d91 2024-09-18T00:18:46.646 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:46.646 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:46.651 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:46.646+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:46.663 INFO:teuthology.orchestra.run.smithi097.stdout:2762 read 0xdf9fe4 thru 0xe009fc (0x6a19 bytes) 2024-09-18T00:18:46.665 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:18:46.665 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:46.672 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is unset 2024-09-18T00:18:46.701 INFO:teuthology.orchestra.run.smithi097.stdout:2764 write 0xe814edf thru 0xe8156e6 (0x808 bytes) 2024-09-18T00:18:46.704 INFO:teuthology.orchestra.run.smithi097.stdout:2765 write 0x7dc29aa thru 0x7dc5513 (0x2b6a bytes) 2024-09-18T00:18:46.709 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:46.709 INFO:teuthology.orchestra.run.smithi097.stdout:2769 write 0x4646519 thru 0x464a2c1 (0x3da9 bytes) 2024-09-18T00:18:46.719 INFO:teuthology.orchestra.run.smithi097.stdout:2770 trunc from 0xe8156e7 to 0xc422e12 2024-09-18T00:18:46.735 INFO:teuthology.orchestra.run.smithi097.stdout:2771 write 0x1a1a327 thru 0x1a1ecf4 (0x49ce bytes) 2024-09-18T00:18:46.740 INFO:teuthology.orchestra.run.smithi097.stdout:2772 punch from 0xaef9921 to 0xaf079ae, (0xe08d bytes) 2024-09-18T00:18:46.740 INFO:teuthology.orchestra.run.smithi097.stdout:2773 write 0x2348233 thru 0x2356d41 (0xeb0f bytes) 2024-09-18T00:18:46.746 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:46.746 INFO:teuthology.orchestra.run.smithi097.stdout:2775 read 0xc0eed36 thru 0xc0f21e6 (0x34b1 bytes) 2024-09-18T00:18:46.746 INFO:teuthology.orchestra.run.smithi097.stdout:2776 write 0xb56239 thru 0xb5a2e5 (0x40ad bytes) 2024-09-18T00:18:46.751 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:46.746+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:46.754 INFO:teuthology.orchestra.run.smithi097.stdout:2777 write 0x9a5d4af thru 0x9a6a9d3 (0xd525 bytes) 2024-09-18T00:18:46.760 INFO:teuthology.orchestra.run.smithi097.stdout:2778 write 0x227a20c thru 0x227d262 (0x3057 bytes) 2024-09-18T00:18:46.764 INFO:teuthology.orchestra.run.smithi097.stdout:2781 write 0x52eb7b8 thru 0x52f27d5 (0x701e bytes) 2024-09-18T00:18:46.771 INFO:teuthology.orchestra.run.smithi097.stdout:2783 writesame 0x15e0130 thru 0x15e0e61 data_size 0x3(0xd32 bytes) 2024-09-18T00:18:46.777 INFO:teuthology.orchestra.run.smithi097.stdout:2784 read 0x54ad997 thru 0x54af038 (0x16a2 bytes) 2024-09-18T00:18:46.777 INFO:teuthology.orchestra.run.smithi097.stdout:2785 read 0x8cd6dae thru 0x8cd7907 (0xb5a bytes) 2024-09-18T00:18:46.778 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:46.778 INFO:teuthology.orchestra.run.smithi097.stdout:2787 write 0xa9350da thru 0xa935705 (0x62c bytes) 2024-09-18T00:18:46.781 INFO:teuthology.orchestra.run.smithi097.stdout:2789 writesame 0x2512492 thru 0x251cabc data_size 0x3b(0xa62b bytes) 2024-09-18T00:18:46.790 INFO:teuthology.orchestra.run.smithi097.stdout:2792 trunc from 0xc422e12 to 0xe98f4f3 2024-09-18T00:18:46.798 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:46.813 INFO:teuthology.orchestra.run.smithi097.stdout:2797 trunc from 0xe98f4f3 to 0x54c7d92 2024-09-18T00:18:46.833 INFO:teuthology.orchestra.run.smithi097.stdout:2798 clone 12 order 25 su 1048576 sc 9 2024-09-18T00:18:46.838 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:46.838 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:46.852 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:46.846+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:46.853 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:18:46.953 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:46.950+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:46.977 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:46.978 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:46.992 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:18:47.077 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:47.074+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:47.154 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:47.150+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:47.157 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:47.157 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:47.177 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:18:47.255 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:47.254+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:47.334 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:47.334 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:47.347 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:18:47.393 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:47.390+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:47.457 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:47.450+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:47.464 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:47.464 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:47.479 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:18:47.573 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:47.570+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:47.651 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:47.651 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:47.658 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:47.654+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:47.664 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:18:47.759 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:47.754+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:47.837 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:47.838 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:47.845 INFO:teuthology.orchestra.run.smithi097.stdout:leaving image image_client.0-clone11 intact 2024-09-18T00:18:47.853 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:18:47.897 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:47.894+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:47.940 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:18:47.961 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:47.954+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:47.982 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:47.983 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:48.000 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:18:48.011 INFO:teuthology.orchestra.run.smithi097.stderr:Exporting 2.4 info 2.4( empty local-lis/les=19/20 n=0 ec=19/19 lis/c=19/19 les/c/f=20/20/0 sis=19) 2024-09-18T00:18:48.016 INFO:teuthology.orchestra.run.smithi097.stderr:Export successful 2024-09-18T00:18:48.024 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-18T00:18:48.061 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:48.058+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:48.095 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:48.095 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:48.095 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:18:48.113 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:48.113 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:48.130 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:48.131 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:48.138 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:48.138 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:48.138 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:48.138 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:48.138 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:48.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:18:48.156 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:48.156 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:48.156 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-18T00:18:48.177 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:48.177 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:48.178 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:48.174+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:48.199 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:48.199 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:48.202 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:18:48.203 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:18:48.263 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:48.258+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:48.299 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:599 3.15 3:ac22c048:::rbd_data.112ae20746f5.0000000000000110:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:11.980706+0000", 2024-09-18T00:18:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.316167749999998, 2024-09-18T00:18:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017399257000000001, 2024-09-18T00:18:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 599 2024-09-18T00:18:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980706+0000", 2024-09-18T00:18:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980706+0000", 2024-09-18T00:18:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980708+0000", 2024-09-18T00:18:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5919999999999999e-06 2024-09-18T00:18:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980722+0000", 2024-09-18T00:18:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.364e-05 2024-09-18T00:18:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980752+0000", 2024-09-18T00:18:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9794999999999998e-05 2024-09-18T00:18:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980778+0000", 2024-09-18T00:18:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6494000000000001e-05 2024-09-18T00:18:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.980885+0000", 2024-09-18T00:18:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000107104 2024-09-18T00:18:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.981079+0000", 2024-09-18T00:18:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019375699999999999 2024-09-18T00:18:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.981866+0000", 2024-09-18T00:18:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00078716099999999998 2024-09-18T00:18:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.997852+0000", 2024-09-18T00:18:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015986305999999999 2024-09-18T00:18:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.997957+0000", 2024-09-18T00:18:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104085 2024-09-18T00:18:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.998037+0000", 2024-09-18T00:18:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0098000000000005e-05 2024-09-18T00:18:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:11.998105+0000", 2024-09-18T00:18:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8225000000000005e-05 2024-09-18T00:18:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:640 3.7 3:e0810bc8:::rbd_data.112ae20746f5.0000000000000019:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T00:18:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.104726+0000", 2024-09-18T00:18:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.192146921999999, 2024-09-18T00:18:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017067268, 2024-09-18T00:18:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 640 2024-09-18T00:18:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104726+0000", 2024-09-18T00:18:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104726+0000", 2024-09-18T00:18:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104728+0000", 2024-09-18T00:18:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.336e-06 2024-09-18T00:18:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104735+0000", 2024-09-18T00:18:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6149999999999999e-06 2024-09-18T00:18:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104753+0000", 2024-09-18T00:18:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7362000000000001e-05 2024-09-18T00:18:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104761+0000", 2024-09-18T00:18:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5609999999999999e-06 2024-09-18T00:18:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104842+0000", 2024-09-18T00:18:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0526999999999997e-05 2024-09-18T00:18:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.104916+0000", 2024-09-18T00:18:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3724000000000001e-05 2024-09-18T00:18:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.106201+0000", 2024-09-18T00:18:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001285046 2024-09-18T00:18:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121677+0000", 2024-09-18T00:18:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01547665 2024-09-18T00:18:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121696+0000", 2024-09-18T00:18:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8304999999999998e-05 2024-09-18T00:18:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121744+0000", 2024-09-18T00:18:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8415000000000003e-05 2024-09-18T00:18:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.121794+0000", 2024-09-18T00:18:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9727e-05 2024-09-18T00:18:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.496706680000003, 2024-09-18T00:18:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:18:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:18:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:18:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:18:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:18:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:18:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:18:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:18:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:18:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:18:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:18:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:18:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:18:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:18:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:18:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:18:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:18:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:18:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:18:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:18:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:18:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:18:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:18:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:18:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:18:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.480498806999996, 2024-09-18T00:18:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:18:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:18:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:18:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:18:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:18:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:18:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:18:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:18:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:18:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:18:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:18:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:18:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:18:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:18:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:18:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:5724 3.1e 3:78324145:::rbd_data.112a3a39fc45.000000000000001e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.341891+0000", 2024-09-18T00:18:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.954982344000001, 2024-09-18T00:18:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014843063, 2024-09-18T00:18:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 5724 2024-09-18T00:18:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.341891+0000", 2024-09-18T00:18:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.341891+0000", 2024-09-18T00:18:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.341893+0000", 2024-09-18T00:18:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1129999999999999e-06 2024-09-18T00:18:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343233+0000", 2024-09-18T00:18:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001339614 2024-09-18T00:18:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343251+0000", 2024-09-18T00:18:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8479e-05 2024-09-18T00:18:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343260+0000", 2024-09-18T00:18:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1470000000000007e-06 2024-09-18T00:18:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343333+0000", 2024-09-18T00:18:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2302e-05 2024-09-18T00:18:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343494+0000", 2024-09-18T00:18:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000161593 2024-09-18T00:18:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 6", 2024-09-18T00:18:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.343778+0000", 2024-09-18T00:18:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00028366099999999999 2024-09-18T00:18:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.351688+0000", 2024-09-18T00:18:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0079100739999999996 2024-09-18T00:18:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.356673+0000", 2024-09-18T00:18:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0049848210000000004 2024-09-18T00:18:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.356706+0000", 2024-09-18T00:18:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3531000000000001e-05 2024-09-18T00:18:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.356734+0000", 2024-09-18T00:18:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7727999999999999e-05 2024-09-18T00:18:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6175 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 8585216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.473970+0000", 2024-09-18T00:18:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.822902971000001, 2024-09-18T00:18:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013962384, 2024-09-18T00:18:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6175 2024-09-18T00:18:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473970+0000", 2024-09-18T00:18:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473970+0000", 2024-09-18T00:18:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473972+0000", 2024-09-18T00:18:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1310000000000001e-06 2024-09-18T00:18:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473977+0000", 2024-09-18T00:18:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1769999999999997e-06 2024-09-18T00:18:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.473993+0000", 2024-09-18T00:18:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5988999999999999e-05 2024-09-18T00:18:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474005+0000", 2024-09-18T00:18:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2622999999999999e-05 2024-09-18T00:18:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.485712+0000", 2024-09-18T00:18:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011706701 2024-09-18T00:18:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487289+0000", 2024-09-18T00:18:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0015770280000000001 2024-09-18T00:18:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487933+0000", 2024-09-18T00:18:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00064373499999999999 2024-09-18T00:18:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6176 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 9371648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474022+0000", 2024-09-18T00:18:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.822851706000002, 2024-09-18T00:18:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014183915, 2024-09-18T00:18:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6176 2024-09-18T00:18:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474022+0000", 2024-09-18T00:18:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474022+0000", 2024-09-18T00:18:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474023+0000", 2024-09-18T00:18:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.02e-06 2024-09-18T00:18:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474028+0000", 2024-09-18T00:18:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9599999999999999e-06 2024-09-18T00:18:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474044+0000", 2024-09-18T00:18:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6113999999999999e-05 2024-09-18T00:18:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474056+0000", 2024-09-18T00:18:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2349e-05 2024-09-18T00:18:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.487994+0000", 2024-09-18T00:18:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013937748 2024-09-18T00:18:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488088+0000", 2024-09-18T00:18:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3888999999999995e-05 2024-09-18T00:18:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488206+0000", 2024-09-18T00:18:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000117835 2024-09-18T00:18:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6177 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474074+0000", 2024-09-18T00:18:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.822799037999999, 2024-09-18T00:18:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01434039, 2024-09-18T00:18:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6177 2024-09-18T00:18:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474074+0000", 2024-09-18T00:18:48.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474074+0000", 2024-09-18T00:18:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474075+0000", 2024-09-18T00:18:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5999999999999991e-07 2024-09-18T00:18:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474080+0000", 2024-09-18T00:18:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9459999999999997e-06 2024-09-18T00:18:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474097+0000", 2024-09-18T00:18:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6705999999999999e-05 2024-09-18T00:18:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474110+0000", 2024-09-18T00:18:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2582000000000001e-05 2024-09-18T00:18:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488252+0000", 2024-09-18T00:18:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014142937 2024-09-18T00:18:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488327+0000", 2024-09-18T00:18:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4184000000000007e-05 2024-09-18T00:18:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488415+0000", 2024-09-18T00:18:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8075e-05 2024-09-18T00:18:48.341 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6178 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474128+0000", 2024-09-18T00:18:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.822744880999998, 2024-09-18T00:18:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014460381, 2024-09-18T00:18:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.342 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6178 2024-09-18T00:18:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474128+0000", 2024-09-18T00:18:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474128+0000", 2024-09-18T00:18:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474130+0000", 2024-09-18T00:18:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1489999999999999e-06 2024-09-18T00:18:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474135+0000", 2024-09-18T00:18:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0830000000000003e-06 2024-09-18T00:18:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474151+0000", 2024-09-18T00:18:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6677e-05 2024-09-18T00:18:48.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474166+0000", 2024-09-18T00:18:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4871e-05 2024-09-18T00:18:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488458+0000", 2024-09-18T00:18:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014292206999999999 2024-09-18T00:18:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488516+0000", 2024-09-18T00:18:48.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7627999999999998e-05 2024-09-18T00:18:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488589+0000", 2024-09-18T00:18:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2766000000000001e-05 2024-09-18T00:18:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6179 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 11730944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474183+0000", 2024-09-18T00:18:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.822690127000001, 2024-09-18T00:18:48.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014618816, 2024-09-18T00:18:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6179 2024-09-18T00:18:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474183+0000", 2024-09-18T00:18:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474183+0000", 2024-09-18T00:18:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474184+0000", 2024-09-18T00:18:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.083e-06 2024-09-18T00:18:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474190+0000", 2024-09-18T00:18:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6570000000000002e-06 2024-09-18T00:18:48.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474206+0000", 2024-09-18T00:18:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5914e-05 2024-09-18T00:18:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474225+0000", 2024-09-18T00:18:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9369e-05 2024-09-18T00:18:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488639+0000", 2024-09-18T00:18:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014413769 2024-09-18T00:18:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488709+0000", 2024-09-18T00:18:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9655999999999999e-05 2024-09-18T00:18:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488802+0000", 2024-09-18T00:18:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3368000000000005e-05 2024-09-18T00:18:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6180 3.8 3:16b24e91:::rbd_data.112aa7ba4578.0000000000000161:5 [read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474244+0000", 2024-09-18T00:18:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.82262983, 2024-09-18T00:18:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014495918999999999, 2024-09-18T00:18:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6180 2024-09-18T00:18:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474244+0000", 2024-09-18T00:18:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474244+0000", 2024-09-18T00:18:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474245+0000", 2024-09-18T00:18:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.105e-06 2024-09-18T00:18:48.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474318+0000", 2024-09-18T00:18:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3182999999999994e-05 2024-09-18T00:18:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474345+0000", 2024-09-18T00:18:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6730999999999999e-05 2024-09-18T00:18:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474359+0000", 2024-09-18T00:18:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4433999999999999e-05 2024-09-18T00:18:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488626+0000", 2024-09-18T00:18:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014266993 2024-09-18T00:18:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488680+0000", 2024-09-18T00:18:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4453999999999997e-05 2024-09-18T00:18:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488739+0000", 2024-09-18T00:18:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9018999999999999e-05 2024-09-18T00:18:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6181 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 12517376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.474384+0000", 2024-09-18T00:18:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.822488893999999, 2024-09-18T00:18:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014686072999999999, 2024-09-18T00:18:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.355 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6181 2024-09-18T00:18:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474384+0000", 2024-09-18T00:18:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474384+0000", 2024-09-18T00:18:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474386+0000", 2024-09-18T00:18:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.688e-06 2024-09-18T00:18:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474392+0000", 2024-09-18T00:18:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.575e-06 2024-09-18T00:18:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474409+0000", 2024-09-18T00:18:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7638999999999998e-05 2024-09-18T00:18:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.474423+0000", 2024-09-18T00:18:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3237e-05 2024-09-18T00:18:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488844+0000", 2024-09-18T00:18:48.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014421079 2024-09-18T00:18:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.488943+0000", 2024-09-18T00:18:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9764000000000002e-05 2024-09-18T00:18:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.489071+0000", 2024-09-18T00:18:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012709099999999999 2024-09-18T00:18:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.359 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6280 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 11075584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.478893+0000", 2024-09-18T00:18:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.817980642999999, 2024-09-18T00:18:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012517118000000001, 2024-09-18T00:18:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6280 2024-09-18T00:18:48.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478893+0000", 2024-09-18T00:18:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478893+0000", 2024-09-18T00:18:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478894+0000", 2024-09-18T00:18:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.003e-06 2024-09-18T00:18:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478905+0000", 2024-09-18T00:18:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1045e-05 2024-09-18T00:18:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478921+0000", 2024-09-18T00:18:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6469000000000001e-05 2024-09-18T00:18:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478934+0000", 2024-09-18T00:18:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2525e-05 2024-09-18T00:18:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491258+0000", 2024-09-18T00:18:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012324422999999999 2024-09-18T00:18:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491327+0000", 2024-09-18T00:18:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8923000000000004e-05 2024-09-18T00:18:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.491410+0000", 2024-09-18T00:18:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2730000000000002e-05 2024-09-18T00:18:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6281 3.18 3:1b99d510:::rbd_data.112a960351d8.0000000000000000:6 [sparse-read 11862016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.478952+0000", 2024-09-18T00:18:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.817921244000001, 2024-09-18T00:18:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013203609, 2024-09-18T00:18:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.364 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6281 2024-09-18T00:18:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478952+0000", 2024-09-18T00:18:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478952+0000", 2024-09-18T00:18:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478953+0000", 2024-09-18T00:18:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0720000000000001e-06 2024-09-18T00:18:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.478958+0000", 2024-09-18T00:18:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7369999999999997e-06 2024-09-18T00:18:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.482143+0000", 2024-09-18T00:18:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0031850870000000001 2024-09-18T00:18:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.482179+0000", 2024-09-18T00:18:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6381000000000003e-05 2024-09-18T00:18:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.492006+0000", 2024-09-18T00:18:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0098267319999999995 2024-09-18T00:18:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.492069+0000", 2024-09-18T00:18:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2564000000000006e-05 2024-09-18T00:18:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.492156+0000", 2024-09-18T00:18:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7035999999999998e-05 2024-09-18T00:18:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6572 3.7 3:e54d9f3a:::rbd_data.112a3a39fc45.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 458184~568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.725733+0000", 2024-09-18T00:18:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.571140083, 2024-09-18T00:18:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016678148, 2024-09-18T00:18:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6572 2024-09-18T00:18:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.725733+0000", 2024-09-18T00:18:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.725733+0000", 2024-09-18T00:18:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.725736+0000", 2024-09-18T00:18:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2919999999999998e-06 2024-09-18T00:18:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726273+0000", 2024-09-18T00:18:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00053760199999999998 2024-09-18T00:18:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726295+0000", 2024-09-18T00:18:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1855999999999999e-05 2024-09-18T00:18:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726305+0000", 2024-09-18T00:18:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0098000000000001e-05 2024-09-18T00:18:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726384+0000", 2024-09-18T00:18:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8731000000000002e-05 2024-09-18T00:18:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726582+0000", 2024-09-18T00:18:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019781399999999999 2024-09-18T00:18:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.726857+0000", 2024-09-18T00:18:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027557299999999999 2024-09-18T00:18:48.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.735232+0000", 2024-09-18T00:18:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083744660000000005 2024-09-18T00:18:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.742317+0000", 2024-09-18T00:18:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0070851789999999996 2024-09-18T00:18:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.742357+0000", 2024-09-18T00:18:48.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9975000000000003e-05 2024-09-18T00:18:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.742411+0000", 2024-09-18T00:18:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4561999999999997e-05 2024-09-18T00:18:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:48.374 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.453457487000001, 2024-09-18T00:18:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:18:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:18:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:18:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:18:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:18:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:18:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:18:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:18:48.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:18:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:18:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:18:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:18:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:18:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:18:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:18:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:18:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:18:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:18:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:48.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:18:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:18:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:18:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:18:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:18:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:18:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.380 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:18:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.432180935000002, 2024-09-18T00:18:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:18:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:48.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:18:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:18:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:18:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:18:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:18:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:18:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:18:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:18:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:18:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:18:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:18:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:18:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:18:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:18:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.384 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11460 3.19 3:9e578b6c:::rbd_data.112ab1fa102a.0000000000000153:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:18:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.029809482999999, 2024-09-18T00:18:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041989011999999999, 2024-09-18T00:18:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11460 2024-09-18T00:18:48.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:18:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:18:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267067+0000", 2024-09-18T00:18:48.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:18:48.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267088+0000", 2024-09-18T00:18:48.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0908999999999999e-05 2024-09-18T00:18:48.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267133+0000", 2024-09-18T00:18:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5232000000000003e-05 2024-09-18T00:18:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267157+0000", 2024-09-18T00:18:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3833e-05 2024-09-18T00:18:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267300+0000", 2024-09-18T00:18:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142282 2024-09-18T00:18:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267447+0000", 2024-09-18T00:18:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147851 2024-09-18T00:18:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308889+0000", 2024-09-18T00:18:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041441779999999998 2024-09-18T00:18:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308956+0000", 2024-09-18T00:18:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7136000000000002e-05 2024-09-18T00:18:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.309053+0000", 2024-09-18T00:18:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6552999999999995e-05 2024-09-18T00:18:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11464 3.b 3:d331f8ac:::rbd_data.112ab1fa102a.0000000000000157:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:18:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.024818997000001, 2024-09-18T00:18:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034709218, 2024-09-18T00:18:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.390 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11464 2024-09-18T00:18:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:18:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:18:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272058+0000", 2024-09-18T00:18:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4070000000000002e-06 2024-09-18T00:18:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272074+0000", 2024-09-18T00:18:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5872999999999999e-05 2024-09-18T00:18:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272113+0000", 2024-09-18T00:18:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9224999999999998e-05 2024-09-18T00:18:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272135+0000", 2024-09-18T00:18:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2093e-05 2024-09-18T00:18:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272238+0000", 2024-09-18T00:18:48.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102939 2024-09-18T00:18:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272327+0000", 2024-09-18T00:18:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8746999999999996e-05 2024-09-18T00:18:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.273486+0000", 2024-09-18T00:18:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011588779999999999 2024-09-18T00:18:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306439+0000", 2024-09-18T00:18:48.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.032953142999999997 2024-09-18T00:18:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306491+0000", 2024-09-18T00:18:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2475000000000002e-05 2024-09-18T00:18:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306648+0000", 2024-09-18T00:18:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015725099999999999 2024-09-18T00:18:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306764+0000", 2024-09-18T00:18:48.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011518699999999999 2024-09-18T00:18:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:14346 3.2 3:44bbdafa:::rbd_data.112a13ec2d6c.0000000000000033:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-18T00:18:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:39.282847+0000", 2024-09-18T00:18:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.0140261539999997, 2024-09-18T00:18:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.016717600999999999, 2024-09-18T00:18:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:48.396 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 14346 2024-09-18T00:18:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282847+0000", 2024-09-18T00:18:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282847+0000", 2024-09-18T00:18:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:48.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282850+0000", 2024-09-18T00:18:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1159999999999999e-06 2024-09-18T00:18:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282867+0000", 2024-09-18T00:18:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6310000000000001e-05 2024-09-18T00:18:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282904+0000", 2024-09-18T00:18:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6952000000000001e-05 2024-09-18T00:18:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.282921+0000", 2024-09-18T00:18:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7765999999999999e-05 2024-09-18T00:18:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.283061+0000", 2024-09-18T00:18:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013918599999999999 2024-09-18T00:18:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.283385+0000", 2024-09-18T00:18:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00032432700000000001 2024-09-18T00:18:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.284421+0000", 2024-09-18T00:18:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0010358889999999999 2024-09-18T00:18:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.299281+0000", 2024-09-18T00:18:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014860329 2024-09-18T00:18:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.299336+0000", 2024-09-18T00:18:48.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4926000000000002e-05 2024-09-18T00:18:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.299458+0000", 2024-09-18T00:18:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000122111 2024-09-18T00:18:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:39.299565+0000", 2024-09-18T00:18:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010668899999999999 2024-09-18T00:18:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.401 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:48.402 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:48.402 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:48.403 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:18:48.403 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:48.398+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:48.440 INFO:teuthology.orchestra.run.smithi097.stdout:2800 read 0x7c6d4 thru 0x84a04 (0x8331 bytes) 2024-09-18T00:18:48.442 INFO:teuthology.orchestra.run.smithi097.stdout:2801 read 0x3e7e2e6 thru 0x3e81e7d (0x3b98 bytes) 2024-09-18T00:18:48.443 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:48.472 INFO:teuthology.orchestra.run.smithi097.stdout:2805 write 0xafe9437 thru 0xaff5b5a (0xc724 bytes) 2024-09-18T00:18:48.490 INFO:teuthology.orchestra.run.smithi097.stdout:2808 punch from 0x29e3c0b to 0x29e8829, (0x4c1e bytes) 2024-09-18T00:18:48.490 INFO:teuthology.orchestra.run.smithi097.stdout:2809 read 0x1c48a2f thru 0x1c53591 (0xab63 bytes) 2024-09-18T00:18:48.504 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:48.498+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:48.505 INFO:teuthology.orchestra.run.smithi097.stdout:2810 write 0xc4d03a5 thru 0xc4d6fdd (0x6c39 bytes) 2024-09-18T00:18:48.539 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:48.539 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:48.539 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:18:48.539 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:48.544 INFO:teuthology.orchestra.run.smithi097.stdout:2814 write 0xcc51e98 thru 0xcc59f28 (0x8091 bytes) 2024-09-18T00:18:48.555 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:18:48.568 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --force --op remove --pgid 2.4 2024-09-18T00:18:48.574 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:48.574 INFO:teuthology.orchestra.run.smithi097.stdout:2818 trunc from 0xcc59f29 to 0xc92faee 2024-09-18T00:18:48.610 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:48.611 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:48.611 INFO:teuthology.orchestra.run.smithi097.stdout:2821 read 0x4f70310 thru 0x4f7d257 (0xcf48 bytes) 2024-09-18T00:18:48.611 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:48.606+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:48.611 INFO:teuthology.orchestra.run.smithi097.stdout:2822 read 0x48dd74b thru 0x48ea343 (0xcbf9 bytes) 2024-09-18T00:18:48.612 INFO:teuthology.orchestra.run.smithi097.stdout:2823 writesame 0xadd9b57 thru 0xade4fd9 data_size 0x1(0xb483 bytes) 2024-09-18T00:18:48.702 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:18:48.718 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:48.719 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:48.732 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:18:48.735 INFO:teuthology.orchestra.run.smithi097.stdout:2824 trunc from 0xc92faee to 0xebcf71 2024-09-18T00:18:48.736 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:48.730+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:48.790 INFO:teuthology.orchestra.run.smithi097.stdout:2826 write 0x5f51c01 thru 0x5f5adb8 (0x91b8 bytes) 2024-09-18T00:18:48.792 INFO:teuthology.orchestra.run.smithi097.stdout:2827 read 0x55d7819 thru 0x55e1636 (0x9e1e bytes) 2024-09-18T00:18:48.802 INFO:teuthology.orchestra.run.smithi097.stdout:2829 write 0x7fd6307 thru 0x7fe2045 (0xbd3f bytes) 2024-09-18T00:18:48.807 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:48.802+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:48.811 INFO:teuthology.orchestra.run.smithi097.stdout:2830 trunc from 0x7fe2046 to 0x356ca0d 2024-09-18T00:18:48.821 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:48.821 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:48.822 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:48.834 INFO:teuthology.orchestra.run.smithi097.stdout:2838 writesame 0x563e15c thru 0x5645df5 data_size 0x2(0x7c9a bytes) 2024-09-18T00:18:48.855 INFO:teuthology.orchestra.run.smithi097.stdout:2839 write 0xc06e6f0 thru 0xc071aaf (0x33c0 bytes) 2024-09-18T00:18:48.857 INFO:teuthology.orchestra.run.smithi097.stdout:2840 writesame 0x2c06319 thru 0x2c06ffc data_size 0x2c(0xce4 bytes) 2024-09-18T00:18:48.868 INFO:teuthology.orchestra.run.smithi097.stdout:2841 write 0xcd4ed79 thru 0xcd53e9a (0x5122 bytes) 2024-09-18T00:18:48.871 INFO:teuthology.orchestra.run.smithi097.stdout:2842 punch from 0xabb7a to 0xb12d8, (0x575e bytes) 2024-09-18T00:18:48.871 INFO:teuthology.orchestra.run.smithi097.stdout:2843 writesame 0x403b079 thru 0x403cec9 data_size 0xd(0x1e51 bytes) 2024-09-18T00:18:48.873 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:48.873 INFO:teuthology.orchestra.run.smithi097.stdout:2845 read 0x679aa1b thru 0x679d464 (0x2a4a bytes) 2024-09-18T00:18:48.873 INFO:teuthology.orchestra.run.smithi097.stdout:2846 write 0x7b87d8a thru 0x7b8e964 (0x6bdb bytes) 2024-09-18T00:18:48.875 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2418 3.1a 3:59a8b40d:::rbd_data.112aa7ba4578.00000000000001b0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:18:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.13557234, 2024-09-18T00:18:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10488562999999999, 2024-09-18T00:18:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2418 2024-09-18T00:18:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:18:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:18:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735592+0000", 2024-09-18T00:18:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5229999999999998e-06 2024-09-18T00:18:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735604+0000", 2024-09-18T00:18:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2143e-05 2024-09-18T00:18:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735629+0000", 2024-09-18T00:18:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.497e-05 2024-09-18T00:18:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735648+0000", 2024-09-18T00:18:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8896999999999999e-05 2024-09-18T00:18:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735775+0000", 2024-09-18T00:18:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126935 2024-09-18T00:18:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.736118+0000", 2024-09-18T00:18:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034330800000000001 2024-09-18T00:18:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-18T00:18:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.741633+0000", 2024-09-18T00:18:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055144139999999996 2024-09-18T00:18:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840134+0000", 2024-09-18T00:18:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.098501618999999999 2024-09-18T00:18:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840184+0000", 2024-09-18T00:18:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9953000000000001e-05 2024-09-18T00:18:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840336+0000", 2024-09-18T00:18:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151476 2024-09-18T00:18:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840475+0000", 2024-09-18T00:18:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013939200000000001 2024-09-18T00:18:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2418 3.1a e37/23 ondisk, result = 0)", 2024-09-18T00:18:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:18:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.116982496999999, 2024-09-18T00:18:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.085979978999999998, 2024-09-18T00:18:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:18:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:18:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754181+0000", 2024-09-18T00:18:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8530000000000001e-06 2024-09-18T00:18:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754194+0000", 2024-09-18T00:18:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.292e-05 2024-09-18T00:18:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754230+0000", 2024-09-18T00:18:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5602999999999998e-05 2024-09-18T00:18:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754249+0000", 2024-09-18T00:18:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8850000000000001e-05 2024-09-18T00:18:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840070+0000", 2024-09-18T00:18:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.085821123999999999 2024-09-18T00:18:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840098+0000", 2024-09-18T00:18:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8546000000000001e-05 2024-09-18T00:18:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840159+0000", 2024-09-18T00:18:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1082999999999998e-05 2024-09-18T00:18:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12538 3.3 3:c6f35e4c:::rbd_data.112ab1fa102a.0000000000000000:9 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.394809+0000", 2024-09-18T00:18:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.476352504999999, 2024-09-18T00:18:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043048292000000002, 2024-09-18T00:18:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12538 2024-09-18T00:18:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394809+0000", 2024-09-18T00:18:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394809+0000", 2024-09-18T00:18:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394811+0000", 2024-09-18T00:18:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4500000000000001e-06 2024-09-18T00:18:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394817+0000", 2024-09-18T00:18:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0530000000000003e-06 2024-09-18T00:18:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394838+0000", 2024-09-18T00:18:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0823e-05 2024-09-18T00:18:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394852+0000", 2024-09-18T00:18:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4129e-05 2024-09-18T00:18:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.437681+0000", 2024-09-18T00:18:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042829205000000002 2024-09-18T00:18:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.437776+0000", 2024-09-18T00:18:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5315000000000005e-05 2024-09-18T00:18:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.437858+0000", 2024-09-18T00:18:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1316999999999994e-05 2024-09-18T00:18:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12539 3.3 3:c6f35e4c:::rbd_data.112ab1fa102a.0000000000000000:9 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.394870+0000", 2024-09-18T00:18:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.476292219999999, 2024-09-18T00:18:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043365935000000001, 2024-09-18T00:18:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12539 2024-09-18T00:18:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394870+0000", 2024-09-18T00:18:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394870+0000", 2024-09-18T00:18:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394871+0000", 2024-09-18T00:18:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2729999999999999e-06 2024-09-18T00:18:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394876+0000", 2024-09-18T00:18:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-18T00:18:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394894+0000", 2024-09-18T00:18:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7915999999999999e-05 2024-09-18T00:18:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394924+0000", 2024-09-18T00:18:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0001999999999998e-05 2024-09-18T00:18:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.437932+0000", 2024-09-18T00:18:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043008192000000001 2024-09-18T00:18:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.438071+0000", 2024-09-18T00:18:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013871099999999999 2024-09-18T00:18:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.438236+0000", 2024-09-18T00:18:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000164351 2024-09-18T00:18:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12540 3.3 3:c6f35e4c:::rbd_data.112ab1fa102a.0000000000000000:9 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.394944+0000", 2024-09-18T00:18:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.47621827, 2024-09-18T00:18:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043684374999999998, 2024-09-18T00:18:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12540 2024-09-18T00:18:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394944+0000", 2024-09-18T00:18:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394944+0000", 2024-09-18T00:18:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394945+0000", 2024-09-18T00:18:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2640000000000001e-06 2024-09-18T00:18:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394951+0000", 2024-09-18T00:18:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9830000000000001e-06 2024-09-18T00:18:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394969+0000", 2024-09-18T00:18:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8266000000000001e-05 2024-09-18T00:18:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.394984+0000", 2024-09-18T00:18:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.466e-05 2024-09-18T00:18:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.438321+0000", 2024-09-18T00:18:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043337734000000003 2024-09-18T00:18:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.438481+0000", 2024-09-18T00:18:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015959500000000001 2024-09-18T00:18:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.438628+0000", 2024-09-18T00:18:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000146873 2024-09-18T00:18:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12541 3.3 3:c6f35e4c:::rbd_data.112ab1fa102a.0000000000000000:9 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395001+0000", 2024-09-18T00:18:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.476161060999999, 2024-09-18T00:18:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043946967000000003, 2024-09-18T00:18:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12541 2024-09-18T00:18:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395001+0000", 2024-09-18T00:18:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395001+0000", 2024-09-18T00:18:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395002+0000", 2024-09-18T00:18:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.096e-06 2024-09-18T00:18:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395007+0000", 2024-09-18T00:18:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0900000000000004e-06 2024-09-18T00:18:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395024+0000", 2024-09-18T00:18:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6736999999999999e-05 2024-09-18T00:18:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395037+0000", 2024-09-18T00:18:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3589e-05 2024-09-18T00:18:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.438689+0000", 2024-09-18T00:18:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043651769 2024-09-18T00:18:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.438776+0000", 2024-09-18T00:18:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6680000000000004e-05 2024-09-18T00:18:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.438948+0000", 2024-09-18T00:18:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017200599999999999 2024-09-18T00:18:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12542 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395055+0000", 2024-09-18T00:18:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.476106855999999, 2024-09-18T00:18:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044315872999999999, 2024-09-18T00:18:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12542 2024-09-18T00:18:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395055+0000", 2024-09-18T00:18:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395055+0000", 2024-09-18T00:18:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395056+0000", 2024-09-18T00:18:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0699999999999999e-06 2024-09-18T00:18:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395061+0000", 2024-09-18T00:18:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2900000000000002e-06 2024-09-18T00:18:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395088+0000", 2024-09-18T00:18:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6939000000000001e-05 2024-09-18T00:18:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395104+0000", 2024-09-18T00:18:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6066000000000002e-05 2024-09-18T00:18:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.439018+0000", 2024-09-18T00:18:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043913212 2024-09-18T00:18:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.439147+0000", 2024-09-18T00:18:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012977099999999999 2024-09-18T00:18:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.439371+0000", 2024-09-18T00:18:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000223525 2024-09-18T00:18:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12543 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395123+0000", 2024-09-18T00:18:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.476038481, 2024-09-18T00:18:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044696828000000001, 2024-09-18T00:18:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12543 2024-09-18T00:18:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395123+0000", 2024-09-18T00:18:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395123+0000", 2024-09-18T00:18:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395125+0000", 2024-09-18T00:18:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.203e-06 2024-09-18T00:18:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395130+0000", 2024-09-18T00:18:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3589999999999999e-06 2024-09-18T00:18:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395147+0000", 2024-09-18T00:18:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7107000000000001e-05 2024-09-18T00:18:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395160+0000", 2024-09-18T00:18:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3346000000000001e-05 2024-09-18T00:18:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.439454+0000", 2024-09-18T00:18:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044293375000000003 2024-09-18T00:18:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.439611+0000", 2024-09-18T00:18:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000157477 2024-09-18T00:18:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.439820+0000", 2024-09-18T00:18:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020896099999999999 2024-09-18T00:18:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12544 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395178+0000", 2024-09-18T00:18:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.475983393, 2024-09-18T00:18:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045040958999999998, 2024-09-18T00:18:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12544 2024-09-18T00:18:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395178+0000", 2024-09-18T00:18:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395178+0000", 2024-09-18T00:18:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395180+0000", 2024-09-18T00:18:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.28e-06 2024-09-18T00:18:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395186+0000", 2024-09-18T00:18:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3829999999999997e-06 2024-09-18T00:18:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395207+0000", 2024-09-18T00:18:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0696e-05 2024-09-18T00:18:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395221+0000", 2024-09-18T00:18:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3876000000000001e-05 2024-09-18T00:18:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.439891+0000", 2024-09-18T00:18:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044670407000000002 2024-09-18T00:18:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440037+0000", 2024-09-18T00:18:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014583700000000001 2024-09-18T00:18:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440219+0000", 2024-09-18T00:18:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018248 2024-09-18T00:18:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12545 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:18:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.475919735, 2024-09-18T00:18:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045427642999999997, 2024-09-18T00:18:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12545 2024-09-18T00:18:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:18:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:18:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395243+0000", 2024-09-18T00:18:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.206e-06 2024-09-18T00:18:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395249+0000", 2024-09-18T00:18:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:18:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395270+0000", 2024-09-18T00:18:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1223e-05 2024-09-18T00:18:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395285+0000", 2024-09-18T00:18:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4725999999999999e-05 2024-09-18T00:18:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440296+0000", 2024-09-18T00:18:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045011402999999998 2024-09-18T00:18:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440453+0000", 2024-09-18T00:18:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015742299999999999 2024-09-18T00:18:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440670+0000", 2024-09-18T00:18:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000216302 2024-09-18T00:18:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12546 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:18:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.475857527000001, 2024-09-18T00:18:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045915171999999997, 2024-09-18T00:18:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12546 2024-09-18T00:18:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:18:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:18:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395306+0000", 2024-09-18T00:18:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.206e-06 2024-09-18T00:18:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395330+0000", 2024-09-18T00:18:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4686e-05 2024-09-18T00:18:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395356+0000", 2024-09-18T00:18:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5418999999999998e-05 2024-09-18T00:18:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395387+0000", 2024-09-18T00:18:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0942000000000003e-05 2024-09-18T00:18:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440743+0000", 2024-09-18T00:18:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045356592000000001 2024-09-18T00:18:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440880+0000", 2024-09-18T00:18:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013641700000000001 2024-09-18T00:18:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441219+0000", 2024-09-18T00:18:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033991000000000002 2024-09-18T00:18:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12547 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 212992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:18:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.475748193999999, 2024-09-18T00:18:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046163691999999999, 2024-09-18T00:18:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12547 2024-09-18T00:18:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:18:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:18:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395415+0000", 2024-09-18T00:18:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.553e-06 2024-09-18T00:18:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395426+0000", 2024-09-18T00:18:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0682999999999999e-05 2024-09-18T00:18:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395449+0000", 2024-09-18T00:18:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3082999999999999e-05 2024-09-18T00:18:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395469+0000", 2024-09-18T00:18:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0173000000000002e-05 2024-09-18T00:18:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441289+0000", 2024-09-18T00:18:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045820195000000001 2024-09-18T00:18:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441426+0000", 2024-09-18T00:18:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013712699999999999 2024-09-18T00:18:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441577+0000", 2024-09-18T00:18:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015087799999999999 2024-09-18T00:18:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12548 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:18:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.475674433, 2024-09-18T00:18:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046293932000000003, 2024-09-18T00:18:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12548 2024-09-18T00:18:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:18:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:18:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395489+0000", 2024-09-18T00:18:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.22e-06 2024-09-18T00:18:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395512+0000", 2024-09-18T00:18:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3312e-05 2024-09-18T00:18:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395621+0000", 2024-09-18T00:18:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010859 2024-09-18T00:18:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395689+0000", 2024-09-18T00:18:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8232000000000007e-05 2024-09-18T00:18:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441610+0000", 2024-09-18T00:18:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045920751000000003 2024-09-18T00:18:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441695+0000", 2024-09-18T00:18:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5379999999999999e-05 2024-09-18T00:18:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441781+0000", 2024-09-18T00:18:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6446999999999994e-05 2024-09-18T00:18:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12549 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 294912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:18:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.475413781, 2024-09-18T00:18:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046410264999999999, 2024-09-18T00:18:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12549 2024-09-18T00:18:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:18:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:18:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395750+0000", 2024-09-18T00:18:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:18:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395758+0000", 2024-09-18T00:18:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5259999999999993e-06 2024-09-18T00:18:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395793+0000", 2024-09-18T00:18:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4459000000000003e-05 2024-09-18T00:18:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395809+0000", 2024-09-18T00:18:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6433e-05 2024-09-18T00:18:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441851+0000", 2024-09-18T00:18:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046041493000000003 2024-09-18T00:18:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442002+0000", 2024-09-18T00:18:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151433 2024-09-18T00:18:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442158+0000", 2024-09-18T00:18:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000156358 2024-09-18T00:18:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12550 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:18:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.475302292, 2024-09-18T00:18:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046498398000000003, 2024-09-18T00:18:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12550 2024-09-18T00:18:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:18:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:18:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395861+0000", 2024-09-18T00:18:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.234e-06 2024-09-18T00:18:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395872+0000", 2024-09-18T00:18:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1409e-05 2024-09-18T00:18:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395892+0000", 2024-09-18T00:18:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9680000000000001e-05 2024-09-18T00:18:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395911+0000", 2024-09-18T00:18:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9283000000000002e-05 2024-09-18T00:18:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442210+0000", 2024-09-18T00:18:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046298372999999997 2024-09-18T00:18:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442279+0000", 2024-09-18T00:18:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9055999999999997e-05 2024-09-18T00:18:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442358+0000", 2024-09-18T00:18:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9363000000000006e-05 2024-09-18T00:18:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12551 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:18:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.47462984, 2024-09-18T00:18:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046066902999999999, 2024-09-18T00:18:48.945 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.945 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.945 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.945 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.945 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.945 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.945 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12551 2024-09-18T00:18:48.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.945 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:18:48.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396534+0000", 2024-09-18T00:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6869999999999999e-06 2024-09-18T00:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396543+0000", 2024-09-18T00:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3430000000000002e-06 2024-09-18T00:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396563+0000", 2024-09-18T00:18:48.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0063000000000001e-05 2024-09-18T00:18:48.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396579+0000", 2024-09-18T00:18:48.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5610000000000001e-05 2024-09-18T00:18:48.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442434+0000", 2024-09-18T00:18:48.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045855094999999998 2024-09-18T00:18:48.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442515+0000", 2024-09-18T00:18:48.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0988000000000005e-05 2024-09-18T00:18:48.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442599+0000", 2024-09-18T00:18:48.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4116999999999994e-05 2024-09-18T00:18:48.948 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.948 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.948 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.948 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12552 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.948 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:18:48.949 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.474555038, 2024-09-18T00:18:48.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046210804000000001, 2024-09-18T00:18:48.949 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.949 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.949 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.949 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.949 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.949 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.949 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12552 2024-09-18T00:18:48.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.949 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:18:48.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:18:48.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396608+0000", 2024-09-18T00:18:48.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4729999999999999e-06 2024-09-18T00:18:48.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396614+0000", 2024-09-18T00:18:48.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1e-06 2024-09-18T00:18:48.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396633+0000", 2024-09-18T00:18:48.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9009e-05 2024-09-18T00:18:48.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396650+0000", 2024-09-18T00:18:48.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7045999999999999e-05 2024-09-18T00:18:48.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442662+0000", 2024-09-18T00:18:48.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046011170999999997 2024-09-18T00:18:48.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.952 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.952 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442737+0000", 2024-09-18T00:18:48.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5759000000000004e-05 2024-09-18T00:18:48.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.952 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.952 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442818+0000", 2024-09-18T00:18:48.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0246000000000004e-05 2024-09-18T00:18:48.952 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.952 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.952 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12553 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:18:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.474492334000001, 2024-09-18T00:18:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046339454000000002, 2024-09-18T00:18:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12553 2024-09-18T00:18:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:18:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:18:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396671+0000", 2024-09-18T00:18:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.314e-06 2024-09-18T00:18:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396676+0000", 2024-09-18T00:18:48.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6659999999999996e-06 2024-09-18T00:18:48.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396695+0000", 2024-09-18T00:18:48.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8383e-05 2024-09-18T00:18:48.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396709+0000", 2024-09-18T00:18:48.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4309e-05 2024-09-18T00:18:48.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442871+0000", 2024-09-18T00:18:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046162295999999998 2024-09-18T00:18:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442939+0000", 2024-09-18T00:18:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7607999999999994e-05 2024-09-18T00:18:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443009+0000", 2024-09-18T00:18:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9877999999999997e-05 2024-09-18T00:18:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.956 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.957 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.957 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12554 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.957 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:18:48.957 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.474434412000001, 2024-09-18T00:18:48.957 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046504050999999998, 2024-09-18T00:18:48.957 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.957 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.957 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:48.957 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.957 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.957 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.957 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12554 2024-09-18T00:18:48.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.957 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:18:48.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:18:48.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396729+0000", 2024-09-18T00:18:48.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2559999999999999e-06 2024-09-18T00:18:48.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396736+0000", 2024-09-18T00:18:48.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6140000000000003e-06 2024-09-18T00:18:48.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396755+0000", 2024-09-18T00:18:48.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8221999999999999e-05 2024-09-18T00:18:48.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396769+0000", 2024-09-18T00:18:48.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4372999999999999e-05 2024-09-18T00:18:48.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.960 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443078+0000", 2024-09-18T00:18:48.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046308850999999998 2024-09-18T00:18:48.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.960 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443152+0000", 2024-09-18T00:18:48.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4083000000000002e-05 2024-09-18T00:18:48.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.960 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443231+0000", 2024-09-18T00:18:48.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9652000000000002e-05 2024-09-18T00:18:48.961 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.961 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.961 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.961 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.961 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12632 3.3 3:c21fac02:::rbd_data.112a7b1589a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1490944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:48.961 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:18:48.961 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.467876127, 2024-09-18T00:18:48.961 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045843916999999998, 2024-09-18T00:18:48.961 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:48.961 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:48.961 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12632 2024-09-18T00:18:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:18:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:18:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:48.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:48.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403288+0000", 2024-09-18T00:18:48.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3630000000000001e-06 2024-09-18T00:18:48.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:48.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403886+0000", 2024-09-18T00:18:48.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00059840100000000003 2024-09-18T00:18:48.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:48.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403913+0000", 2024-09-18T00:18:48.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6376e-05 2024-09-18T00:18:48.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:48.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403927+0000", 2024-09-18T00:18:48.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4188e-05 2024-09-18T00:18:48.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:48.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443268+0000", 2024-09-18T00:18:48.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.039340591000000001 2024-09-18T00:18:48.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:48.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443488+0000", 2024-09-18T00:18:48.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000220545 2024-09-18T00:18:48.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.965 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.965 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:48.965 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443918+0000", 2024-09-18T00:18:48.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043021800000000001 2024-09-18T00:18:48.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.965 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.965 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:48.965 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.448558+0000", 2024-09-18T00:18:48.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046391449999999999 2024-09-18T00:18:48.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.965 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.965 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:48.965 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449015+0000", 2024-09-18T00:18:48.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045766600000000002 2024-09-18T00:18:48.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:48.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449085+0000", 2024-09-18T00:18:48.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0191999999999995e-05 2024-09-18T00:18:48.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:48.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:48.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:48.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449130+0000", 2024-09-18T00:18:48.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4231999999999999e-05 2024-09-18T00:18:48.966 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.966 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.967 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.967 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:48.967 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:48.967 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:48.967 INFO:teuthology.orchestra.run.smithi097.stdout:2848 writesame 0xcc12dc2 thru 0xcc201de data_size 0x1(0xd41d bytes) 2024-09-18T00:18:48.968 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:18:48.969 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:48.962+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:49.006 INFO:teuthology.orchestra.run.smithi097.stdout:2849 trunc from 0xcd53e9b to 0xca6ec83 2024-09-18T00:18:49.038 INFO:teuthology.orchestra.run.smithi097.stdout:2850 trunc from 0xca6ec83 to 0x392838e 2024-09-18T00:18:49.057 INFO:teuthology.orchestra.run.smithi097.stdout:2851 trunc from 0x392838e to 0x6393b0f 2024-09-18T00:18:49.065 INFO:teuthology.orchestra.run.smithi097.stdout:2852 punch from 0x1620190 to 0x162ac90, (0xab00 bytes) 2024-09-18T00:18:49.069 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:49.062+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:49.093 INFO:teuthology.orchestra.run.smithi097.stdout:2853 writesame 0xdc82fa6 thru 0xdc87265 data_size 0xc(0x42c0 bytes) 2024-09-18T00:18:49.095 INFO:teuthology.orchestra.run.smithi097.stdout:2854 write 0xf79531 thru 0xf7f395 (0x5e65 bytes) 2024-09-18T00:18:49.099 INFO:teuthology.orchestra.run.smithi097.stdout:2857 write 0xc6cd495 thru 0xc6dae71 (0xd9dd bytes) 2024-09-18T00:18:49.108 INFO:teuthology.orchestra.run.smithi097.stdout:2859 writesame 0xa5b796 thru 0xa62475 data_size 0x68(0x6ce0 bytes) 2024-09-18T00:18:49.112 INFO:teuthology.orchestra.run.smithi097.stdout:2860 punch from 0xc3475ab to 0xc3521b8, (0xac0d bytes) 2024-09-18T00:18:49.124 INFO:teuthology.orchestra.run.smithi097.stdout:2861 write 0xdefecf0 thru 0xdefef85 (0x296 bytes) 2024-09-18T00:18:49.127 INFO:teuthology.orchestra.run.smithi097.stdout:2862 read 0x8ab0f5b thru 0x8abe39f (0xd445 bytes) 2024-09-18T00:18:49.128 INFO:teuthology.orchestra.run.smithi097.stdout:2863 punch from 0xa01c0a9 to 0xa01d22c, (0x1183 bytes) 2024-09-18T00:18:49.128 INFO:teuthology.orchestra.run.smithi097.stdout:2864 read 0x7f8c1d0 thru 0x7f948b1 (0x86e2 bytes) 2024-09-18T00:18:49.129 INFO:teuthology.orchestra.run.smithi097.stdout:2865 read 0x8473bc2 thru 0x847cb71 (0x8fb0 bytes) 2024-09-18T00:18:49.140 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:49.140 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:49.140 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:18:49.140 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:49.141 INFO:teuthology.orchestra.run.smithi097.stdout:2866 trunc from 0xdefef86 to 0xa361f62 2024-09-18T00:18:49.157 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:18:49.158 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:49.158 INFO:teuthology.orchestra.run.smithi097.stdout:2868 write 0x21522f3 thru 0x21536c8 (0x13d6 bytes) 2024-09-18T00:18:49.170 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:49.166+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:49.183 INFO:teuthology.orchestra.run.smithi097.stdout:2869 writesame 0xda52a51 thru 0xda5cf37 data_size 0x1(0xa4e7 bytes) 2024-09-18T00:18:49.185 INFO:teuthology.orchestra.run.smithi097.stdout:2870 read 0x42e2661 thru 0x42eaee1 (0x8881 bytes) 2024-09-18T00:18:49.186 INFO:teuthology.orchestra.run.smithi097.stdout:2872 punch from 0x20a9e9b to 0x20b065d, (0x67c2 bytes) 2024-09-18T00:18:49.186 INFO:teuthology.orchestra.run.smithi097.stdout:2873 read 0x7eea263 thru 0x7ef9cec (0xfa8a bytes) 2024-09-18T00:18:49.187 INFO:teuthology.orchestra.run.smithi097.stdout:2874 read 0x5adcc90 thru 0x5ae334b (0x66bc bytes) 2024-09-18T00:18:49.187 INFO:teuthology.orchestra.run.smithi097.stdout:2875 read 0xc1f0d6a thru 0xc1f20aa (0x1341 bytes) 2024-09-18T00:18:49.188 INFO:teuthology.orchestra.run.smithi097.stdout:2876 read 0x565df88 thru 0x566c8f6 (0xe96f bytes) 2024-09-18T00:18:49.189 INFO:teuthology.orchestra.run.smithi097.stdout:2877 write 0x122d525 thru 0x123ad35 (0xd811 bytes) 2024-09-18T00:18:49.198 INFO:teuthology.orchestra.run.smithi097.stdout:2878 trunc from 0xda5cf38 to 0xcb1fb35 2024-09-18T00:18:49.215 INFO:teuthology.orchestra.run.smithi097.stdout:2879 read 0x7770df0 thru 0x777688a (0x5a9b bytes) 2024-09-18T00:18:49.216 INFO:teuthology.orchestra.run.smithi097.stdout:2880 write 0x7722d8d thru 0x772fd12 (0xcf86 bytes) 2024-09-18T00:18:49.235 INFO:teuthology.orchestra.run.smithi097.stdout:2881 trunc from 0xcb1fb35 to 0x927fd77 2024-09-18T00:18:49.247 INFO:teuthology.orchestra.run.smithi097.stdout:2882 write 0x281321 thru 0x2876dc (0x63bc bytes) 2024-09-18T00:18:49.251 INFO:teuthology.orchestra.run.smithi097.stdout:2883 trunc from 0x927fd77 to 0x82b806d 2024-09-18T00:18:49.261 INFO:teuthology.orchestra.run.smithi097.stdout:2885 write 0xc9abd4 thru 0xca1d27 (0x7154 bytes) 2024-09-18T00:18:49.269 INFO:teuthology.orchestra.run.smithi097.stdout:2886 trunc from 0x82b806d to 0xb1f6f7b 2024-09-18T00:18:49.271 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:49.266+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:49.292 INFO:teuthology.orchestra.run.smithi097.stdout:2889 write 0xbc2d1ec thru 0xbc3357e (0x6393 bytes) 2024-09-18T00:18:49.306 INFO:teuthology.orchestra.run.smithi097.stdout:2890 writesame 0xc2f266f thru 0xc2ff10e data_size 0x10(0xcaa0 bytes) 2024-09-18T00:18:49.308 INFO:teuthology.orchestra.run.smithi097.stdout:2893 read 0xba78e3 thru 0xbb75d3 (0xfcf1 bytes) 2024-09-18T00:18:49.325 INFO:teuthology.orchestra.run.smithi097.stdout:2894 trunc from 0xc2ff10f to 0x11419ad 2024-09-18T00:18:49.336 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:49.336 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:49.336 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:49.336 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:49.336 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:49.338 INFO:teuthology.orchestra.run.smithi097.stdout:2895 read 0xcb2fc3 thru 0xcbf7e2 (0xc820 bytes) 2024-09-18T00:18:49.339 INFO:teuthology.orchestra.run.smithi097.stdout:2898 read 0xad53a2 thru 0xae0ef2 (0xbb51 bytes) 2024-09-18T00:18:49.341 INFO:teuthology.orchestra.run.smithi097.stdout:2899 write 0x594e2e thru 0x59fadc (0xacaf bytes) 2024-09-18T00:18:49.352 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:18:49.368 INFO:teuthology.orchestra.run.smithi097.stdout:2901 writesame 0x673ffe9 thru 0x674c0e0 data_size 0x1a(0xc0f8 bytes) 2024-09-18T00:18:49.370 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:49.371 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:49.366+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:49.392 INFO:teuthology.orchestra.run.smithi097.stdout:2904 writesame 0xd7f9835 thru 0xd80127b data_size 0x1(0x7a47 bytes) 2024-09-18T00:18:49.394 INFO:teuthology.orchestra.run.smithi097.stdout:2905 read 0x39b84a3 thru 0x39be01c (0x5b7a bytes) 2024-09-18T00:18:49.395 INFO:teuthology.orchestra.run.smithi097.stdout:2906 write 0x2b2b7ca thru 0x2b34d42 (0x9579 bytes) 2024-09-18T00:18:49.401 INFO:teuthology.orchestra.run.smithi097.stdout:2907 punch from 0xccaf653 to 0xccb5569, (0x5f16 bytes) 2024-09-18T00:18:49.401 INFO:teuthology.orchestra.run.smithi097.stdout:2908 writesame 0x8002b56 thru 0x80094f6 data_size 0x1(0x69a1 bytes) 2024-09-18T00:18:49.437 INFO:teuthology.orchestra.run.smithi097.stdout:2909 punch from 0xd6d917d to 0xd6e760f, (0xe492 bytes) 2024-09-18T00:18:49.437 INFO:teuthology.orchestra.run.smithi097.stdout:2910 read 0x9c7ab7b thru 0x9c83cb9 (0x913f bytes) 2024-09-18T00:18:49.437 INFO:teuthology.orchestra.run.smithi097.stdout:2911 read 0xb1996c1 thru 0xb199885 (0x1c5 bytes) 2024-09-18T00:18:49.438 INFO:teuthology.orchestra.run.smithi097.stdout:2912 punch from 0x5aa4e29 to 0x5ab2941, (0xdb18 bytes) 2024-09-18T00:18:49.438 INFO:teuthology.orchestra.run.smithi097.stdout:2913 write 0x441b08 thru 0x44c753 (0xac4c bytes) 2024-09-18T00:18:49.446 INFO:teuthology.orchestra.run.smithi097.stdout:2914 read 0x40cb37b thru 0x40cf652 (0x42d8 bytes) 2024-09-18T00:18:49.447 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:49.447 INFO:teuthology.orchestra.run.smithi097.stdout:2917 write 0x600190b thru 0x600f50d (0xdc03 bytes) 2024-09-18T00:18:49.452 INFO:teuthology.orchestra.run.smithi097.stdout:2918 read 0xacbf631 thru 0xaccb42e (0xbdfe bytes) 2024-09-18T00:18:49.453 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:49.453 INFO:teuthology.orchestra.run.smithi097.stdout:2920 writesame 0x17dc12f thru 0x17e119e data_size 0x27(0x5070 bytes) 2024-09-18T00:18:49.455 INFO:teuthology.orchestra.run.smithi097.stdout:2922 punch from 0x334de23 to 0x3354446, (0x6623 bytes) 2024-09-18T00:18:49.455 INFO:teuthology.orchestra.run.smithi097.stdout:2924 read 0x1361c85 thru 0x13713f8 (0xf774 bytes) 2024-09-18T00:18:49.456 INFO:teuthology.orchestra.run.smithi097.stdout:2925 read 0x9599a14 thru 0x9599ac3 (0xb0 bytes) 2024-09-18T00:18:49.456 INFO:teuthology.orchestra.run.smithi097.stdout:2926 write 0xb14ef01 thru 0xb151c91 (0x2d91 bytes) 2024-09-18T00:18:49.459 INFO:teuthology.orchestra.run.smithi097.stdout:2927 read 0x304e541 thru 0x3058545 (0xa005 bytes) 2024-09-18T00:18:49.460 INFO:teuthology.orchestra.run.smithi097.stdout:2928 writesame 0x9a3399d thru 0x9a3b8ee data_size 0x2(0x7f52 bytes) 2024-09-18T00:18:49.472 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:49.466+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:49.483 INFO:teuthology.orchestra.run.smithi097.stdout:2930 write 0xae33111 thru 0xae352f8 (0x21e8 bytes) 2024-09-18T00:18:49.486 INFO:teuthology.orchestra.run.smithi097.stdout:2931 read 0xa00b17f thru 0xa00c49c (0x131e bytes) 2024-09-18T00:18:49.504 INFO:teuthology.orchestra.run.smithi097.stdout:2932 writesame 0xee5508f thru 0xee5eae0 data_size 0x1(0x9a52 bytes) 2024-09-18T00:18:49.507 INFO:teuthology.orchestra.run.smithi097.stdout:2933 write 0x46ac813 thru 0x46bc103 (0xf8f1 bytes) 2024-09-18T00:18:49.517 INFO:teuthology.orchestra.run.smithi097.stdout:2934 read 0x3e48bd2 thru 0x3e4b92b (0x2d5a bytes) 2024-09-18T00:18:49.517 INFO:teuthology.orchestra.run.smithi097.stdout:2936 read 0xd93794 thru 0xd961ba (0x2a27 bytes) 2024-09-18T00:18:49.518 INFO:teuthology.orchestra.run.smithi097.stdout:2937 write 0x811b1e2 thru 0x8126e8c (0xbcab bytes) 2024-09-18T00:18:49.524 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2229 3.10 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:18:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:18:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 29.256835254999999, 2024-09-18T00:18:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048730086999999998, 2024-09-18T00:18:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2229 2024-09-18T00:18:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:18:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:18:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263362+0000", 2024-09-18T00:18:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3869999999999999e-06 2024-09-18T00:18:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263381+0000", 2024-09-18T00:18:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9153000000000001e-05 2024-09-18T00:18:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263418+0000", 2024-09-18T00:18:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6346000000000001e-05 2024-09-18T00:18:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263435+0000", 2024-09-18T00:18:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7309999999999999e-05 2024-09-18T00:18:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263589+0000", 2024-09-18T00:18:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015451199999999999 2024-09-18T00:18:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263902+0000", 2024-09-18T00:18:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031291799999999997 2024-09-18T00:18:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.265995+0000", 2024-09-18T00:18:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020931809999999999 2024-09-18T00:18:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311788+0000", 2024-09-18T00:18:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045792078 2024-09-18T00:18:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311903+0000", 2024-09-18T00:18:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011510699999999999 2024-09-18T00:18:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312009+0000", 2024-09-18T00:18:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010630599999999999 2024-09-18T00:18:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312089+0000", 2024-09-18T00:18:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9789000000000005e-05 2024-09-18T00:18:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2229 3.10 e36/23 ondisk, result = 0)", 2024-09-18T00:18:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:18:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 29.237689595999999, 2024-09-18T00:18:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029600955000000002, 2024-09-18T00:18:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:18:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:18:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282507+0000", 2024-09-18T00:18:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9160000000000001e-06 2024-09-18T00:18:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282522+0000", 2024-09-18T00:18:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5e-05 2024-09-18T00:18:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282558+0000", 2024-09-18T00:18:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5694000000000001e-05 2024-09-18T00:18:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282580+0000", 2024-09-18T00:18:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2323e-05 2024-09-18T00:18:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311873+0000", 2024-09-18T00:18:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029293013999999999 2024-09-18T00:18:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311892+0000", 2024-09-18T00:18:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8556e-05 2024-09-18T00:18:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312105+0000", 2024-09-18T00:18:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021345199999999999 2024-09-18T00:18:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5714 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:18:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.209987092999999, 2024-09-18T00:18:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027448727999999999, 2024-09-18T00:18:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5714 2024-09-18T00:18:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:18:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:18:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310211+0000", 2024-09-18T00:18:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5639999999999998e-06 2024-09-18T00:18:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312678+0000", 2024-09-18T00:18:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002466761 2024-09-18T00:18:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312746+0000", 2024-09-18T00:18:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8046000000000005e-05 2024-09-18T00:18:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312774+0000", 2024-09-18T00:18:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7813e-05 2024-09-18T00:18:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312927+0000", 2024-09-18T00:18:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015328499999999999 2024-09-18T00:18:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313214+0000", 2024-09-18T00:18:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028683099999999998 2024-09-18T00:18:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313708+0000", 2024-09-18T00:18:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049379100000000004 2024-09-18T00:18:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336895+0000", 2024-09-18T00:18:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023186801 2024-09-18T00:18:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337454+0000", 2024-09-18T00:18:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055979399999999996 2024-09-18T00:18:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337571+0000", 2024-09-18T00:18:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116821 2024-09-18T00:18:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337656+0000", 2024-09-18T00:18:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4221000000000006e-05 2024-09-18T00:18:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6801 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:18:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.483150644999998, 2024-09-18T00:18:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051953602000000002, 2024-09-18T00:18:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6801 2024-09-18T00:18:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:18:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:18:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037046+0000", 2024-09-18T00:18:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.19e-06 2024-09-18T00:18:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037062+0000", 2024-09-18T00:18:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586999999999998e-05 2024-09-18T00:18:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037096+0000", 2024-09-18T00:18:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4112000000000001e-05 2024-09-18T00:18:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037120+0000", 2024-09-18T00:18:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.355e-05 2024-09-18T00:18:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037246+0000", 2024-09-18T00:18:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012652200000000001 2024-09-18T00:18:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037385+0000", 2024-09-18T00:18:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013924599999999999 2024-09-18T00:18:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.040881+0000", 2024-09-18T00:18:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034954830000000002 2024-09-18T00:18:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088794+0000", 2024-09-18T00:18:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047912897000000003 2024-09-18T00:18:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088828+0000", 2024-09-18T00:18:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3880000000000001e-05 2024-09-18T00:18:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088916+0000", 2024-09-18T00:18:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.865e-05 2024-09-18T00:18:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088997+0000", 2024-09-18T00:18:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0484999999999999e-05 2024-09-18T00:18:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:6801 3.10 e42/23 ondisk, result = 0)", 2024-09-18T00:18:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:18:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.469343235, 2024-09-18T00:18:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037960394000000001, 2024-09-18T00:18:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:18:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:18:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050852+0000", 2024-09-18T00:18:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:18:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050860+0000", 2024-09-18T00:18:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7729999999999999e-06 2024-09-18T00:18:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050891+0000", 2024-09-18T00:18:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0617000000000002e-05 2024-09-18T00:18:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050910+0000", 2024-09-18T00:18:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.944e-05 2024-09-18T00:18:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088766+0000", 2024-09-18T00:18:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037855633999999999 2024-09-18T00:18:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088784+0000", 2024-09-18T00:18:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7767000000000001e-05 2024-09-18T00:18:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088811+0000", 2024-09-18T00:18:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7382999999999999e-05 2024-09-18T00:18:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12501 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 442368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391396+0000", 2024-09-18T00:18:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.128797666000001, 2024-09-18T00:18:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025996424000000001, 2024-09-18T00:18:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12501 2024-09-18T00:18:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391396+0000", 2024-09-18T00:18:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391396+0000", 2024-09-18T00:18:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391398+0000", 2024-09-18T00:18:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.353e-06 2024-09-18T00:18:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391419+0000", 2024-09-18T00:18:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1123000000000001e-05 2024-09-18T00:18:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391448+0000", 2024-09-18T00:18:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9638999999999999e-05 2024-09-18T00:18:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391463+0000", 2024-09-18T00:18:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5163e-05 2024-09-18T00:18:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417122+0000", 2024-09-18T00:18:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025658017000000002 2024-09-18T00:18:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417250+0000", 2024-09-18T00:18:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000128016 2024-09-18T00:18:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417393+0000", 2024-09-18T00:18:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000143113 2024-09-18T00:18:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12502 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391496+0000", 2024-09-18T00:18:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.128697402, 2024-09-18T00:18:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026233784, 2024-09-18T00:18:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12502 2024-09-18T00:18:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391496+0000", 2024-09-18T00:18:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391496+0000", 2024-09-18T00:18:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391498+0000", 2024-09-18T00:18:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3260000000000001e-06 2024-09-18T00:18:49.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391504+0000", 2024-09-18T00:18:49.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.806e-06 2024-09-18T00:18:49.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391531+0000", 2024-09-18T00:18:49.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7555999999999998e-05 2024-09-18T00:18:49.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391568+0000", 2024-09-18T00:18:49.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6495999999999998e-05 2024-09-18T00:18:49.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417454+0000", 2024-09-18T00:18:49.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025885899 2024-09-18T00:18:49.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417596+0000", 2024-09-18T00:18:49.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000141948 2024-09-18T00:18:49.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417730+0000", 2024-09-18T00:18:49.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013475299999999999 2024-09-18T00:18:49.558 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.558 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.558 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.558 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12594 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:49.558 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:18:49.558 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.128566697, 2024-09-18T00:18:49.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026492600000000002, 2024-09-18T00:18:49.558 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.558 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.558 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.558 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.558 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.558 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.559 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12594 2024-09-18T00:18:49.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.559 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:18:49.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:18:49.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391629+0000", 2024-09-18T00:18:49.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.86e-06 2024-09-18T00:18:49.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391637+0000", 2024-09-18T00:18:49.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3999999999999992e-06 2024-09-18T00:18:49.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391664+0000", 2024-09-18T00:18:49.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6536000000000001e-05 2024-09-18T00:18:49.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391681+0000", 2024-09-18T00:18:49.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6563e-05 2024-09-18T00:18:49.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417791+0000", 2024-09-18T00:18:49.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026110936000000001 2024-09-18T00:18:49.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417933+0000", 2024-09-18T00:18:49.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014112099999999999 2024-09-18T00:18:49.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418120+0000", 2024-09-18T00:18:49.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018718399999999999 2024-09-18T00:18:49.562 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.562 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.562 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.562 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12595 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:49.562 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:18:49.562 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.128490499, 2024-09-18T00:18:49.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026819428999999999, 2024-09-18T00:18:49.562 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.562 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.562 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.563 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.563 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.563 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.563 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12595 2024-09-18T00:18:49.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.563 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:18:49.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:18:49.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391705+0000", 2024-09-18T00:18:49.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3999999999999999e-06 2024-09-18T00:18:49.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391711+0000", 2024-09-18T00:18:49.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7459999999999997e-06 2024-09-18T00:18:49.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391733+0000", 2024-09-18T00:18:49.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2923000000000001e-05 2024-09-18T00:18:49.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391748+0000", 2024-09-18T00:18:49.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4790000000000001e-05 2024-09-18T00:18:49.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418180+0000", 2024-09-18T00:18:49.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026431888000000001 2024-09-18T00:18:49.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418328+0000", 2024-09-18T00:18:49.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000147725 2024-09-18T00:18:49.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418523+0000", 2024-09-18T00:18:49.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000194957 2024-09-18T00:18:49.566 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.566 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.566 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.567 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12596 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:49.567 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:18:49.567 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.128427531, 2024-09-18T00:18:49.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027220296000000001, 2024-09-18T00:18:49.567 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.567 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.567 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.567 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.567 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.567 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.567 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12596 2024-09-18T00:18:49.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.568 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:18:49.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:18:49.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391768+0000", 2024-09-18T00:18:49.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.283e-06 2024-09-18T00:18:49.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391773+0000", 2024-09-18T00:18:49.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3970000000000002e-06 2024-09-18T00:18:49.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391795+0000", 2024-09-18T00:18:49.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1835e-05 2024-09-18T00:18:49.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391809+0000", 2024-09-18T00:18:49.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4107000000000001e-05 2024-09-18T00:18:49.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418582+0000", 2024-09-18T00:18:49.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026772918999999999 2024-09-18T00:18:49.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418780+0000", 2024-09-18T00:18:49.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000198042 2024-09-18T00:18:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418987+0000", 2024-09-18T00:18:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000206713 2024-09-18T00:18:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12597 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:18:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.128367013, 2024-09-18T00:18:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027612714, 2024-09-18T00:18:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12597 2024-09-18T00:18:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:18:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:18:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391828+0000", 2024-09-18T00:18:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.203e-06 2024-09-18T00:18:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391834+0000", 2024-09-18T00:18:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6529999999999999e-06 2024-09-18T00:18:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391856+0000", 2024-09-18T00:18:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2195999999999999e-05 2024-09-18T00:18:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391870+0000", 2024-09-18T00:18:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4063000000000001e-05 2024-09-18T00:18:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419049+0000", 2024-09-18T00:18:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027178751000000001 2024-09-18T00:18:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419239+0000", 2024-09-18T00:18:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019039400000000001 2024-09-18T00:18:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419440+0000", 2024-09-18T00:18:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000200454 2024-09-18T00:18:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12598 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:18:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.128303962, 2024-09-18T00:18:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028269056000000001, 2024-09-18T00:18:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12598 2024-09-18T00:18:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:18:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:18:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391891+0000", 2024-09-18T00:18:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.257e-06 2024-09-18T00:18:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391897+0000", 2024-09-18T00:18:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9560000000000002e-06 2024-09-18T00:18:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391920+0000", 2024-09-18T00:18:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3025999999999999e-05 2024-09-18T00:18:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391935+0000", 2024-09-18T00:18:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4853e-05 2024-09-18T00:18:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419527+0000", 2024-09-18T00:18:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027592378000000001 2024-09-18T00:18:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419681+0000", 2024-09-18T00:18:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015316 2024-09-18T00:18:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420159+0000", 2024-09-18T00:18:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000478426 2024-09-18T00:18:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12599 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:18:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.128240107, 2024-09-18T00:18:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028649857000000001, 2024-09-18T00:18:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12599 2024-09-18T00:18:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:18:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:18:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391955+0000", 2024-09-18T00:18:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-06 2024-09-18T00:18:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391961+0000", 2024-09-18T00:18:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7100000000000004e-06 2024-09-18T00:18:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391982+0000", 2024-09-18T00:18:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1492e-05 2024-09-18T00:18:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391996+0000", 2024-09-18T00:18:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4194e-05 2024-09-18T00:18:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420237+0000", 2024-09-18T00:18:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028240818000000001 2024-09-18T00:18:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420430+0000", 2024-09-18T00:18:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019228299999999999 2024-09-18T00:18:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420604+0000", 2024-09-18T00:18:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000174117 2024-09-18T00:18:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12600 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 270336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:18:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.128179113, 2024-09-18T00:18:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028964586, 2024-09-18T00:18:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12600 2024-09-18T00:18:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:18:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:18:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392016+0000", 2024-09-18T00:18:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.31e-06 2024-09-18T00:18:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392022+0000", 2024-09-18T00:18:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5600000000000001e-06 2024-09-18T00:18:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392045+0000", 2024-09-18T00:18:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3595999999999999e-05 2024-09-18T00:18:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392060+0000", 2024-09-18T00:18:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4727e-05 2024-09-18T00:18:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420669+0000", 2024-09-18T00:18:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028608855999999998 2024-09-18T00:18:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420813+0000", 2024-09-18T00:18:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014429500000000001 2024-09-18T00:18:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420979+0000", 2024-09-18T00:18:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016624200000000001 2024-09-18T00:18:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12601 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:18:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.128115208000001, 2024-09-18T00:18:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029292832000000001, 2024-09-18T00:18:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12601 2024-09-18T00:18:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:18:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:18:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392080+0000", 2024-09-18T00:18:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:18:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392086+0000", 2024-09-18T00:18:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7570000000000001e-06 2024-09-18T00:18:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392105+0000", 2024-09-18T00:18:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9415999999999998e-05 2024-09-18T00:18:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392132+0000", 2024-09-18T00:18:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7019e-05 2024-09-18T00:18:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421049+0000", 2024-09-18T00:18:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028916659000000001 2024-09-18T00:18:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421204+0000", 2024-09-18T00:18:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155224 2024-09-18T00:18:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421372+0000", 2024-09-18T00:18:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016754699999999999 2024-09-18T00:18:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12602 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:18:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.128042577, 2024-09-18T00:18:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029612833000000002, 2024-09-18T00:18:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12602 2024-09-18T00:18:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:18:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:18:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392152+0000", 2024-09-18T00:18:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0470000000000001e-06 2024-09-18T00:18:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392158+0000", 2024-09-18T00:18:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:18:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392181+0000", 2024-09-18T00:18:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2629e-05 2024-09-18T00:18:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392222+0000", 2024-09-18T00:18:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0772000000000002e-05 2024-09-18T00:18:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421441+0000", 2024-09-18T00:18:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029219327 2024-09-18T00:18:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421608+0000", 2024-09-18T00:18:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000166813 2024-09-18T00:18:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421764+0000", 2024-09-18T00:18:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015628500000000001 2024-09-18T00:18:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12603 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:18:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.127952183, 2024-09-18T00:18:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029902524, 2024-09-18T00:18:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12603 2024-09-18T00:18:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:18:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:18:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392243+0000", 2024-09-18T00:18:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:18:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392260+0000", 2024-09-18T00:18:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6860000000000001e-05 2024-09-18T00:18:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392282+0000", 2024-09-18T00:18:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2713000000000001e-05 2024-09-18T00:18:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392297+0000", 2024-09-18T00:18:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4375e-05 2024-09-18T00:18:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421830+0000", 2024-09-18T00:18:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029532668000000002 2024-09-18T00:18:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421985+0000", 2024-09-18T00:18:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155613 2024-09-18T00:18:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422144+0000", 2024-09-18T00:18:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000159102 2024-09-18T00:18:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12604 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:18:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.127877132, 2024-09-18T00:18:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030228695, 2024-09-18T00:18:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12604 2024-09-18T00:18:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:18:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:18:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392318+0000", 2024-09-18T00:18:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:18:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392337+0000", 2024-09-18T00:18:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:18:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392364+0000", 2024-09-18T00:18:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7288999999999999e-05 2024-09-18T00:18:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392379+0000", 2024-09-18T00:18:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:18:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422209+0000", 2024-09-18T00:18:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029829050999999999 2024-09-18T00:18:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422388+0000", 2024-09-18T00:18:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017988699999999999 2024-09-18T00:18:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422545+0000", 2024-09-18T00:18:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015701199999999999 2024-09-18T00:18:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12605 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:18:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.127792571000001, 2024-09-18T00:18:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030488674, 2024-09-18T00:18:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12605 2024-09-18T00:18:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:18:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:18:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392403+0000", 2024-09-18T00:18:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.883e-06 2024-09-18T00:18:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392421+0000", 2024-09-18T00:18:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7526000000000001e-05 2024-09-18T00:18:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392450+0000", 2024-09-18T00:18:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8972e-05 2024-09-18T00:18:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392488+0000", 2024-09-18T00:18:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8522000000000001e-05 2024-09-18T00:18:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422614+0000", 2024-09-18T00:18:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030125945000000001 2024-09-18T00:18:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422752+0000", 2024-09-18T00:18:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000138178 2024-09-18T00:18:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422890+0000", 2024-09-18T00:18:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013764800000000001 2024-09-18T00:18:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12606 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:18:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.127683911, 2024-09-18T00:18:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033727073000000003, 2024-09-18T00:18:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12606 2024-09-18T00:18:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:18:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:18:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392511+0000", 2024-09-18T00:18:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3090000000000001e-06 2024-09-18T00:18:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392517+0000", 2024-09-18T00:18:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.993e-06 2024-09-18T00:18:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392543+0000", 2024-09-18T00:18:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6152999999999998e-05 2024-09-18T00:18:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392558+0000", 2024-09-18T00:18:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4610000000000001e-05 2024-09-18T00:18:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.425959+0000", 2024-09-18T00:18:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033400484000000001 2024-09-18T00:18:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426106+0000", 2024-09-18T00:18:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014779800000000001 2024-09-18T00:18:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426237+0000", 2024-09-18T00:18:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013072600000000001 2024-09-18T00:18:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.616 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:49.618 INFO:teuthology.orchestra.run.smithi097.stdout:2939 read 0xdf3774b thru 0xdf3e3ce (0x6c84 bytes) 2024-09-18T00:18:49.618 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:49.618 INFO:teuthology.orchestra.run.smithi097.stdout:2943 punch from 0xc609285 to 0xc617d5e, (0xead9 bytes) 2024-09-18T00:18:49.618 INFO:teuthology.orchestra.run.smithi097.stdout:2944 trunc from 0xee5eae1 to 0xe5031ae 2024-09-18T00:18:49.618 INFO:teuthology.orchestra.run.smithi097.stdout:2946 read 0xb9fc185 thru 0xba04093 (0x7f0f bytes) 2024-09-18T00:18:49.618 INFO:teuthology.orchestra.run.smithi097.stdout:2947 trunc from 0xe5031ae to 0x23ed3d2 2024-09-18T00:18:49.618 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:18:49.619 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:49.610+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:49.656 INFO:teuthology.orchestra.run.smithi097.stdout:2948 write 0xac81093 thru 0xac868bc (0x582a bytes) 2024-09-18T00:18:49.659 INFO:teuthology.orchestra.run.smithi097.stdout:2951 write 0x786b777 thru 0x7874d2e (0x95b8 bytes) 2024-09-18T00:18:49.676 INFO:teuthology.orchestra.run.smithi097.stdout:2952 trunc from 0xac868bd to 0x6cd6d0d 2024-09-18T00:18:49.691 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:49.691 INFO:teuthology.orchestra.run.smithi097.stdout:2954 punch from 0x12eb2d6 to 0x12f9241, (0xdf6b bytes) 2024-09-18T00:18:49.691 INFO:teuthology.orchestra.run.smithi097.stdout:2956 write 0xaf21aa thru 0xaf6be8 (0x4a3f bytes) 2024-09-18T00:18:49.695 INFO:teuthology.orchestra.run.smithi097.stdout:2957 trunc from 0x6cd6d0d to 0x6144eb6 2024-09-18T00:18:49.697 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is unset 2024-09-18T00:18:49.718 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:49.714+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:49.719 INFO:teuthology.orchestra.run.smithi097.stdout:2958 trunc from 0x6144eb6 to 0x9391aa4 2024-09-18T00:18:49.727 INFO:teuthology.orchestra.run.smithi097.stdout:2960 write 0x2f91e42 thru 0x2f96e4b (0x500a bytes) 2024-09-18T00:18:49.743 INFO:teuthology.orchestra.run.smithi097.stdout:2961 write 0xd0daf41 thru 0xd0dc384 (0x1444 bytes) 2024-09-18T00:18:49.745 INFO:teuthology.orchestra.run.smithi097.stdout:2963 write 0xc5c9d37 thru 0xc5d92e4 (0xf5ae bytes) 2024-09-18T00:18:49.774 INFO:teuthology.orchestra.run.smithi097.stdout:2964 trunc from 0xd0dc385 to 0x10ca797 2024-09-18T00:18:49.804 INFO:teuthology.orchestra.run.smithi097.stdout:2965 trunc from 0x10ca797 to 0xc346102 2024-09-18T00:18:49.806 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:49.806 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:49.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.806 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17478 3.15 3:ad4938b4:::rbd_object_map.112a6efb7564:head [call lock.assert_locked in=44b,call rbd.object_map_resize in=29b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T00:18:49.806 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.804837+0000", 2024-09-18T00:18:49.806 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.00088875900000000003, 2024-09-18T00:18:49.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00094339100000000002, 2024-09-18T00:18:49.807 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:49.807 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:49.807 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "waiting for sub ops", 2024-09-18T00:18:49.807 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:49.807 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:49.807 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:49.807 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17478 2024-09-18T00:18:49.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.807 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:49.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:49.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.804837+0000", 2024-09-18T00:18:49.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:49.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.804837+0000", 2024-09-18T00:18:49.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:49.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:49.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.804840+0000", 2024-09-18T00:18:49.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.243e-06 2024-09-18T00:18:49.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:49.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.804851+0000", 2024-09-18T00:18:49.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1473e-05 2024-09-18T00:18:49.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:49.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.804882+0000", 2024-09-18T00:18:49.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0613e-05 2024-09-18T00:18:49.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:49.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.804903+0000", 2024-09-18T00:18:49.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0948999999999999e-05 2024-09-18T00:18:49.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.805049+0000", 2024-09-18T00:18:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014660000000000001 2024-09-18T00:18:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.805198+0000", 2024-09-18T00:18:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014834000000000001 2024-09-18T00:18:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:18:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.805578+0000", 2024-09-18T00:18:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00038052099999999998 2024-09-18T00:18:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.811 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:49.811 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.811 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:49.811 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-18T00:18:49.811 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 1 2024-09-18T00:18:49.811 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:49.816 INFO:teuthology.orchestra.run.smithi097.stdout:2966 write 0x1749994 thru 0x1755117 (0xb784 bytes) 2024-09-18T00:18:49.819 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:49.814+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:49.822 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:18:49.824 INFO:teuthology.orchestra.run.smithi097.stdout:2967 read 0x860137d thru 0x860b6f9 (0xa37d bytes) 2024-09-18T00:18:49.824 INFO:teuthology.orchestra.run.smithi097.stdout:2968 read 0x228fbd6 thru 0x2295dc9 (0x61f4 bytes) 2024-09-18T00:18:49.825 INFO:teuthology.orchestra.run.smithi097.stdout:2969 write 0x25db726 thru 0x25e3163 (0x7a3e bytes) 2024-09-18T00:18:49.829 INFO:teuthology.orchestra.run.smithi097.stdout:2970 read 0x12f3210 thru 0x12fe0b3 (0xaea4 bytes) 2024-09-18T00:18:49.830 INFO:teuthology.orchestra.run.smithi097.stdout:2971 write 0x2fa6ab5 thru 0x2fa8ca0 (0x21ec bytes) 2024-09-18T00:18:49.844 INFO:teuthology.orchestra.run.smithi097.stdout:2972 write 0xdf2a297 thru 0xdf300f5 (0x5e5f bytes) 2024-09-18T00:18:49.861 INFO:teuthology.orchestra.run.smithi097.stdout:2974 trunc from 0xdf300f6 to 0x67b9b2b 2024-09-18T00:18:49.880 INFO:teuthology.orchestra.run.smithi097.stdout:2975 trunc from 0x67b9b2b to 0x9293c93 2024-09-18T00:18:49.902 INFO:teuthology.orchestra.run.smithi097.stdout:2976 write 0xa21faa4 thru 0xa2224ee (0x2a4b bytes) 2024-09-18T00:18:49.906 INFO:teuthology.orchestra.run.smithi097.stdout:2977 write 0x9f47e39 thru 0x9f55797 (0xd95f bytes) 2024-09-18T00:18:49.914 INFO:teuthology.orchestra.run.smithi097.stdout:2979 read 0x16677b9 thru 0x1670ad2 (0x931a bytes) 2024-09-18T00:18:49.915 INFO:teuthology.orchestra.run.smithi097.stdout:2980 read 0x45821e8 thru 0x458d840 (0xb659 bytes) 2024-09-18T00:18:49.917 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:49.917 INFO:teuthology.orchestra.run.smithi097.stdout:2983 punch from 0x2b10ca9 to 0x2b1120a, (0x561 bytes) 2024-09-18T00:18:49.917 INFO:teuthology.orchestra.run.smithi097.stdout:2984 punch from 0x5454666 to 0x545488d, (0x227 bytes) 2024-09-18T00:18:49.917 INFO:teuthology.orchestra.run.smithi097.stdout:2986 punch from 0x434a847 to 0x4359b77, (0xf330 bytes) 2024-09-18T00:18:49.917 INFO:teuthology.orchestra.run.smithi097.stdout:2987 writesame 0x7851ca9 thru 0x785a9fb data_size 0xd(0x8d53 bytes) 2024-09-18T00:18:49.920 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:49.914+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:49.925 INFO:teuthology.orchestra.run.smithi097.stdout:2988 punch from 0x9ec46e7 to 0x9eca50f, (0x5e28 bytes) 2024-09-18T00:18:49.925 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:49.925 INFO:teuthology.orchestra.run.smithi097.stdout:2990 punch from 0x5af92d to 0x5b3773, (0x3e46 bytes) 2024-09-18T00:18:49.938 INFO:teuthology.orchestra.run.smithi097.stdout:2991 trunc from 0xa2224ef to 0x53801cd 2024-09-18T00:18:49.950 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:49.950 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:49.950 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:49.950 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:49.951 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:49.957 INFO:teuthology.orchestra.run.smithi097.stdout:2992 writesame 0x1e24788 thru 0x1e2771a data_size 0x1(0x2f93 bytes) 2024-09-18T00:18:49.965 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:18:50.010 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:50.020 INFO:teuthology.orchestra.run.smithi097.stdout:2995 trunc from 0x53801cd to 0x19a82d3 2024-09-18T00:18:50.021 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:50.018+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:50.034 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:50.034 INFO:teuthology.orchestra.run.smithi097.stdout:2997 punch from 0xc4a1c2 to 0xc579d0, (0xd80e bytes) 2024-09-18T00:18:50.034 INFO:teuthology.orchestra.run.smithi097.stdout:2998 punch from 0xda3d5 to 0xe31e9, (0x8e14 bytes) 2024-09-18T00:18:50.034 INFO:teuthology.orchestra.run.smithi097.stdout:2999 read 0xeb78af thru 0xeb9290 (0x19e2 bytes) 2024-09-18T00:18:50.034 INFO:teuthology.orchestra.run.smithi097.stdout:3000 read 0x10d0512 thru 0x10d1df6 (0x18e5 bytes) 2024-09-18T00:18:50.035 INFO:teuthology.orchestra.run.smithi097.stdout:3001 read 0x94e277 thru 0x94f773 (0x14fd bytes) 2024-09-18T00:18:50.047 INFO:teuthology.orchestra.run.smithi097.stdout:3003 write 0x27445fc thru 0x2744bf1 (0x5f6 bytes) 2024-09-18T00:18:50.058 INFO:teuthology.orchestra.run.smithi097.stdout:3004 writesame 0x30405c6 thru 0x3045afd data_size 0x6c(0x5538 bytes) 2024-09-18T00:18:50.060 INFO:teuthology.orchestra.run.smithi097.stdout:3005 read 0x48a357 thru 0x48e720 (0x43ca bytes) 2024-09-18T00:18:50.087 INFO:teuthology.orchestra.run.smithi097.stdout:3006 write 0xb67a334 thru 0xb687477 (0xd144 bytes) 2024-09-18T00:18:50.089 INFO:teuthology.orchestra.run.smithi097.stdout:3007 punch from 0x41a8f59 to 0x41b1b94, (0x8c3b bytes) 2024-09-18T00:18:50.091 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:50.091 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:50.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:50.091 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:50.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.091 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2228 3.15 3:aa2fc059:::rbd_data.112aa7ba4578.000000000000014a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:18:50.091 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:18:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 29.825929546000001, 2024-09-18T00:18:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.068966377999999995, 2024-09-18T00:18:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2228 2024-09-18T00:18:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:18:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:18:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263099+0000", 2024-09-18T00:18:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2359999999999999e-06 2024-09-18T00:18:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263115+0000", 2024-09-18T00:18:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5996999999999999e-05 2024-09-18T00:18:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263156+0000", 2024-09-18T00:18:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:18:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263183+0000", 2024-09-18T00:18:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7233000000000002e-05 2024-09-18T00:18:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263304+0000", 2024-09-18T00:18:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121209 2024-09-18T00:18:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263528+0000", 2024-09-18T00:18:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022401800000000001 2024-09-18T00:18:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:18:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.266458+0000", 2024-09-18T00:18:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029296980000000001 2024-09-18T00:18:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331628+0000", 2024-09-18T00:18:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.065170045999999995 2024-09-18T00:18:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331819+0000", 2024-09-18T00:18:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000190819 2024-09-18T00:18:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331971+0000", 2024-09-18T00:18:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151972 2024-09-18T00:18:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332063+0000", 2024-09-18T00:18:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2083999999999995e-05 2024-09-18T00:18:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2228 3.15 e36/32 ondisk, result = 0)", 2024-09-18T00:18:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:18:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 29.812876434, 2024-09-18T00:18:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055930148999999998, 2024-09-18T00:18:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:18:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:18:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276152+0000", 2024-09-18T00:18:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0200000000000001e-06 2024-09-18T00:18:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276162+0000", 2024-09-18T00:18:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0767e-05 2024-09-18T00:18:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276202+0000", 2024-09-18T00:18:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000003e-05 2024-09-18T00:18:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276226+0000", 2024-09-18T00:18:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.351e-05 2024-09-18T00:18:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331782+0000", 2024-09-18T00:18:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055556296999999998 2024-09-18T00:18:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331801+0000", 2024-09-18T00:18:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9357000000000001e-05 2024-09-18T00:18:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332080+0000", 2024-09-18T00:18:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027849800000000002 2024-09-18T00:18:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2854 3.f 3:f02b0bca:::rbd_data.112aa7ba4578.0000000000000135:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:18:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:18:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.218957056000001, 2024-09-18T00:18:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.082582802999999996, 2024-09-18T00:18:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2854 2024-09-18T00:18:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:18:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:18:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870071+0000", 2024-09-18T00:18:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.227e-06 2024-09-18T00:18:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870083+0000", 2024-09-18T00:18:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1909999999999999e-05 2024-09-18T00:18:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870114+0000", 2024-09-18T00:18:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0576e-05 2024-09-18T00:18:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870129+0000", 2024-09-18T00:18:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.485e-05 2024-09-18T00:18:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870256+0000", 2024-09-18T00:18:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012691300000000001 2024-09-18T00:18:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870493+0000", 2024-09-18T00:18:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023737399999999999 2024-09-18T00:18:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.874933+0000", 2024-09-18T00:18:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044398299999999996 2024-09-18T00:18:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952452+0000", 2024-09-18T00:18:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.077518918000000006 2024-09-18T00:18:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952486+0000", 2024-09-18T00:18:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4035999999999997e-05 2024-09-18T00:18:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952575+0000", 2024-09-18T00:18:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.954e-05 2024-09-18T00:18:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952652+0000", 2024-09-18T00:18:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6629000000000001e-05 2024-09-18T00:18:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2854 3.f e38/32 ondisk, result = 0)", 2024-09-18T00:18:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:18:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.201426241, 2024-09-18T00:18:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064868625999999999, 2024-09-18T00:18:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:18:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:18:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887602+0000", 2024-09-18T00:18:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.677e-06 2024-09-18T00:18:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887609+0000", 2024-09-18T00:18:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6729999999999992e-06 2024-09-18T00:18:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887637+0000", 2024-09-18T00:18:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8206000000000001e-05 2024-09-18T00:18:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887651+0000", 2024-09-18T00:18:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3387000000000001e-05 2024-09-18T00:18:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952422+0000", 2024-09-18T00:18:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064771366999999996 2024-09-18T00:18:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952441+0000", 2024-09-18T00:18:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8737e-05 2024-09-18T00:18:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952469+0000", 2024-09-18T00:18:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7579000000000001e-05 2024-09-18T00:18:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5633 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297194+0000", 2024-09-18T00:18:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.791832412000002, 2024-09-18T00:18:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046979053, 2024-09-18T00:18:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5633 2024-09-18T00:18:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297194+0000", 2024-09-18T00:18:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297194+0000", 2024-09-18T00:18:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297195+0000", 2024-09-18T00:18:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2440000000000001e-06 2024-09-18T00:18:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297201+0000", 2024-09-18T00:18:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5489999999999997e-06 2024-09-18T00:18:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297218+0000", 2024-09-18T00:18:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7652999999999999e-05 2024-09-18T00:18:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297245+0000", 2024-09-18T00:18:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7212999999999998e-05 2024-09-18T00:18:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343955+0000", 2024-09-18T00:18:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046709564000000002 2024-09-18T00:18:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344028+0000", 2024-09-18T00:18:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3370999999999999e-05 2024-09-18T00:18:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344173+0000", 2024-09-18T00:18:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014445899999999999 2024-09-18T00:18:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5634 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297263+0000", 2024-09-18T00:18:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.791763136, 2024-09-18T00:18:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047153893000000002, 2024-09-18T00:18:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5634 2024-09-18T00:18:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297263+0000", 2024-09-18T00:18:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297263+0000", 2024-09-18T00:18:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297264+0000", 2024-09-18T00:18:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4169999999999999e-06 2024-09-18T00:18:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297270+0000", 2024-09-18T00:18:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4829999999999999e-06 2024-09-18T00:18:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297287+0000", 2024-09-18T00:18:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6993000000000001e-05 2024-09-18T00:18:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297300+0000", 2024-09-18T00:18:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.324e-05 2024-09-18T00:18:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344221+0000", 2024-09-18T00:18:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046921030000000002 2024-09-18T00:18:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344310+0000", 2024-09-18T00:18:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8533000000000001e-05 2024-09-18T00:18:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344417+0000", 2024-09-18T00:18:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107197 2024-09-18T00:18:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5635 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297317+0000", 2024-09-18T00:18:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.791709386000001, 2024-09-18T00:18:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047292222000000002, 2024-09-18T00:18:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5635 2024-09-18T00:18:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297317+0000", 2024-09-18T00:18:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297317+0000", 2024-09-18T00:18:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297318+0000", 2024-09-18T00:18:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2869999999999999e-06 2024-09-18T00:18:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297323+0000", 2024-09-18T00:18:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1760000000000001e-06 2024-09-18T00:18:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297343+0000", 2024-09-18T00:18:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9473000000000001e-05 2024-09-18T00:18:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297356+0000", 2024-09-18T00:18:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3767e-05 2024-09-18T00:18:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344455+0000", 2024-09-18T00:18:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047098220000000003 2024-09-18T00:18:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344518+0000", 2024-09-18T00:18:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2856999999999998e-05 2024-09-18T00:18:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344609+0000", 2024-09-18T00:18:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1441999999999996e-05 2024-09-18T00:18:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5674 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 10682368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298168+0000", 2024-09-18T00:18:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.790858582999999, 2024-09-18T00:18:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047098783999999998, 2024-09-18T00:18:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5674 2024-09-18T00:18:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298168+0000", 2024-09-18T00:18:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298168+0000", 2024-09-18T00:18:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298169+0000", 2024-09-18T00:18:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1370000000000001e-06 2024-09-18T00:18:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298174+0000", 2024-09-18T00:18:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6500000000000001e-06 2024-09-18T00:18:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298192+0000", 2024-09-18T00:18:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7620000000000001e-05 2024-09-18T00:18:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298205+0000", 2024-09-18T00:18:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3363000000000001e-05 2024-09-18T00:18:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345013+0000", 2024-09-18T00:18:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046807418000000003 2024-09-18T00:18:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345125+0000", 2024-09-18T00:18:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111901 2024-09-18T00:18:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345266+0000", 2024-09-18T00:18:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000141695 2024-09-18T00:18:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5675 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298223+0000", 2024-09-18T00:18:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.79080269, 2024-09-18T00:18:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047324643999999999, 2024-09-18T00:18:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5675 2024-09-18T00:18:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298223+0000", 2024-09-18T00:18:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298223+0000", 2024-09-18T00:18:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298225+0000", 2024-09-18T00:18:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.13e-06 2024-09-18T00:18:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298230+0000", 2024-09-18T00:18:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1599999999999997e-06 2024-09-18T00:18:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298247+0000", 2024-09-18T00:18:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6813999999999999e-05 2024-09-18T00:18:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298260+0000", 2024-09-18T00:18:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3036e-05 2024-09-18T00:18:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345315+0000", 2024-09-18T00:18:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047055420000000001 2024-09-18T00:18:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345384+0000", 2024-09-18T00:18:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9375999999999994e-05 2024-09-18T00:18:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345548+0000", 2024-09-18T00:18:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016370800000000001 2024-09-18T00:18:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5676 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 12255232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:18:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.790750383999999, 2024-09-18T00:18:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047571816000000003, 2024-09-18T00:18:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5676 2024-09-18T00:18:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:18:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:18:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298277+0000", 2024-09-18T00:18:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.454e-06 2024-09-18T00:18:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298282+0000", 2024-09-18T00:18:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6299999999999997e-06 2024-09-18T00:18:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298299+0000", 2024-09-18T00:18:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6959000000000001e-05 2024-09-18T00:18:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298313+0000", 2024-09-18T00:18:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.376e-05 2024-09-18T00:18:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345610+0000", 2024-09-18T00:18:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047297128000000001 2024-09-18T00:18:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345708+0000", 2024-09-18T00:18:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8245999999999994e-05 2024-09-18T00:18:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345848+0000", 2024-09-18T00:18:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000139639 2024-09-18T00:18:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5677 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 12779520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:18:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.790696871000002, 2024-09-18T00:18:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047810679000000002, 2024-09-18T00:18:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5677 2024-09-18T00:18:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:18:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:18:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298330+0000", 2024-09-18T00:18:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.217e-06 2024-09-18T00:18:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298354+0000", 2024-09-18T00:18:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3753000000000001e-05 2024-09-18T00:18:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298388+0000", 2024-09-18T00:18:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4276999999999997e-05 2024-09-18T00:18:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298415+0000", 2024-09-18T00:18:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6403000000000001e-05 2024-09-18T00:18:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345896+0000", 2024-09-18T00:18:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047481007999999998 2024-09-18T00:18:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345992+0000", 2024-09-18T00:18:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6141e-05 2024-09-18T00:18:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346140+0000", 2024-09-18T00:18:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014788 2024-09-18T00:18:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5678 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:18:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.790590691999999, 2024-09-18T00:18:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047985466999999997, 2024-09-18T00:18:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5678 2024-09-18T00:18:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:18:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:18:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298437+0000", 2024-09-18T00:18:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1799999999999999e-06 2024-09-18T00:18:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298443+0000", 2024-09-18T00:18:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1140000000000002e-06 2024-09-18T00:18:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298460+0000", 2024-09-18T00:18:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7615999999999999e-05 2024-09-18T00:18:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298474+0000", 2024-09-18T00:18:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4017e-05 2024-09-18T00:18:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346180+0000", 2024-09-18T00:18:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047705801999999999 2024-09-18T00:18:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346274+0000", 2024-09-18T00:18:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3814999999999995e-05 2024-09-18T00:18:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346421+0000", 2024-09-18T00:18:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000146923 2024-09-18T00:18:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5679 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:18:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.790531647999998, 2024-09-18T00:18:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048190771, 2024-09-18T00:18:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5679 2024-09-18T00:18:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:18:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:18:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298496+0000", 2024-09-18T00:18:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4699999999999999e-06 2024-09-18T00:18:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298501+0000", 2024-09-18T00:18:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:18:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298519+0000", 2024-09-18T00:18:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7329e-05 2024-09-18T00:18:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298532+0000", 2024-09-18T00:18:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3083e-05 2024-09-18T00:18:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346462+0000", 2024-09-18T00:18:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047930449999999999 2024-09-18T00:18:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346556+0000", 2024-09-18T00:18:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3222999999999998e-05 2024-09-18T00:18:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346685+0000", 2024-09-18T00:18:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000129696 2024-09-18T00:18:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5680 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:18:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.790474239000002, 2024-09-18T00:18:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048535847, 2024-09-18T00:18:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5680 2024-09-18T00:18:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:18:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:18:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298553+0000", 2024-09-18T00:18:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0759999999999999e-06 2024-09-18T00:18:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298575+0000", 2024-09-18T00:18:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2019999999999999e-05 2024-09-18T00:18:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298593+0000", 2024-09-18T00:18:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7900000000000001e-05 2024-09-18T00:18:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298607+0000", 2024-09-18T00:18:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3844e-05 2024-09-18T00:18:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346791+0000", 2024-09-18T00:18:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048184449999999997 2024-09-18T00:18:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346921+0000", 2024-09-18T00:18:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012994699999999999 2024-09-18T00:18:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347088+0000", 2024-09-18T00:18:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016661000000000001 2024-09-18T00:18:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5681 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:18:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.790401829, 2024-09-18T00:18:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048743337999999997, 2024-09-18T00:18:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5681 2024-09-18T00:18:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:18:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:18:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298626+0000", 2024-09-18T00:18:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4899999999999999e-06 2024-09-18T00:18:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298631+0000", 2024-09-18T00:18:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0300000000000001e-06 2024-09-18T00:18:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298650+0000", 2024-09-18T00:18:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9499000000000001e-05 2024-09-18T00:18:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298664+0000", 2024-09-18T00:18:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3767e-05 2024-09-18T00:18:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347139+0000", 2024-09-18T00:18:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048475348000000001 2024-09-18T00:18:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347228+0000", 2024-09-18T00:18:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8843000000000003e-05 2024-09-18T00:18:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347368+0000", 2024-09-18T00:18:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013936100000000001 2024-09-18T00:18:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5682 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:18:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.790345014, 2024-09-18T00:18:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048916864999999997, 2024-09-18T00:18:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5682 2024-09-18T00:18:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:18:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:18:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298682+0000", 2024-09-18T00:18:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:18:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298687+0000", 2024-09-18T00:18:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9239999999999998e-06 2024-09-18T00:18:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298704+0000", 2024-09-18T00:18:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6569999999999999e-05 2024-09-18T00:18:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298731+0000", 2024-09-18T00:18:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6752999999999999e-05 2024-09-18T00:18:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347416+0000", 2024-09-18T00:18:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048685056999999997 2024-09-18T00:18:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347506+0000", 2024-09-18T00:18:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9936000000000001e-05 2024-09-18T00:18:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347598+0000", 2024-09-18T00:18:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2335000000000003e-05 2024-09-18T00:18:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15879 3.16 3:692b47b6:::rbd_object_map.112a13ec2d6c.000000000000000b:head [call] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:18:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:18:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4342623339999996, 2024-09-18T00:18:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.011184018, 2024-09-18T00:18:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15879 2024-09-18T00:18:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:18:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:18:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654768+0000", 2024-09-18T00:18:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9129999999999996e-06 2024-09-18T00:18:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654786+0000", 2024-09-18T00:18:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8695999999999999e-05 2024-09-18T00:18:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654832+0000", 2024-09-18T00:18:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5735000000000002e-05 2024-09-18T00:18:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654860+0000", 2024-09-18T00:18:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7789000000000001e-05 2024-09-18T00:18:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655048+0000", 2024-09-18T00:18:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018781900000000001 2024-09-18T00:18:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:18:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655070+0000", 2024-09-18T00:18:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1916000000000002e-05 2024-09-18T00:18:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663335+0000", 2024-09-18T00:18:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0082651210000000003 2024-09-18T00:18:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:18:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663358+0000", 2024-09-18T00:18:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2986e-05 2024-09-18T00:18:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663739+0000", 2024-09-18T00:18:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0003814820000001 2024-09-18T00:18:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663920+0000", 2024-09-18T00:18:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018116900000000001 2024-09-18T00:18:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.665948+0000", 2024-09-18T00:18:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020273919999999998 2024-09-18T00:18:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15881 3.16 3:6cc14920:::rbd_header.112a3285400b:head [watch reconnect cookie 140685950184608 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:18:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:18:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4341013389999997, 2024-09-18T00:18:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0138723599999999, 2024-09-18T00:18:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15881 2024-09-18T00:18:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:18:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:18:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654926+0000", 2024-09-18T00:18:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6470000000000001e-06 2024-09-18T00:18:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654935+0000", 2024-09-18T00:18:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8859999999999993e-06 2024-09-18T00:18:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654960+0000", 2024-09-18T00:18:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4689999999999999e-05 2024-09-18T00:18:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654976+0000", 2024-09-18T00:18:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6319000000000001e-05 2024-09-18T00:18:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655238+0000", 2024-09-18T00:18:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026145999999999998 2024-09-18T00:18:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:18:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655255+0000", 2024-09-18T00:18:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7249999999999999e-05 2024-09-18T00:18:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663416+0000", 2024-09-18T00:18:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081607959999999997 2024-09-18T00:18:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:18:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663436+0000", 2024-09-18T00:18:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0384e-05 2024-09-18T00:18:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666184+0000", 2024-09-18T00:18:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002748121 2024-09-18T00:18:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666452+0000", 2024-09-18T00:18:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026801700000000001 2024-09-18T00:18:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.668797+0000", 2024-09-18T00:18:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023447899999999998 2024-09-18T00:18:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17089 3.f 3:f7681386:::rbd_data.112a6efb7564.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:18:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:18:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.2091742430000001, 2024-09-18T00:18:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.12573889399999999, 2024-09-18T00:18:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17089 2024-09-18T00:18:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:18:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:18:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879854+0000", 2024-09-18T00:18:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:18:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879867+0000", 2024-09-18T00:18:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3103e-05 2024-09-18T00:18:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879898+0000", 2024-09-18T00:18:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0103e-05 2024-09-18T00:18:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879912+0000", 2024-09-18T00:18:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4636e-05 2024-09-18T00:18:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879992+0000", 2024-09-18T00:18:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9814000000000005e-05 2024-09-18T00:18:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.880101+0000", 2024-09-18T00:18:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109172 2024-09-18T00:18:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.887269+0000", 2024-09-18T00:18:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071679300000000003 2024-09-18T00:18:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005269+0000", 2024-09-18T00:18:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.117999621 2024-09-18T00:18:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005317+0000", 2024-09-18T00:18:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8279000000000003e-05 2024-09-18T00:18:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005464+0000", 2024-09-18T00:18:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014661200000000001 2024-09-18T00:18:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005591+0000", 2024-09-18T00:18:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000127107 2024-09-18T00:18:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17089 3.f e59/32 ondisk, result = 0)", 2024-09-18T00:18:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:18:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.1850673780000001, 2024-09-18T00:18:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101334403, 2024-09-18T00:18:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:18:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:18:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903960+0000", 2024-09-18T00:18:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.33e-06 2024-09-18T00:18:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903966+0000", 2024-09-18T00:18:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5929999999999996e-06 2024-09-18T00:18:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903986+0000", 2024-09-18T00:18:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.018e-05 2024-09-18T00:18:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903997+0000", 2024-09-18T00:18:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0988999999999999e-05 2024-09-18T00:18:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005171+0000", 2024-09-18T00:18:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101174267 2024-09-18T00:18:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005200+0000", 2024-09-18T00:18:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8540000000000001e-05 2024-09-18T00:18:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005293+0000", 2024-09-18T00:18:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504000000000005e-05 2024-09-18T00:18:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:50.204 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:50.205 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:18:50.207 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:18:50.209 INFO:teuthology.orchestra.run.smithi097.stdout:3012 trunc from 0xb687478 to 0x876d71c 2024-09-18T00:18:50.209 INFO:teuthology.orchestra.run.smithi097.stdout:3013 writesame 0x6b5fb11 thru 0x6b61bb9 data_size 0x3(0x20a9 bytes) 2024-09-18T00:18:50.209 INFO:teuthology.orchestra.run.smithi097.stdout:3014 trunc from 0x876d71c to 0xba94bec 2024-09-18T00:18:50.209 INFO:teuthology.orchestra.run.smithi097.stdout:3017 write 0x750d83e thru 0x751513b (0x78fe bytes) 2024-09-18T00:18:50.209 INFO:teuthology.orchestra.run.smithi097.stdout:3019 punch from 0xcfda6d to 0xd000a7, (0x263a bytes) 2024-09-18T00:18:50.209 INFO:teuthology.orchestra.run.smithi097.stdout:3020 write 0x349d705 thru 0x34a5d6a (0x8666 bytes) 2024-09-18T00:18:50.210 INFO:teuthology.orchestra.run.smithi097.stdout:3021 writesame 0xdf59bd thru 0xdfb7c8 data_size 0x1a(0x5e0c bytes) 2024-09-18T00:18:50.210 INFO:teuthology.orchestra.run.smithi097.stdout:3022 write 0x509d2c6 thru 0x50a8648 (0xb383 bytes) 2024-09-18T00:18:50.210 INFO:teuthology.orchestra.run.smithi097.stdout:3023 writesame 0x5ef5b26 thru 0x5ef915e data_size 0x7(0x3639 bytes) 2024-09-18T00:18:50.210 INFO:teuthology.orchestra.run.smithi097.stdout:3024 write 0xa777a72 thru 0xa7792cf (0x185e bytes) 2024-09-18T00:18:50.210 INFO:teuthology.orchestra.run.smithi097.stdout:3025 trunc from 0xba94bec to 0xd88f2f1 2024-09-18T00:18:50.210 INFO:teuthology.orchestra.run.smithi097.stdout:3027 write 0x61a3278 thru 0x61aff55 (0xccde bytes) 2024-09-18T00:18:50.210 INFO:teuthology.orchestra.run.smithi097.stdout:3030 read 0x8dda98e thru 0x8dde069 (0x36dc bytes) 2024-09-18T00:18:50.210 INFO:teuthology.orchestra.run.smithi097.stdout:3032 read 0xa5a1d17 thru 0xa5add37 (0xc021 bytes) 2024-09-18T00:18:50.210 INFO:teuthology.orchestra.run.smithi097.stdout:3033 punch from 0xa8305fb to 0xa834187, (0x3b8c bytes) 2024-09-18T00:18:50.210 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:50.210 INFO:teuthology.orchestra.run.smithi097.stdout:3035 write 0x9d1355e thru 0x9d1da06 (0xa4a9 bytes) 2024-09-18T00:18:50.210 INFO:teuthology.orchestra.run.smithi097.stdout:3037 read 0xbaacb18 thru 0xbabbd51 (0xf23a bytes) 2024-09-18T00:18:50.211 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:50.211 INFO:teuthology.orchestra.run.smithi097.stdout:3040 write 0x74c3994 thru 0x74cd74e (0x9dbb bytes) 2024-09-18T00:18:50.211 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:50.202+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:50.245 INFO:teuthology.orchestra.run.smithi097.stdout:3041 trunc from 0xd88f2f1 to 0x108ec56 2024-09-18T00:18:50.281 INFO:teuthology.orchestra.run.smithi097.stdout:3042 writesame 0x29a0439 thru 0x29a522f data_size 0x1(0x4df7 bytes) 2024-09-18T00:18:50.301 INFO:teuthology.orchestra.run.smithi097.stdout:3043 write 0x5ad0750 thru 0x5ada585 (0x9e36 bytes) 2024-09-18T00:18:50.304 INFO:teuthology.orchestra.run.smithi097.stdout:3044 write 0x4553b70 thru 0x45626f4 (0xeb85 bytes) 2024-09-18T00:18:50.306 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:50.302+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:50.326 INFO:teuthology.orchestra.run.smithi097.stdout:3045 writesame 0x5d7d595 thru 0x5d87889 data_size 0x1(0xa2f5 bytes) 2024-09-18T00:18:50.342 INFO:teuthology.orchestra.run.smithi097.stdout:3047 write 0x731b2d3 thru 0x7321363 (0x6091 bytes) 2024-09-18T00:18:50.345 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:50.345 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:50.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.345 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:17723 3.12 3:49e640ca:::rbd_data.112a6efb7564.0000000000000007:head [stat out=16b,write 12694227~24721 in=24721b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T00:18:50.345 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:50.342888+0000", 2024-09-18T00:18:50.345 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0026303030000000001, 2024-09-18T00:18:50.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0026676069999999998, 2024-09-18T00:18:50.345 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:50.346 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:50.346 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "waiting for sub ops", 2024-09-18T00:18:50.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:50.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:50.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:50.346 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 17723 2024-09-18T00:18:50.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.346 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:50.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:50.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.342888+0000", 2024-09-18T00:18:50.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:50.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:50.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.342888+0000", 2024-09-18T00:18:50.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:50.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:50.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.342891+0000", 2024-09-18T00:18:50.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.179e-06 2024-09-18T00:18:50.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:50.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.342922+0000", 2024-09-18T00:18:50.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0198000000000001e-05 2024-09-18T00:18:50.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:50.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.342947+0000", 2024-09-18T00:18:50.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5286000000000002e-05 2024-09-18T00:18:50.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.342960+0000", 2024-09-18T00:18:50.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3467e-05 2024-09-18T00:18:50.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:50.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.343059+0000", 2024-09-18T00:18:50.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8221999999999996e-05 2024-09-18T00:18:50.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:50.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.343178+0000", 2024-09-18T00:18:50.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119578 2024-09-18T00:18:50.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:50.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.343362+0000", 2024-09-18T00:18:50.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018415900000000001 2024-09-18T00:18:50.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:50.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.344464+0000", 2024-09-18T00:18:50.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011018169999999999 2024-09-18T00:18:50.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:50.350 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:50.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:50.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.350 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:17723 3.12 e60/32)", 2024-09-18T00:18:50.350 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:50.345533+0000", 2024-09-18T00:18:50.351 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4294967295.9999852, 2024-09-18T00:18:50.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014621, 2024-09-18T00:18:50.351 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:50.351 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:50.351 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:50.351 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:50.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:50.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.345533+0000", 2024-09-18T00:18:50.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:50.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:50.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.345533+0000", 2024-09-18T00:18:50.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:50.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:50.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.345535+0000", 2024-09-18T00:18:50.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.341e-06 2024-09-18T00:18:50.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:50.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.345544+0000", 2024-09-18T00:18:50.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4339999999999998e-06 2024-09-18T00:18:50.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:50.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.345559+0000", 2024-09-18T00:18:50.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4552000000000001e-05 2024-09-18T00:18:50.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:50.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.345566+0000", 2024-09-18T00:18:50.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2150000000000004e-06 2024-09-18T00:18:50.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:50.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.345664+0000", 2024-09-18T00:18:50.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7627000000000006e-05 2024-09-18T00:18:50.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:50.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:50.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:50.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:50.345676+0000", 2024-09-18T00:18:50.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2351000000000001e-05 2024-09-18T00:18:50.354 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:50.354 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:50.354 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:50.355 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:50.355 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-18T00:18:50.355 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 2 2024-09-18T00:18:50.355 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:50.362 INFO:teuthology.orchestra.run.smithi097.stdout:3048 write 0xb8b8ab5 thru 0xb8baddb (0x2327 bytes) 2024-09-18T00:18:50.363 INFO:teuthology.orchestra.run.smithi097.stdout: marking collection for removal 2024-09-18T00:18:50.363 INFO:teuthology.orchestra.run.smithi097.stdout:setting '_remove' omap key 2024-09-18T00:18:50.363 INFO:teuthology.orchestra.run.smithi097.stdout:finish_remove_pgs 2.4_head removing 2.4 2024-09-18T00:18:50.364 INFO:teuthology.orchestra.run.smithi097.stdout:Remove successful 2024-09-18T00:18:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:3050 read 0x70be213 thru 0x70ca8a9 (0xc697 bytes) 2024-09-18T00:18:50.367 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:18:50.367 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:50.367 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:50.368 INFO:teuthology.orchestra.run.smithi097.stdout:3051 write 0xf9e46 thru 0x102985 (0x8b40 bytes) 2024-09-18T00:18:50.368 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:18:50.370 INFO:teuthology.orchestra.run.smithi097.stdout:3054 punch from 0x2619222 to 0x2620972, (0x7750 bytes) 2024-09-18T00:18:50.370 INFO:teuthology.orchestra.run.smithi097.stdout:3055 read 0xa6eab82 thru 0xa6ed0bb (0x253a bytes) 2024-09-18T00:18:50.370 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:50.382 INFO:teuthology.orchestra.run.smithi097.stdout:3057 writesame 0xc936002 thru 0xc936820 data_size 0xb(0x81f bytes) 2024-09-18T00:18:50.385 INFO:teuthology.orchestra.run.smithi097.stdout:3058 write 0x3bfaa2b thru 0x3c0426c (0x9842 bytes) 2024-09-18T00:18:50.387 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:50.387 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:50.391 INFO:teuthology.orchestra.run.smithi097.stdout:3059 read 0x3445cc9 thru 0x344aa3c (0x4d74 bytes) 2024-09-18T00:18:50.392 INFO:teuthology.orchestra.run.smithi097.stdout:3062 punch from 0x84a47e0 to 0x84a5306, (0xb26 bytes) 2024-09-18T00:18:50.392 INFO:teuthology.orchestra.run.smithi097.stdout:3063 writesame 0x90e5fda thru 0x90e69bc data_size 0x1(0x9e3 bytes) 2024-09-18T00:18:50.393 INFO:teuthology.orchestra.run.smithi097.stdout:3064 read 0x6420802 thru 0x642aaf1 (0xa2f0 bytes) 2024-09-18T00:18:50.394 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:50.394 INFO:teuthology.orchestra.run.smithi097.stdout:3068 punch from 0xb52e2b5 to 0xb531bb5, (0x3900 bytes) 2024-09-18T00:18:50.394 INFO:teuthology.orchestra.run.smithi097.stdout:3069 write 0xc1fe59d thru 0xc20dd20 (0xf784 bytes) 2024-09-18T00:18:50.404 INFO:teuthology.orchestra.run.smithi097.stdout:3070 read 0x541b43f thru 0x5421565 (0x6127 bytes) 2024-09-18T00:18:50.404 INFO:teuthology.orchestra.run.smithi097.stdout:3071 writesame 0x3674529 thru 0x367ae68 data_size 0x20(0x6940 bytes) 2024-09-18T00:18:50.405 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:50.405 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:50.406 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:50.402+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:50.407 INFO:teuthology.orchestra.run.smithi097.stdout:3074 read 0xa55a992 thru 0xa5658d8 (0xaf47 bytes) 2024-09-18T00:18:50.407 INFO:teuthology.orchestra.run.smithi097.stdout:3075 writesame 0xb3bfece thru 0xb3cc907 data_size 0x1f(0xca3a bytes) 2024-09-18T00:18:50.418 INFO:teuthology.orchestra.run.smithi097.stdout:3076 read 0xa407049 thru 0xa40a881 (0x3839 bytes) 2024-09-18T00:18:50.418 INFO:teuthology.orchestra.run.smithi097.stdout:3077 writesame 0x7dd3ccd thru 0x7de1faa data_size 0x47(0xe2de bytes) 2024-09-18T00:18:50.422 INFO:teuthology.orchestra.run.smithi097.stdout:3078 write 0x5b14fb thru 0x5b8d54 (0x785a bytes) 2024-09-18T00:18:50.428 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:50.428 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:50.428 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-18T00:18:50.429 INFO:teuthology.orchestra.run.smithi097.stdout:3083 trunc from 0xc936821 to 0xc2a1f8b 2024-09-18T00:18:50.444 INFO:teuthology.orchestra.run.smithi097.stdout:3084 writesame 0x234d47b thru 0x23598d1 data_size 0x1(0xc457 bytes) 2024-09-18T00:18:50.453 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:50.453 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:50.473 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:50.474 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:50.478 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:18:50.478 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:18:50.488 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:50.488 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:50.502 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:18:50.507 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:50.502+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:50.595 INFO:teuthology.orchestra.run.smithi097.stdout:3085 punch from 0x9f9b82b to 0x9fa175e, (0x5f33 bytes) 2024-09-18T00:18:50.596 INFO:teuthology.orchestra.run.smithi097.stdout:3087 read 0x59be5d2 thru 0x59cdcee (0xf71d bytes) 2024-09-18T00:18:50.597 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:50.597 INFO:teuthology.orchestra.run.smithi097.stdout:3089 trunc from 0xc2a1f8b to 0xbf11c28 2024-09-18T00:18:50.607 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:50.602+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:50.609 INFO:teuthology.orchestra.run.smithi097.stdout:3090 read 0xbb1a1d8 thru 0xbb1ca2f (0x2858 bytes) 2024-09-18T00:18:50.610 INFO:teuthology.orchestra.run.smithi097.stdout:3091 writesame 0x6b80f75 thru 0x6b82d3a data_size 0x25(0x1dc6 bytes) 2024-09-18T00:18:50.612 INFO:teuthology.orchestra.run.smithi097.stdout:3092 punch from 0xd841eb to 0xd8d69e, (0x94b3 bytes) 2024-09-18T00:18:50.613 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:50.613 INFO:teuthology.orchestra.run.smithi097.stdout:3099 writesame 0x6fc5e51 thru 0x6fc881e data_size 0x1(0x29ce bytes) 2024-09-18T00:18:50.619 INFO:teuthology.orchestra.run.smithi097.stdout:3101 punch from 0x49dae4b to 0x49ea77b, (0xf930 bytes) 2024-09-18T00:18:50.621 INFO:teuthology.orchestra.run.smithi097.stdout:3102 trunc from 0xbf11c28 to 0xcf79a4e 2024-09-18T00:18:50.634 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:50.634 INFO:teuthology.orchestra.run.smithi097.stdout:3104 writesame 0x4e1243b thru 0x4e213f9 data_size 0x19(0xefbf bytes) 2024-09-18T00:18:50.657 INFO:teuthology.orchestra.run.smithi097.stdout:3106 write 0xe22cfe6 thru 0xe22da8c (0xaa7 bytes) 2024-09-18T00:18:50.659 INFO:teuthology.orchestra.run.smithi097.stdout:3107 write 0x7269600 thru 0x726d0b4 (0x3ab5 bytes) 2024-09-18T00:18:50.664 INFO:teuthology.orchestra.run.smithi097.stdout:3108 write 0x1aa15fd thru 0x1aab74a (0xa14e bytes) 2024-09-18T00:18:50.670 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:50.670 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:50.671 INFO:teuthology.orchestra.run.smithi097.stdout:3109 punch from 0x4fb0359 to 0x4fb9e98, (0x9b3f bytes) 2024-09-18T00:18:50.674 INFO:teuthology.orchestra.run.smithi097.stdout:3110 trunc from 0xe22da8d to 0xd0df2ea 2024-09-18T00:18:50.687 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:18:50.709 INFO:teuthology.orchestra.run.smithi097.stdout:3111 write 0xed1cbaa thru 0xed275d7 (0xaa2e bytes) 2024-09-18T00:18:50.710 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:50.706+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:50.715 INFO:teuthology.orchestra.run.smithi097.stdout:3112 read 0xb2828bd thru 0xb292071 (0xf7b5 bytes) 2024-09-18T00:18:50.716 INFO:teuthology.orchestra.run.smithi097.stdout:3113 write 0x2818bb5 thru 0x2824b57 (0xbfa3 bytes) 2024-09-18T00:18:50.724 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:50.724 INFO:teuthology.orchestra.run.smithi097.stdout:3115 writesame 0x698ff1b thru 0x699c3fe data_size 0x2(0xc4e4 bytes) 2024-09-18T00:18:50.810 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:50.802+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:50.811 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:50.811 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:50.826 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:18:50.827 INFO:teuthology.orchestra.run.smithi097.stdout:3120 writesame 0x3cda596 thru 0x3ce47e3 data_size 0x1e(0xa24e bytes) 2024-09-18T00:18:50.832 INFO:teuthology.orchestra.run.smithi097.stdout:3121 punch from 0x5e9b3bc to 0x5eaafcc, (0xfc10 bytes) 2024-09-18T00:18:50.832 INFO:teuthology.orchestra.run.smithi097.stdout:3122 read 0x941bccc thru 0x9428263 (0xc598 bytes) 2024-09-18T00:18:50.833 INFO:teuthology.orchestra.run.smithi097.stdout:3124 write 0x1ec85ea thru 0x1ecfa13 (0x742a bytes) 2024-09-18T00:18:50.855 INFO:teuthology.orchestra.run.smithi097.stdout:3125 trunc from 0xed275d8 to 0x3e4c58d 2024-09-18T00:18:50.870 INFO:teuthology.orchestra.run.smithi097.stdout:3126 read 0x3014cda thru 0x30197e8 (0x4b0f bytes) 2024-09-18T00:18:50.885 INFO:teuthology.orchestra.run.smithi097.stdout:3127 write 0x4dc98c0 thru 0x4dd36af (0x9df0 bytes) 2024-09-18T00:18:50.888 INFO:teuthology.orchestra.run.smithi097.stdout:3128 clone 13 order 20 su 262144 sc 13 2024-09-18T00:18:50.911 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:50.906+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:50.937 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op list-pgs 2024-09-18T00:18:50.951 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:50.951 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:50.965 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:18:51.013 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:51.010+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:51.113 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:51.106+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:51.129 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:51.129 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:51.143 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:18:51.214 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:51.210+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:51.267 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:51.267 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:51.281 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:18:51.315 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:51.310+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:51.401 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:51.402 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:51.416 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:18:51.416 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:51.410+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:51.517 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:51.514+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:51.535 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:51.536 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:51.550 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:18:51.618 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:51.614+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:51.671 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:51.672 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:51.687 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:18:51.724 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:18:51.731 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:51.726+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:51.820 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:51.814+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:51.859 INFO:teuthology.orchestra.run.smithi097.stdout:leaving image image_client.0-clone12 intact 2024-09-18T00:18:51.861 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:51.861 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:51.877 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:18:51.921 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:51.918+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:51.997 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:51.997 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:52.011 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:18:52.022 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:52.018+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:52.028 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-18T00:18:52.123 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:52.118+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:52.135 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:52.135 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:52.149 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:18:52.223 INFO:teuthology.orchestra.run.smithi097.stdout:3129 punch from 0x456a548 to 0x456b9de, (0x1496 bytes) 2024-09-18T00:18:52.224 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:52.218+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:52.252 INFO:teuthology.orchestra.run.smithi097.stdout:3130 write 0xd8fc668 thru 0xd9067f5 (0xa18e bytes) 2024-09-18T00:18:52.259 INFO:teuthology.orchestra.run.smithi097.stdout:3131 writesame 0xc7a737f thru 0xc7a9019 data_size 0x1(0x1c9b bytes) 2024-09-18T00:18:52.271 INFO:teuthology.orchestra.run.smithi097.stdout:3132 write 0xcfa2012 thru 0xcfa23cf (0x3be bytes) 2024-09-18T00:18:52.273 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:52.274 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:52.276 INFO:teuthology.orchestra.run.smithi097.stdout:3133 punch from 0x3761cc2 to 0x376f013, (0xd351 bytes) 2024-09-18T00:18:52.276 INFO:teuthology.orchestra.run.smithi097.stdout:3134 write 0x3b44be5 thru 0x3b48eef (0x430b bytes) 2024-09-18T00:18:52.282 INFO:teuthology.orchestra.run.smithi097.stdout:3135 read 0x6effa97 thru 0x6f05990 (0x5efa bytes) 2024-09-18T00:18:52.282 INFO:teuthology.orchestra.run.smithi097.stdout:3136 punch from 0xc7c1c94 to 0xc7ca650, (0x89bc bytes) 2024-09-18T00:18:52.290 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:18:52.300 INFO:teuthology.orchestra.run.smithi097.stdout:3137 trunc from 0xd9067f6 to 0x42b3b5c 2024-09-18T00:18:52.334 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:52.330+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:52.338 INFO:teuthology.orchestra.run.smithi097.stdout:3139 punch from 0x16ab0be to 0x16ad21e, (0x2160 bytes) 2024-09-18T00:18:52.338 INFO:teuthology.orchestra.run.smithi097.stdout:3141 read 0x9a4f94 thru 0x9aaeb7 (0x5f24 bytes) 2024-09-18T00:18:52.354 INFO:teuthology.orchestra.run.smithi097.stdout:3143 write 0x738c2e7 thru 0x739a415 (0xe12f bytes) 2024-09-18T00:18:52.381 INFO:teuthology.orchestra.run.smithi097.stdout:3144 write 0xa905a80 thru 0xa90e2e4 (0x8865 bytes) 2024-09-18T00:18:52.386 INFO:teuthology.orchestra.run.smithi097.stdout:3145 punch from 0x9da6b2a to 0x9da9cf7, (0x31cd bytes) 2024-09-18T00:18:52.403 INFO:teuthology.orchestra.run.smithi097.stdout:3147 write 0xc1a371c thru 0xc1acc18 (0x94fd bytes) 2024-09-18T00:18:52.407 INFO:teuthology.orchestra.run.smithi097.stdout:3148 write 0x8814576 thru 0x881a6a6 (0x6131 bytes) 2024-09-18T00:18:52.415 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:52.415 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:52.416 INFO:teuthology.orchestra.run.smithi097.stdout:3150 write 0xbf44733 thru 0xbf44ad0 (0x39e bytes) 2024-09-18T00:18:52.421 INFO:teuthology.orchestra.run.smithi097.stdout:3151 read 0x985ad8 thru 0x985f08 (0x431 bytes) 2024-09-18T00:18:52.422 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:52.426 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:52.418+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:52.430 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:18:52.441 INFO:teuthology.orchestra.run.smithi097.stdout:3153 write 0xed8ce33 thru 0xed8dc59 (0xe27 bytes) 2024-09-18T00:18:52.447 INFO:teuthology.orchestra.run.smithi097.stdout:3154 punch from 0xbbad8cb to 0xbbb646c, (0x8ba1 bytes) 2024-09-18T00:18:52.447 INFO:teuthology.orchestra.run.smithi097.stdout:3155 writesame 0x8696b45 thru 0x86972eb data_size 0x1(0x7a7 bytes) 2024-09-18T00:18:52.453 INFO:teuthology.orchestra.run.smithi097.stdout:3156 read 0x59ad63 thru 0x5a373b (0x89d9 bytes) 2024-09-18T00:18:52.456 INFO:teuthology.orchestra.run.smithi097.stdout:3157 read 0xa15c21f thru 0xa15ea7a (0x285c bytes) 2024-09-18T00:18:52.456 INFO:teuthology.orchestra.run.smithi097.stdout:3158 writesame 0x989c808 thru 0x98a2747 data_size 0x40(0x5f40 bytes) 2024-09-18T00:18:52.462 INFO:teuthology.orchestra.run.smithi097.stdout:3159 writesame 0x5ac1fb3 thru 0x5acf1ad data_size 0xd(0xd1fb bytes) 2024-09-18T00:18:52.474 INFO:teuthology.orchestra.run.smithi097.stdout:3160 read 0x9f7f0d1 thru 0x9f850ad (0x5fdd bytes) 2024-09-18T00:18:52.474 INFO:teuthology.orchestra.run.smithi097.stdout:3162 writesame 0xab9291d thru 0xab94dc9 data_size 0x1(0x24ad bytes) 2024-09-18T00:18:52.479 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:18:52.486 INFO:teuthology.orchestra.run.smithi097.stdout:3163 writesame 0xb539c83 thru 0xb546fce data_size 0x2(0xd34c bytes) 2024-09-18T00:18:52.505 INFO:teuthology.orchestra.run.smithi097.stdout:3164 clone 14 order 19 su 8192 sc 2 2024-09-18T00:18:52.526 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:52.522+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:52.596 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:52.596 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:52.601 INFO:teuthology.orchestra.run.smithi114.stdout:3.d 2024-09-18T00:18:52.601 INFO:teuthology.orchestra.run.smithi114.stdout:3.c 2024-09-18T00:18:52.602 INFO:teuthology.orchestra.run.smithi114.stdout:3.1f 2024-09-18T00:18:52.602 INFO:teuthology.orchestra.run.smithi114.stdout:3.1c 2024-09-18T00:18:52.602 INFO:teuthology.orchestra.run.smithi114.stdout:3.17 2024-09-18T00:18:52.602 INFO:teuthology.orchestra.run.smithi114.stdout:3.16 2024-09-18T00:18:52.602 INFO:teuthology.orchestra.run.smithi114.stdout:3.5 2024-09-18T00:18:52.602 INFO:teuthology.orchestra.run.smithi114.stdout:3.1d 2024-09-18T00:18:52.602 INFO:teuthology.orchestra.run.smithi114.stdout:2.3 2024-09-18T00:18:52.602 INFO:teuthology.orchestra.run.smithi114.stdout:3.2 2024-09-18T00:18:52.602 INFO:teuthology.orchestra.run.smithi114.stdout:2.2 2024-09-18T00:18:52.611 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:18:52.628 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:52.622+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:52.631 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:52.632 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:52.632 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:18:52.648 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:52.648 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:52.664 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:52.665 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:52.682 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:52.682 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:52.683 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-18T00:18:52.699 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:52.699 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:52.716 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:52.717 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:52.720 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:18:52.721 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:18:52.727 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is set 2024-09-18T00:18:52.728 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:52.722+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:52.781 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:52.781 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:52.781 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:52.781 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:52.781 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:52.797 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:18:52.842 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:52.838+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:52.910 INFO:teuthology.orchestra.run.smithi097.stdout:truncating image image_client.0-clone13 from 0xed8dc5a (overlap 0x42b3b5c) to 0x0 2024-09-18T00:18:52.929 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:52.926+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:52.984 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:52.984 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:52.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:52.985 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:52.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.985 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2228 3.15 3:aa2fc059:::rbd_data.112aa7ba4578.000000000000014a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:18:52.985 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:18:52.985 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.719661395999999, 2024-09-18T00:18:52.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.068966377999999995, 2024-09-18T00:18:52.985 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:52.985 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:52.985 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:52.985 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:52.985 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:52.985 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:52.985 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2228 2024-09-18T00:18:52.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.986 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:52.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:52.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:18:52.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:52.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:52.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:18:52.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:52.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:52.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263099+0000", 2024-09-18T00:18:52.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2359999999999999e-06 2024-09-18T00:18:52.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:52.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263115+0000", 2024-09-18T00:18:52.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5996999999999999e-05 2024-09-18T00:18:52.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:52.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263156+0000", 2024-09-18T00:18:52.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:18:52.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:52.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263183+0000", 2024-09-18T00:18:52.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7233000000000002e-05 2024-09-18T00:18:52.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:52.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263304+0000", 2024-09-18T00:18:52.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121209 2024-09-18T00:18:52.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:52.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263528+0000", 2024-09-18T00:18:52.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022401800000000001 2024-09-18T00:18:52.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:18:52.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.266458+0000", 2024-09-18T00:18:52.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029296980000000001 2024-09-18T00:18:52.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:52.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331628+0000", 2024-09-18T00:18:52.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.065170045999999995 2024-09-18T00:18:52.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:52.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331819+0000", 2024-09-18T00:18:52.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000190819 2024-09-18T00:18:52.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:52.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331971+0000", 2024-09-18T00:18:52.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151972 2024-09-18T00:18:52.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:52.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332063+0000", 2024-09-18T00:18:52.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2083999999999995e-05 2024-09-18T00:18:52.990 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:52.990 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:52.990 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:52.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.991 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2228 3.15 e36/32 ondisk, result = 0)", 2024-09-18T00:18:52.991 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:18:52.991 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.706608283999998, 2024-09-18T00:18:52.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055930148999999998, 2024-09-18T00:18:52.991 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:52.991 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:52.991 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:52.991 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:52.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:52.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:18:52.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:52.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:52.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:18:52.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:52.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:52.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276152+0000", 2024-09-18T00:18:52.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0200000000000001e-06 2024-09-18T00:18:52.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:52.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276162+0000", 2024-09-18T00:18:52.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0767e-05 2024-09-18T00:18:52.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:52.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276202+0000", 2024-09-18T00:18:52.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000003e-05 2024-09-18T00:18:52.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:52.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276226+0000", 2024-09-18T00:18:52.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.351e-05 2024-09-18T00:18:52.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:52.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331782+0000", 2024-09-18T00:18:52.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055556296999999998 2024-09-18T00:18:52.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:52.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331801+0000", 2024-09-18T00:18:52.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9357000000000001e-05 2024-09-18T00:18:52.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:52.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332080+0000", 2024-09-18T00:18:52.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027849800000000002 2024-09-18T00:18:52.994 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:52.994 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:52.994 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:52.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.995 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2854 3.f 3:f02b0bca:::rbd_data.112aa7ba4578.0000000000000135:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:18:52.995 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:18:52.995 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.112688905999999, 2024-09-18T00:18:52.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.082582802999999996, 2024-09-18T00:18:52.995 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:52.995 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:52.995 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:52.995 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:52.995 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:52.995 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:52.996 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2854 2024-09-18T00:18:52.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.996 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:52.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:52.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:18:52.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:52.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:52.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:18:52.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:52.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:52.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870071+0000", 2024-09-18T00:18:52.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.227e-06 2024-09-18T00:18:52.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:52.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870083+0000", 2024-09-18T00:18:52.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1909999999999999e-05 2024-09-18T00:18:52.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:52.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870114+0000", 2024-09-18T00:18:52.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0576e-05 2024-09-18T00:18:52.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:52.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870129+0000", 2024-09-18T00:18:52.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.485e-05 2024-09-18T00:18:52.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:52.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870256+0000", 2024-09-18T00:18:52.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012691300000000001 2024-09-18T00:18:52.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:52.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870493+0000", 2024-09-18T00:18:52.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023737399999999999 2024-09-18T00:18:52.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:52.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.874933+0000", 2024-09-18T00:18:52.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044398299999999996 2024-09-18T00:18:52.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:52.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952452+0000", 2024-09-18T00:18:52.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.077518918000000006 2024-09-18T00:18:52.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:52.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:52.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:52.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952486+0000", 2024-09-18T00:18:53.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4035999999999997e-05 2024-09-18T00:18:53.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:53.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952575+0000", 2024-09-18T00:18:53.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.954e-05 2024-09-18T00:18:53.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952652+0000", 2024-09-18T00:18:53.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6629000000000001e-05 2024-09-18T00:18:53.000 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.000 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.000 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.001 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2854 3.f e38/32 ondisk, result = 0)", 2024-09-18T00:18:53.001 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:18:53.001 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.095158090999998, 2024-09-18T00:18:53.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064868625999999999, 2024-09-18T00:18:53.001 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.001 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.001 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:53.001 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:18:53.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:18:53.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887602+0000", 2024-09-18T00:18:53.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.677e-06 2024-09-18T00:18:53.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887609+0000", 2024-09-18T00:18:53.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6729999999999992e-06 2024-09-18T00:18:53.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887637+0000", 2024-09-18T00:18:53.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8206000000000001e-05 2024-09-18T00:18:53.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887651+0000", 2024-09-18T00:18:53.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3387000000000001e-05 2024-09-18T00:18:53.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952422+0000", 2024-09-18T00:18:53.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064771366999999996 2024-09-18T00:18:53.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952441+0000", 2024-09-18T00:18:53.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8737e-05 2024-09-18T00:18:53.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952469+0000", 2024-09-18T00:18:53.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7579000000000001e-05 2024-09-18T00:18:53.004 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.004 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.004 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.005 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5633 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:53.005 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297194+0000", 2024-09-18T00:18:53.005 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.685564262, 2024-09-18T00:18:53.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046979053, 2024-09-18T00:18:53.005 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.005 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.005 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:53.005 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:53.005 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:53.005 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.005 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5633 2024-09-18T00:18:53.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297194+0000", 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297194+0000", 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297195+0000", 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2440000000000001e-06 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297201+0000", 2024-09-18T00:18:53.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5489999999999997e-06 2024-09-18T00:18:53.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297218+0000", 2024-09-18T00:18:53.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7652999999999999e-05 2024-09-18T00:18:53.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297245+0000", 2024-09-18T00:18:53.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7212999999999998e-05 2024-09-18T00:18:53.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343955+0000", 2024-09-18T00:18:53.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046709564000000002 2024-09-18T00:18:53.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344028+0000", 2024-09-18T00:18:53.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3370999999999999e-05 2024-09-18T00:18:53.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344173+0000", 2024-09-18T00:18:53.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014445899999999999 2024-09-18T00:18:53.008 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.009 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.009 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.009 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5634 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:53.009 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297263+0000", 2024-09-18T00:18:53.009 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.685494985999998, 2024-09-18T00:18:53.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047153893000000002, 2024-09-18T00:18:53.009 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.009 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.009 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:53.009 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:53.009 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:53.009 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.010 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5634 2024-09-18T00:18:53.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.010 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297263+0000", 2024-09-18T00:18:53.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297263+0000", 2024-09-18T00:18:53.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297264+0000", 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4169999999999999e-06 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297270+0000", 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4829999999999999e-06 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297287+0000", 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6993000000000001e-05 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297300+0000", 2024-09-18T00:18:53.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.324e-05 2024-09-18T00:18:53.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344221+0000", 2024-09-18T00:18:53.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046921030000000002 2024-09-18T00:18:53.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344310+0000", 2024-09-18T00:18:53.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8533000000000001e-05 2024-09-18T00:18:53.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344417+0000", 2024-09-18T00:18:53.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107197 2024-09-18T00:18:53.013 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.013 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.013 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.013 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5635 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:53.013 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297317+0000", 2024-09-18T00:18:53.013 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.685441235999999, 2024-09-18T00:18:53.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047292222000000002, 2024-09-18T00:18:53.013 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.013 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.013 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:53.013 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:53.013 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:53.013 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.014 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5635 2024-09-18T00:18:53.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.014 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297317+0000", 2024-09-18T00:18:53.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297317+0000", 2024-09-18T00:18:53.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297318+0000", 2024-09-18T00:18:53.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2869999999999999e-06 2024-09-18T00:18:53.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297323+0000", 2024-09-18T00:18:53.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1760000000000001e-06 2024-09-18T00:18:53.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297343+0000", 2024-09-18T00:18:53.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9473000000000001e-05 2024-09-18T00:18:53.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297356+0000", 2024-09-18T00:18:53.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3767e-05 2024-09-18T00:18:53.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344455+0000", 2024-09-18T00:18:53.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047098220000000003 2024-09-18T00:18:53.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344518+0000", 2024-09-18T00:18:53.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2856999999999998e-05 2024-09-18T00:18:53.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344609+0000", 2024-09-18T00:18:53.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1441999999999996e-05 2024-09-18T00:18:53.017 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.017 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.017 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.017 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5674 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 10682368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:53.017 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298168+0000", 2024-09-18T00:18:53.017 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.684590433, 2024-09-18T00:18:53.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047098783999999998, 2024-09-18T00:18:53.018 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.018 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.018 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:53.018 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:53.018 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:53.018 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.018 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5674 2024-09-18T00:18:53.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.018 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298168+0000", 2024-09-18T00:18:53.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298168+0000", 2024-09-18T00:18:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298169+0000", 2024-09-18T00:18:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1370000000000001e-06 2024-09-18T00:18:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298174+0000", 2024-09-18T00:18:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6500000000000001e-06 2024-09-18T00:18:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298192+0000", 2024-09-18T00:18:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7620000000000001e-05 2024-09-18T00:18:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298205+0000", 2024-09-18T00:18:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3363000000000001e-05 2024-09-18T00:18:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345013+0000", 2024-09-18T00:18:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046807418000000003 2024-09-18T00:18:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345125+0000", 2024-09-18T00:18:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111901 2024-09-18T00:18:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345266+0000", 2024-09-18T00:18:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000141695 2024-09-18T00:18:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.022 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5675 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:53.022 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298223+0000", 2024-09-18T00:18:53.022 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.684534540000001, 2024-09-18T00:18:53.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047324643999999999, 2024-09-18T00:18:53.022 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.022 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.022 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:53.022 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:53.022 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:53.022 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.022 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5675 2024-09-18T00:18:53.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.022 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298223+0000", 2024-09-18T00:18:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298223+0000", 2024-09-18T00:18:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298225+0000", 2024-09-18T00:18:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.13e-06 2024-09-18T00:18:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298230+0000", 2024-09-18T00:18:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1599999999999997e-06 2024-09-18T00:18:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298247+0000", 2024-09-18T00:18:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6813999999999999e-05 2024-09-18T00:18:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298260+0000", 2024-09-18T00:18:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3036e-05 2024-09-18T00:18:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345315+0000", 2024-09-18T00:18:53.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047055420000000001 2024-09-18T00:18:53.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345384+0000", 2024-09-18T00:18:53.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9375999999999994e-05 2024-09-18T00:18:53.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345548+0000", 2024-09-18T00:18:53.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016370800000000001 2024-09-18T00:18:53.025 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.026 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.026 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.026 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5676 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 12255232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:53.026 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:18:53.026 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.684482234000001, 2024-09-18T00:18:53.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047571816000000003, 2024-09-18T00:18:53.026 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.026 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.026 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:53.026 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:53.026 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:53.026 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.026 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5676 2024-09-18T00:18:53.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.027 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:18:53.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:18:53.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298277+0000", 2024-09-18T00:18:53.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.454e-06 2024-09-18T00:18:53.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298282+0000", 2024-09-18T00:18:53.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6299999999999997e-06 2024-09-18T00:18:53.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298299+0000", 2024-09-18T00:18:53.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6959000000000001e-05 2024-09-18T00:18:53.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298313+0000", 2024-09-18T00:18:53.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.376e-05 2024-09-18T00:18:53.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345610+0000", 2024-09-18T00:18:53.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047297128000000001 2024-09-18T00:18:53.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345708+0000", 2024-09-18T00:18:53.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8245999999999994e-05 2024-09-18T00:18:53.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345848+0000", 2024-09-18T00:18:53.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000139639 2024-09-18T00:18:53.030 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.030 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.030 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.030 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5677 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 12779520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:53.030 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:18:53.030 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.684428721, 2024-09-18T00:18:53.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047810679000000002, 2024-09-18T00:18:53.030 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.030 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.031 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:53.031 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:53.031 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:53.031 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.031 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5677 2024-09-18T00:18:53.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.031 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:18:53.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:18:53.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298330+0000", 2024-09-18T00:18:53.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.217e-06 2024-09-18T00:18:53.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298354+0000", 2024-09-18T00:18:53.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3753000000000001e-05 2024-09-18T00:18:53.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298388+0000", 2024-09-18T00:18:53.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4276999999999997e-05 2024-09-18T00:18:53.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298415+0000", 2024-09-18T00:18:53.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6403000000000001e-05 2024-09-18T00:18:53.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345896+0000", 2024-09-18T00:18:53.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047481007999999998 2024-09-18T00:18:53.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345992+0000", 2024-09-18T00:18:53.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6141e-05 2024-09-18T00:18:53.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346140+0000", 2024-09-18T00:18:53.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014788 2024-09-18T00:18:53.035 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.035 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.035 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.035 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5678 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:53.035 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:18:53.035 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.684322542, 2024-09-18T00:18:53.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047985466999999997, 2024-09-18T00:18:53.035 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.035 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.035 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:53.035 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:53.035 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:53.036 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.036 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5678 2024-09-18T00:18:53.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.036 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:18:53.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:18:53.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298437+0000", 2024-09-18T00:18:53.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1799999999999999e-06 2024-09-18T00:18:53.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298443+0000", 2024-09-18T00:18:53.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1140000000000002e-06 2024-09-18T00:18:53.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298460+0000", 2024-09-18T00:18:53.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7615999999999999e-05 2024-09-18T00:18:53.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298474+0000", 2024-09-18T00:18:53.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4017e-05 2024-09-18T00:18:53.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346180+0000", 2024-09-18T00:18:53.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047705801999999999 2024-09-18T00:18:53.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346274+0000", 2024-09-18T00:18:53.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3814999999999995e-05 2024-09-18T00:18:53.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346421+0000", 2024-09-18T00:18:53.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000146923 2024-09-18T00:18:53.039 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.039 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.039 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.039 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5679 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:53.039 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:18:53.039 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.684263498, 2024-09-18T00:18:53.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048190771, 2024-09-18T00:18:53.040 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.040 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.040 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:53.040 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:53.040 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:53.040 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.040 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5679 2024-09-18T00:18:53.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.040 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:18:53.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:18:53.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298496+0000", 2024-09-18T00:18:53.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4699999999999999e-06 2024-09-18T00:18:53.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298501+0000", 2024-09-18T00:18:53.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:18:53.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298519+0000", 2024-09-18T00:18:53.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7329e-05 2024-09-18T00:18:53.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298532+0000", 2024-09-18T00:18:53.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3083e-05 2024-09-18T00:18:53.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346462+0000", 2024-09-18T00:18:53.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047930449999999999 2024-09-18T00:18:53.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346556+0000", 2024-09-18T00:18:53.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3222999999999998e-05 2024-09-18T00:18:53.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346685+0000", 2024-09-18T00:18:53.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000129696 2024-09-18T00:18:53.044 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.044 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.044 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.044 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5680 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:53.044 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:18:53.044 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.684206089, 2024-09-18T00:18:53.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048535847, 2024-09-18T00:18:53.044 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.044 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.044 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:53.044 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:53.045 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:53.045 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.045 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5680 2024-09-18T00:18:53.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.045 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:18:53.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:18:53.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298553+0000", 2024-09-18T00:18:53.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0759999999999999e-06 2024-09-18T00:18:53.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298575+0000", 2024-09-18T00:18:53.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2019999999999999e-05 2024-09-18T00:18:53.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298593+0000", 2024-09-18T00:18:53.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7900000000000001e-05 2024-09-18T00:18:53.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298607+0000", 2024-09-18T00:18:53.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3844e-05 2024-09-18T00:18:53.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346791+0000", 2024-09-18T00:18:53.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048184449999999997 2024-09-18T00:18:53.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346921+0000", 2024-09-18T00:18:53.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012994699999999999 2024-09-18T00:18:53.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347088+0000", 2024-09-18T00:18:53.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016661000000000001 2024-09-18T00:18:53.048 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.048 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.048 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.048 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5681 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:53.048 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:18:53.048 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.684133678999999, 2024-09-18T00:18:53.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048743337999999997, 2024-09-18T00:18:53.048 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.049 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.049 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:53.049 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:53.049 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:53.049 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.049 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5681 2024-09-18T00:18:53.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.049 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:18:53.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:18:53.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298626+0000", 2024-09-18T00:18:53.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4899999999999999e-06 2024-09-18T00:18:53.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298631+0000", 2024-09-18T00:18:53.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0300000000000001e-06 2024-09-18T00:18:53.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298650+0000", 2024-09-18T00:18:53.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9499000000000001e-05 2024-09-18T00:18:53.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298664+0000", 2024-09-18T00:18:53.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3767e-05 2024-09-18T00:18:53.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347139+0000", 2024-09-18T00:18:53.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048475348000000001 2024-09-18T00:18:53.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347228+0000", 2024-09-18T00:18:53.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8843000000000003e-05 2024-09-18T00:18:53.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347368+0000", 2024-09-18T00:18:53.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013936100000000001 2024-09-18T00:18:53.052 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.052 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.053 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.053 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5682 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:53.053 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:18:53.053 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.684076864000001, 2024-09-18T00:18:53.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048916864999999997, 2024-09-18T00:18:53.053 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.053 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.053 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:53.053 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:53.053 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:53.053 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.053 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5682 2024-09-18T00:18:53.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.054 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:18:53.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:18:53.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298682+0000", 2024-09-18T00:18:53.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:18:53.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298687+0000", 2024-09-18T00:18:53.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9239999999999998e-06 2024-09-18T00:18:53.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298704+0000", 2024-09-18T00:18:53.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6569999999999999e-05 2024-09-18T00:18:53.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298731+0000", 2024-09-18T00:18:53.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6752999999999999e-05 2024-09-18T00:18:53.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347416+0000", 2024-09-18T00:18:53.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048685056999999997 2024-09-18T00:18:53.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347506+0000", 2024-09-18T00:18:53.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9936000000000001e-05 2024-09-18T00:18:53.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347598+0000", 2024-09-18T00:18:53.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2335000000000003e-05 2024-09-18T00:18:53.057 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.057 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.057 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.057 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15879 3.16 3:692b47b6:::rbd_object_map.112a13ec2d6c.000000000000000b:head [call] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:18:53.057 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:18:53.057 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.3279941839999996, 2024-09-18T00:18:53.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.011184018, 2024-09-18T00:18:53.057 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.057 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.057 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:53.057 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:53.058 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:53.058 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.058 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15879 2024-09-18T00:18:53.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.058 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:18:53.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:18:53.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654768+0000", 2024-09-18T00:18:53.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9129999999999996e-06 2024-09-18T00:18:53.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654786+0000", 2024-09-18T00:18:53.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8695999999999999e-05 2024-09-18T00:18:53.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654832+0000", 2024-09-18T00:18:53.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5735000000000002e-05 2024-09-18T00:18:53.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654860+0000", 2024-09-18T00:18:53.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7789000000000001e-05 2024-09-18T00:18:53.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655048+0000", 2024-09-18T00:18:53.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018781900000000001 2024-09-18T00:18:53.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:18:53.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655070+0000", 2024-09-18T00:18:53.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1916000000000002e-05 2024-09-18T00:18:53.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663335+0000", 2024-09-18T00:18:53.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0082651210000000003 2024-09-18T00:18:53.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:18:53.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663358+0000", 2024-09-18T00:18:53.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2986e-05 2024-09-18T00:18:53.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663739+0000", 2024-09-18T00:18:53.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0003814820000001 2024-09-18T00:18:53.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663920+0000", 2024-09-18T00:18:53.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018116900000000001 2024-09-18T00:18:53.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.665948+0000", 2024-09-18T00:18:53.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020273919999999998 2024-09-18T00:18:53.063 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.063 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.063 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.063 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15881 3.16 3:6cc14920:::rbd_header.112a3285400b:head [watch reconnect cookie 140685950184608 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:18:53.063 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:18:53.063 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.3278331889999997, 2024-09-18T00:18:53.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0138723599999999, 2024-09-18T00:18:53.064 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.064 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.064 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:53.064 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:53.064 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:53.064 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.064 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15881 2024-09-18T00:18:53.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.064 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:18:53.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:18:53.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654926+0000", 2024-09-18T00:18:53.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6470000000000001e-06 2024-09-18T00:18:53.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654935+0000", 2024-09-18T00:18:53.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8859999999999993e-06 2024-09-18T00:18:53.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654960+0000", 2024-09-18T00:18:53.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4689999999999999e-05 2024-09-18T00:18:53.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654976+0000", 2024-09-18T00:18:53.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6319000000000001e-05 2024-09-18T00:18:53.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655238+0000", 2024-09-18T00:18:53.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026145999999999998 2024-09-18T00:18:53.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:18:53.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655255+0000", 2024-09-18T00:18:53.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7249999999999999e-05 2024-09-18T00:18:53.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663416+0000", 2024-09-18T00:18:53.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081607959999999997 2024-09-18T00:18:53.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:18:53.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663436+0000", 2024-09-18T00:18:53.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0384e-05 2024-09-18T00:18:53.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666184+0000", 2024-09-18T00:18:53.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002748121 2024-09-18T00:18:53.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666452+0000", 2024-09-18T00:18:53.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026801700000000001 2024-09-18T00:18:53.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.668797+0000", 2024-09-18T00:18:53.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023447899999999998 2024-09-18T00:18:53.069 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.069 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.069 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.069 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17089 3.f 3:f7681386:::rbd_data.112a6efb7564.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:18:53.069 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:18:53.069 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.1029060929999996, 2024-09-18T00:18:53.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.12573889399999999, 2024-09-18T00:18:53.069 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.070 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.070 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:53.070 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:53.070 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:53.070 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.070 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17089 2024-09-18T00:18:53.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.070 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:18:53.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:18:53.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879854+0000", 2024-09-18T00:18:53.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:18:53.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879867+0000", 2024-09-18T00:18:53.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3103e-05 2024-09-18T00:18:53.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879898+0000", 2024-09-18T00:18:53.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0103e-05 2024-09-18T00:18:53.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879912+0000", 2024-09-18T00:18:53.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4636e-05 2024-09-18T00:18:53.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879992+0000", 2024-09-18T00:18:53.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9814000000000005e-05 2024-09-18T00:18:53.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.880101+0000", 2024-09-18T00:18:53.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109172 2024-09-18T00:18:53.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:53.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.887269+0000", 2024-09-18T00:18:53.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071679300000000003 2024-09-18T00:18:53.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:53.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005269+0000", 2024-09-18T00:18:53.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.117999621 2024-09-18T00:18:53.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:53.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005317+0000", 2024-09-18T00:18:53.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8279000000000003e-05 2024-09-18T00:18:53.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:53.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005464+0000", 2024-09-18T00:18:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014661200000000001 2024-09-18T00:18:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005591+0000", 2024-09-18T00:18:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000127107 2024-09-18T00:18:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17089 3.f e59/32 ondisk, result = 0)", 2024-09-18T00:18:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:18:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.0787992280000003, 2024-09-18T00:18:53.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101334403, 2024-09-18T00:18:53.076 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:53.076 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:53.076 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:53.076 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:53.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:53.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:18:53.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:53.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:18:53.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:53.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:53.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903960+0000", 2024-09-18T00:18:53.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.33e-06 2024-09-18T00:18:53.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:53.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903966+0000", 2024-09-18T00:18:53.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5929999999999996e-06 2024-09-18T00:18:53.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:53.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903986+0000", 2024-09-18T00:18:53.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.018e-05 2024-09-18T00:18:53.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903997+0000", 2024-09-18T00:18:53.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0988999999999999e-05 2024-09-18T00:18:53.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:53.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005171+0000", 2024-09-18T00:18:53.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101174267 2024-09-18T00:18:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005200+0000", 2024-09-18T00:18:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8540000000000001e-05 2024-09-18T00:18:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005293+0000", 2024-09-18T00:18:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504000000000005e-05 2024-09-18T00:18:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:53.080 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:53.081 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:18:53.081 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:53.078+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:53.139 INFO:tasks.thrashosds.thrasher:Moving pg 2.4 from osd.1 to osd.5 2024-09-18T00:18:53.140 INFO:tasks.thrashosds.thrasher:Transfer export file from ubuntu@smithi097.front.sepia.ceph.com to ubuntu@smithi114.front.sepia.ceph.com 2024-09-18T00:18:53.140 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /var/log/ceph 2024-09-18T00:18:53.151 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 777 /var/log/ceph 2024-09-18T00:18:53.165 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-09-18T00:18:53.197 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.ri1taFlOtB 2024-09-18T00:18:53.198 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /var/log/ceph/exp.2.4.1 /tmp/tmp.ri1taFlOtB 2024-09-18T00:18:53.205 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-18T00:18:53.210 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:53.206+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:53.211 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:53.211 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:53.211 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:53.211 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:53.216 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.ri1taFlOtB 2024-09-18T00:18:53.230 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:18:53.282 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:53.278+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:53.333 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.ri1taFlOtB is 7KB 2024-09-18T00:18:53.346 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.ri1taFlOtB 2024-09-18T00:18:53.362 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.4.1 2024-09-18T00:18:53.369 INFO:teuthology.orchestra.run.smithi097.stdout:3165 write 0x1d990cb thru 0x1da844a (0xf380 bytes) 2024-09-18T00:18:53.405 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:53.405 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:53.405 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:53.406 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:53.406 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:53.406 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:53.402+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:53.420 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:18:53.484 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:53.482+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:53.585 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:53.582+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:53.588 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:53.589 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:53.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:53.589 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:53.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.589 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2230 3.17 3:ef492888:::rbd_data.112aa7ba4578.000000000000014f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:18:53.589 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:53.589 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 33.318010041000001, 2024-09-18T00:18:53.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075065283999999996, 2024-09-18T00:18:53.589 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.589 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.589 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:53.589 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:53.590 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:53.590 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.590 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2230 2024-09-18T00:18:53.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.590 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:53.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:18:53.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266014+0000", 2024-09-18T00:18:53.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:18:53.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266029+0000", 2024-09-18T00:18:53.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4616e-05 2024-09-18T00:18:53.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266053+0000", 2024-09-18T00:18:53.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4369999999999999e-05 2024-09-18T00:18:53.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266067+0000", 2024-09-18T00:18:53.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4243999999999999e-05 2024-09-18T00:18:53.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266157+0000", 2024-09-18T00:18:53.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9909999999999998e-05 2024-09-18T00:18:53.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266327+0000", 2024-09-18T00:18:53.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016981200000000001 2024-09-18T00:18:53.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:53.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269180+0000", 2024-09-18T00:18:53.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028526559999999999 2024-09-18T00:18:53.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:53.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340955+0000", 2024-09-18T00:18:53.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.071774904 2024-09-18T00:18:53.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:53.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340974+0000", 2024-09-18T00:18:53.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8824000000000001e-05 2024-09-18T00:18:53.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:53.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341028+0000", 2024-09-18T00:18:53.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4456999999999997e-05 2024-09-18T00:18:53.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341076+0000", 2024-09-18T00:18:53.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8284000000000001e-05 2024-09-18T00:18:53.595 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.595 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.595 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.595 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2230 3.17 e36/23 ondisk, result = 0)", 2024-09-18T00:18:53.595 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:53.595 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 33.300666460000002, 2024-09-18T00:18:53.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057609645000000001, 2024-09-18T00:18:53.595 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.595 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.595 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:53.595 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:53.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:18:53.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283356+0000", 2024-09-18T00:18:53.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.44e-06 2024-09-18T00:18:53.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283362+0000", 2024-09-18T00:18:53.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2539999999999998e-06 2024-09-18T00:18:53.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283382+0000", 2024-09-18T00:18:53.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9896000000000001e-05 2024-09-18T00:18:53.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283398+0000", 2024-09-18T00:18:53.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5489999999999999e-05 2024-09-18T00:18:53.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340936+0000", 2024-09-18T00:18:53.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057538649999999997 2024-09-18T00:18:53.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340947+0000", 2024-09-18T00:18:53.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0981e-05 2024-09-18T00:18:53.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340964+0000", 2024-09-18T00:18:53.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6934000000000001e-05 2024-09-18T00:18:53.599 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.599 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.599 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.599 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2419 3.17 3:ed94b133:::rbd_data.112aa7ba4578.00000000000001b5:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:53.599 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:18:53.599 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.846544987000001, 2024-09-18T00:18:53.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.064761812000000002, 2024-09-18T00:18:53.599 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.600 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.600 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:53.600 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:53.600 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:53.600 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.600 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2419 2024-09-18T00:18:53.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.600 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:18:53.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:18:53.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737479+0000", 2024-09-18T00:18:53.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6089999999999999e-06 2024-09-18T00:18:53.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737487+0000", 2024-09-18T00:18:53.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7900000000000005e-06 2024-09-18T00:18:53.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737507+0000", 2024-09-18T00:18:53.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9936e-05 2024-09-18T00:18:53.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737518+0000", 2024-09-18T00:18:53.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1046999999999999e-05 2024-09-18T00:18:53.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737599+0000", 2024-09-18T00:18:53.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0697000000000003e-05 2024-09-18T00:18:53.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737697+0000", 2024-09-18T00:18:53.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8058000000000002e-05 2024-09-18T00:18:53.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:18:53.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.740445+0000", 2024-09-18T00:18:53.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027481279999999999 2024-09-18T00:18:53.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:53.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802023+0000", 2024-09-18T00:18:53.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.061577321999999997 2024-09-18T00:18:53.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:53.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802060+0000", 2024-09-18T00:18:53.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7586000000000002e-05 2024-09-18T00:18:53.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:53.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802158+0000", 2024-09-18T00:18:53.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7583000000000004e-05 2024-09-18T00:18:53.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802238+0000", 2024-09-18T00:18:53.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0055999999999994e-05 2024-09-18T00:18:53.605 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.605 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.605 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.605 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2419 3.17 e37/23 ondisk, result = 0)", 2024-09-18T00:18:53.605 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:18:53.605 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.834320040999998, 2024-09-18T00:18:53.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052342948, 2024-09-18T00:18:53.606 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.606 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.606 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:53.606 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:18:53.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:18:53.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749702+0000", 2024-09-18T00:18:53.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:18:53.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749710+0000", 2024-09-18T00:18:53.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7279999999999995e-06 2024-09-18T00:18:53.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749736+0000", 2024-09-18T00:18:53.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5797999999999999e-05 2024-09-18T00:18:53.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749751+0000", 2024-09-18T00:18:53.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5238999999999999e-05 2024-09-18T00:18:53.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801745+0000", 2024-09-18T00:18:53.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051994022000000001 2024-09-18T00:18:53.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801770+0000", 2024-09-18T00:18:53.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5103e-05 2024-09-18T00:18:53.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802044+0000", 2024-09-18T00:18:53.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027395799999999998 2024-09-18T00:18:53.609 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.609 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.609 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.609 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6503 3.1b 3:d9387a1f:::rbd_data.112a3a39fc45.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:53.609 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:18:53.609 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.019647071000001, 2024-09-18T00:18:53.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050510547000000003, 2024-09-18T00:18:53.610 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.610 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.610 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:53.610 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:53.610 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:53.610 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.610 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6503 2024-09-18T00:18:53.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.610 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:18:53.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:18:53.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564376+0000", 2024-09-18T00:18:53.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-06 2024-09-18T00:18:53.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564386+0000", 2024-09-18T00:18:53.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0312999999999999e-05 2024-09-18T00:18:53.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564408+0000", 2024-09-18T00:18:53.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2001000000000002e-05 2024-09-18T00:18:53.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564421+0000", 2024-09-18T00:18:53.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2271e-05 2024-09-18T00:18:53.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564491+0000", 2024-09-18T00:18:53.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0630000000000006e-05 2024-09-18T00:18:53.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564606+0000", 2024-09-18T00:18:53.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011449 2024-09-18T00:18:53.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:53.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.566743+0000", 2024-09-18T00:18:53.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021373830000000001 2024-09-18T00:18:53.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:53.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614738+0000", 2024-09-18T00:18:53.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047995359000000001 2024-09-18T00:18:53.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:53.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614759+0000", 2024-09-18T00:18:53.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1010999999999999e-05 2024-09-18T00:18:53.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:53.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614826+0000", 2024-09-18T00:18:53.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6654000000000003e-05 2024-09-18T00:18:53.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614884+0000", 2024-09-18T00:18:53.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8332999999999999e-05 2024-09-18T00:18:53.615 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.615 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.615 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.615 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6503 3.1b e42/23 ondisk, result = 0)", 2024-09-18T00:18:53.615 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:18:53.615 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.010387603000002, 2024-09-18T00:18:53.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041115407, 2024-09-18T00:18:53.615 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.615 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.616 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:53.616 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:18:53.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:18:53.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573635+0000", 2024-09-18T00:18:53.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5689999999999999e-06 2024-09-18T00:18:53.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573643+0000", 2024-09-18T00:18:53.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9699999999999999e-06 2024-09-18T00:18:53.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573668+0000", 2024-09-18T00:18:53.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5536000000000001e-05 2024-09-18T00:18:53.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573683+0000", 2024-09-18T00:18:53.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4025e-05 2024-09-18T00:18:53.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614717+0000", 2024-09-18T00:18:53.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041034517 2024-09-18T00:18:53.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614730+0000", 2024-09-18T00:18:53.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2945999999999999e-05 2024-09-18T00:18:53.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614749+0000", 2024-09-18T00:18:53.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8844000000000001e-05 2024-09-18T00:18:53.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.619 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.620 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6518 3.1 3:8061bf9f:::rbd_data.112a3a39fc45.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:53.620 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:18:53.620 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.933208796999999, 2024-09-18T00:18:53.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062303123000000002, 2024-09-18T00:18:53.620 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.620 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.620 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:53.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:53.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:53.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.620 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6518 2024-09-18T00:18:53.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.620 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:18:53.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:18:53.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650814+0000", 2024-09-18T00:18:53.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.629e-06 2024-09-18T00:18:53.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650823+0000", 2024-09-18T00:18:53.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1800000000000002e-06 2024-09-18T00:18:53.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650843+0000", 2024-09-18T00:18:53.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9836000000000001e-05 2024-09-18T00:18:53.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650854+0000", 2024-09-18T00:18:53.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1564e-05 2024-09-18T00:18:53.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650917+0000", 2024-09-18T00:18:53.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3076000000000004e-05 2024-09-18T00:18:53.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.651029+0000", 2024-09-18T00:18:53.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111154 2024-09-18T00:18:53.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:53.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.655531+0000", 2024-09-18T00:18:53.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045027629999999999 2024-09-18T00:18:53.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:53.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712924+0000", 2024-09-18T00:18:53.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057393006000000003 2024-09-18T00:18:53.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:53.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712988+0000", 2024-09-18T00:18:53.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000005e-05 2024-09-18T00:18:53.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:53.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713056+0000", 2024-09-18T00:18:53.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7904999999999994e-05 2024-09-18T00:18:53.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713115+0000", 2024-09-18T00:18:53.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9339999999999998e-05 2024-09-18T00:18:53.625 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.625 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.626 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.626 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6518 3.1 e42/23 ondisk, result = 0)", 2024-09-18T00:18:53.626 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:18:53.626 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.920783713999999, 2024-09-18T00:18:53.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049887304, 2024-09-18T00:18:53.626 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.626 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.626 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:53.626 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:18:53.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:18:53.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663239+0000", 2024-09-18T00:18:53.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:18:53.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663246+0000", 2024-09-18T00:18:53.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4189999999999996e-06 2024-09-18T00:18:53.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663265+0000", 2024-09-18T00:18:53.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8323000000000001e-05 2024-09-18T00:18:53.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663277+0000", 2024-09-18T00:18:53.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2065999999999999e-05 2024-09-18T00:18:53.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712965+0000", 2024-09-18T00:18:53.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049688317000000003 2024-09-18T00:18:53.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712977+0000", 2024-09-18T00:18:53.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.206e-05 2024-09-18T00:18:53.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713125+0000", 2024-09-18T00:18:53.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014749899999999999 2024-09-18T00:18:53.630 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.630 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.630 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.630 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:8831 3.12 3:49cac8cf:::rbd_data.112a9f4259df.0000000000000130:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T00:18:53.630 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:18:53.630 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 21.302923464999999, 2024-09-18T00:18:53.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104125047, 2024-09-18T00:18:53.630 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.630 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.630 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:53.630 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:53.631 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:53.631 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.631 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 8831 2024-09-18T00:18:53.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.631 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:18:53.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:18:53.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281099+0000", 2024-09-18T00:18:53.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.029e-06 2024-09-18T00:18:53.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281103+0000", 2024-09-18T00:18:53.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7759999999999997e-06 2024-09-18T00:18:53.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281114+0000", 2024-09-18T00:18:53.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1090000000000001e-05 2024-09-18T00:18:53.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281120+0000", 2024-09-18T00:18:53.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.604e-06 2024-09-18T00:18:53.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281164+0000", 2024-09-18T00:18:53.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3696999999999999e-05 2024-09-18T00:18:53.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281270+0000", 2024-09-18T00:18:53.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106041 2024-09-18T00:18:53.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:53.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.285596+0000", 2024-09-18T00:18:53.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043266659999999998 2024-09-18T00:18:53.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:53.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385028+0000", 2024-09-18T00:18:53.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.099431808999999996 2024-09-18T00:18:53.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:53.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385094+0000", 2024-09-18T00:18:53.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5740000000000004e-05 2024-09-18T00:18:53.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:53.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385166+0000", 2024-09-18T00:18:53.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1637000000000005e-05 2024-09-18T00:18:53.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385223+0000", 2024-09-18T00:18:53.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6957999999999999e-05 2024-09-18T00:18:53.636 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.636 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.636 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.636 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:8831 3.12 e48/32 ondisk, result = 0)", 2024-09-18T00:18:53.636 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:18:53.636 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 21.286409355, 2024-09-18T00:18:53.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087621558000000002, 2024-09-18T00:18:53.636 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.636 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.636 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:53.637 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:18:53.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:18:53.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297613+0000", 2024-09-18T00:18:53.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3400000000000001e-06 2024-09-18T00:18:53.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297620+0000", 2024-09-18T00:18:53.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5869999999999998e-06 2024-09-18T00:18:53.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297639+0000", 2024-09-18T00:18:53.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9888999999999999e-05 2024-09-18T00:18:53.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297661+0000", 2024-09-18T00:18:53.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1772000000000001e-05 2024-09-18T00:18:53.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385073+0000", 2024-09-18T00:18:53.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087411303999999995 2024-09-18T00:18:53.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385086+0000", 2024-09-18T00:18:53.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3040000000000001e-05 2024-09-18T00:18:53.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385233+0000", 2024-09-18T00:18:53.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014762600000000001 2024-09-18T00:18:53.640 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.640 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.640 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.640 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10099 3.6 3:62e7f20b:::rbd_data.112a9f4259df.0000000000000001:8 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:53.640 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.572938+0000", 2024-09-18T00:18:53.640 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.011082715000001, 2024-09-18T00:18:53.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026138630999999999, 2024-09-18T00:18:53.640 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.640 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.641 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:53.641 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:53.641 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:53.641 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.641 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10099 2024-09-18T00:18:53.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.641 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572938+0000", 2024-09-18T00:18:53.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572938+0000", 2024-09-18T00:18:53.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572939+0000", 2024-09-18T00:18:53.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7400000000000003e-07 2024-09-18T00:18:53.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572942+0000", 2024-09-18T00:18:53.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7159999999999999e-06 2024-09-18T00:18:53.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572949+0000", 2024-09-18T00:18:53.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7379999999999994e-06 2024-09-18T00:18:53.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572955+0000", 2024-09-18T00:18:53.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7860000000000002e-06 2024-09-18T00:18:53.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.598855+0000", 2024-09-18T00:18:53.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025900175000000001 2024-09-18T00:18:53.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.598939+0000", 2024-09-18T00:18:53.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3244000000000005e-05 2024-09-18T00:18:53.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599077+0000", 2024-09-18T00:18:53.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138398 2024-09-18T00:18:53.645 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.645 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.645 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.645 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10108 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:53.645 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.572963+0000", 2024-09-18T00:18:53.645 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.011058272, 2024-09-18T00:18:53.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026310093999999999, 2024-09-18T00:18:53.645 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.645 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.645 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:53.646 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:53.646 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:53.646 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.646 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10108 2024-09-18T00:18:53.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.646 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572963+0000", 2024-09-18T00:18:53.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572963+0000", 2024-09-18T00:18:53.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572963+0000", 2024-09-18T00:18:53.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7299999999999996e-07 2024-09-18T00:18:53.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572966+0000", 2024-09-18T00:18:53.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4389999999999999e-06 2024-09-18T00:18:53.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572973+0000", 2024-09-18T00:18:53.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6520000000000006e-06 2024-09-18T00:18:53.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572980+0000", 2024-09-18T00:18:53.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1959999999999996e-06 2024-09-18T00:18:53.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599154+0000", 2024-09-18T00:18:53.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026174096000000001 2024-09-18T00:18:53.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599196+0000", 2024-09-18T00:18:53.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1993000000000003e-05 2024-09-18T00:18:53.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599273+0000", 2024-09-18T00:18:53.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7144999999999994e-05 2024-09-18T00:18:53.649 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.649 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.649 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.649 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10109 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:53.650 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:18:53.650 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.011033794999999, 2024-09-18T00:18:53.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026513858000000001, 2024-09-18T00:18:53.650 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.650 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.650 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:53.650 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:53.650 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:53.650 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.650 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10109 2024-09-18T00:18:53.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.650 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:18:53.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:18:53.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572988+0000", 2024-09-18T00:18:53.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1099999999999996e-07 2024-09-18T00:18:53.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572990+0000", 2024-09-18T00:18:53.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3329999999999999e-06 2024-09-18T00:18:53.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572998+0000", 2024-09-18T00:18:53.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2039999999999997e-06 2024-09-18T00:18:53.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573012+0000", 2024-09-18T00:18:53.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3879e-05 2024-09-18T00:18:53.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599312+0000", 2024-09-18T00:18:53.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026300093 2024-09-18T00:18:53.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599380+0000", 2024-09-18T00:18:53.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7401000000000001e-05 2024-09-18T00:18:53.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599501+0000", 2024-09-18T00:18:53.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000121437 2024-09-18T00:18:53.654 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.654 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.654 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.654 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10110 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:53.654 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:18:53.654 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.011000335999999, 2024-09-18T00:18:53.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026746066999999998, 2024-09-18T00:18:53.654 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.654 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.654 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:53.655 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:53.655 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:53.655 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.655 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10110 2024-09-18T00:18:53.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.655 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:18:53.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:18:53.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:18:53.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2799999999999996e-07 2024-09-18T00:18:53.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573030+0000", 2024-09-18T00:18:53.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5469999999999997e-06 2024-09-18T00:18:53.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573039+0000", 2024-09-18T00:18:53.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6080000000000002e-06 2024-09-18T00:18:53.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573049+0000", 2024-09-18T00:18:53.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0759999999999992e-06 2024-09-18T00:18:53.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599539+0000", 2024-09-18T00:18:53.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026490952000000002 2024-09-18T00:18:53.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599621+0000", 2024-09-18T00:18:53.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1601999999999996e-05 2024-09-18T00:18:53.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599767+0000", 2024-09-18T00:18:53.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145654 2024-09-18T00:18:53.658 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.658 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.658 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.658 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10111 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:53.659 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:18:53.659 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.010962120999999, 2024-09-18T00:18:53.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027006651999999999, 2024-09-18T00:18:53.659 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.659 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.659 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:53.659 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:53.659 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:53.659 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.659 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10111 2024-09-18T00:18:53.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.659 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:18:53.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:18:53.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:18:53.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9500000000000002e-07 2024-09-18T00:18:53.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573062+0000", 2024-09-18T00:18:53.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6539999999999999e-06 2024-09-18T00:18:53.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573071+0000", 2024-09-18T00:18:53.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5380000000000002e-06 2024-09-18T00:18:53.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573077+0000", 2024-09-18T00:18:53.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999999e-06 2024-09-18T00:18:53.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599818+0000", 2024-09-18T00:18:53.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02674093 2024-09-18T00:18:53.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599923+0000", 2024-09-18T00:18:53.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010490200000000001 2024-09-18T00:18:53.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600066+0000", 2024-09-18T00:18:53.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014284500000000001 2024-09-18T00:18:53.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.663 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.663 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10112 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:53.663 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:18:53.663 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.010936327, 2024-09-18T00:18:53.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027148835, 2024-09-18T00:18:53.663 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.663 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.664 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:53.664 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:53.664 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:53.664 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.664 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10112 2024-09-18T00:18:53.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.664 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:18:53.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:18:53.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:18:53.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9500000000000003e-07 2024-09-18T00:18:53.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573088+0000", 2024-09-18T00:18:53.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9569999999999998e-06 2024-09-18T00:18:53.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573097+0000", 2024-09-18T00:18:53.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8599999999999999e-06 2024-09-18T00:18:53.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573104+0000", 2024-09-18T00:18:53.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5710000000000003e-06 2024-09-18T00:18:53.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600091+0000", 2024-09-18T00:18:53.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026987315000000001 2024-09-18T00:18:53.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600158+0000", 2024-09-18T00:18:53.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6727e-05 2024-09-18T00:18:53.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600234+0000", 2024-09-18T00:18:53.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5909999999999997e-05 2024-09-18T00:18:53.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.667 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.668 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10113 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:53.668 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:18:53.668 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.010892069000001, 2024-09-18T00:18:53.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027356020000000002, 2024-09-18T00:18:53.668 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.668 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.668 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:53.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:53.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:53.668 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.668 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10113 2024-09-18T00:18:53.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.668 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:18:53.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:18:53.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573130+0000", 2024-09-18T00:18:53.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1999999999999999e-07 2024-09-18T00:18:53.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573133+0000", 2024-09-18T00:18:53.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:18:53.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573141+0000", 2024-09-18T00:18:53.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5110000000000003e-06 2024-09-18T00:18:53.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573155+0000", 2024-09-18T00:18:53.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3730000000000001e-05 2024-09-18T00:18:53.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600275+0000", 2024-09-18T00:18:53.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027119621999999999 2024-09-18T00:18:53.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600349+0000", 2024-09-18T00:18:53.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4624999999999998e-05 2024-09-18T00:18:53.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600485+0000", 2024-09-18T00:18:53.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013570499999999999 2024-09-18T00:18:53.671 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.671 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.672 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.672 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10114 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:53.672 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:18:53.672 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.010856705999998, 2024-09-18T00:18:53.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027471173000000002, 2024-09-18T00:18:53.672 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.672 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.672 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:53.672 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:53.672 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:53.672 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.672 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10114 2024-09-18T00:18:53.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.673 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:18:53.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:18:53.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573165+0000", 2024-09-18T00:18:53.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.99e-07 2024-09-18T00:18:53.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573168+0000", 2024-09-18T00:18:53.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8339999999999999e-06 2024-09-18T00:18:53.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573176+0000", 2024-09-18T00:18:53.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0199999999999994e-06 2024-09-18T00:18:53.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573182+0000", 2024-09-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.455e-06 2024-09-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600521+0000", 2024-09-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027338989000000001 2024-09-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600577+0000", 2024-09-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5717000000000002e-05 2024-09-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600635+0000", 2024-09-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8559e-05 2024-09-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10115 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.010803829, 2024-09-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027643300999999999, 2024-09-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10115 2024-09-18T00:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573218+0000", 2024-09-18T00:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-18T00:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573221+0000", 2024-09-18T00:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3079999999999999e-06 2024-09-18T00:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573233+0000", 2024-09-18T00:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1544e-05 2024-09-18T00:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573240+0000", 2024-09-18T00:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4070000000000004e-06 2024-09-18T00:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600665+0000", 2024-09-18T00:18:53.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02742522 2024-09-18T00:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600745+0000", 2024-09-18T00:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0014999999999999e-05 2024-09-18T00:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600860+0000", 2024-09-18T00:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115127 2024-09-18T00:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11459 3.1b 3:db862ee8:::rbd_data.112ab1fa102a.0000000000000152:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.322375017999999, 2024-09-18T00:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.032246431999999998, 2024-09-18T00:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11459 2024-09-18T00:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.681 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:53.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261649+0000", 2024-09-18T00:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1200000000000002e-06 2024-09-18T00:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261665+0000", 2024-09-18T00:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5668999999999999e-05 2024-09-18T00:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261700+0000", 2024-09-18T00:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4925000000000002e-05 2024-09-18T00:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261723+0000", 2024-09-18T00:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3189000000000001e-05 2024-09-18T00:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261815+0000", 2024-09-18T00:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1662000000000004e-05 2024-09-18T00:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261904+0000", 2024-09-18T00:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9201000000000002e-05 2024-09-18T00:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.262955+0000", 2024-09-18T00:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00105156 2024-09-18T00:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293760+0000", 2024-09-18T00:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030804292 2024-09-18T00:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293773+0000", 2024-09-18T00:18:53.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3361e-05 2024-09-18T00:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293833+0000", 2024-09-18T00:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9626000000000001e-05 2024-09-18T00:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293892+0000", 2024-09-18T00:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9827000000000002e-05 2024-09-18T00:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:53.686 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:53.687 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout:3166 writesame 0x1d77dc1 thru 0x1d7e084 data_size 0x62(0x62c4 bytes) 2024-09-18T00:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout:3168 read 0x22d3138 thru 0x22e0164 (0xd02d bytes) 2024-09-18T00:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout:3169 read 0x66f2a8e thru 0x66fcffd (0xa570 bytes) 2024-09-18T00:18:53.688 INFO:teuthology.orchestra.run.smithi097.stdout:3170 trunc from 0xed8dc5a to 0xe87a467 2024-09-18T00:18:53.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:18:53.689 INFO:teuthology.orchestra.run.smithi097.stdout:3171 writesame 0xadc4eef thru 0xadd2e7b data_size 0x1(0xdf8d bytes) 2024-09-18T00:18:53.690 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:53.682+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:53.697 INFO:teuthology.orchestra.run.smithi097.stdout:3172 writesame 0x6494eb6 thru 0x64987d8 data_size 0x1(0x3923 bytes) 2024-09-18T00:18:53.759 INFO:teuthology.orchestra.run.smithi097.stdout:3174 writesame 0xea24a9d thru 0xea34743 data_size 0x1(0xfca7 bytes) 2024-09-18T00:18:53.759 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:53.759 INFO:teuthology.orchestra.run.smithi097.stdout:3176 write 0x7042d7f thru 0x704f572 (0xc7f4 bytes) 2024-09-18T00:18:53.773 INFO:teuthology.orchestra.run.smithi097.stdout:3177 write 0xc10b6ac thru 0xc11ab04 (0xf459 bytes) 2024-09-18T00:18:53.786 INFO:teuthology.orchestra.run.smithi097.stdout:3178 write 0xe3bde3b thru 0xe3c8594 (0xa75a bytes) 2024-09-18T00:18:53.788 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:53.782+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:53.795 INFO:teuthology.orchestra.run.smithi097.stdout:3179 write 0x60bc939 thru 0x60c5baa (0x9272 bytes) 2024-09-18T00:18:53.804 INFO:teuthology.orchestra.run.smithi097.stdout:3181 read 0xb0ff766 thru 0xb10b3bc (0xbc57 bytes) 2024-09-18T00:18:53.823 INFO:teuthology.orchestra.run.smithi097.stdout:3186 trunc from 0xea34744 to 0x64ee4f5 2024-09-18T00:18:53.834 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:53.835 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:53.835 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:53.835 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:53.842 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:53.842 INFO:teuthology.orchestra.run.smithi097.stdout:3188 write 0x25a3601 thru 0x25a6392 (0x2d92 bytes) 2024-09-18T00:18:53.852 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:18:53.871 INFO:teuthology.orchestra.run.smithi097.stdout:3189 read 0x2950e71 thru 0x2960268 (0xf3f8 bytes) 2024-09-18T00:18:53.875 INFO:teuthology.orchestra.run.smithi097.stdout:3190 read 0x51b66d thru 0x522aad (0x7441 bytes) 2024-09-18T00:18:53.878 INFO:teuthology.orchestra.run.smithi097.stdout:3191 writesame 0xfa0d3f thru 0xfae5a6 data_size 0x64(0xd868 bytes) 2024-09-18T00:18:53.889 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:53.882+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:53.939 INFO:teuthology.orchestra.run.smithi097.stdout:3192 write 0x4ce4ae thru 0x4d5879 (0x73cc bytes) 2024-09-18T00:18:53.977 INFO:teuthology.orchestra.run.smithi097.stdout:3193 read 0x4a5bae9 thru 0x4a6a18f (0xe6a7 bytes) 2024-09-18T00:18:53.983 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:53.984 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:53.984 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:53.984 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:53.984 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:53.990 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:53.986+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:53.994 INFO:teuthology.orchestra.run.smithi097.stdout:3195 writesame 0x995fca7 thru 0x9962c86 data_size 0x10(0x2fe0 bytes) 2024-09-18T00:18:54.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:18:54.015 INFO:teuthology.orchestra.run.smithi097.stdout:3196 trunc from 0x9962c87 to 0x35df19 2024-09-18T00:18:54.074 INFO:teuthology.orchestra.run.smithi097.stdout:3197 write 0x3ceb33d thru 0x3cf27ff (0x74c3 bytes) 2024-09-18T00:18:54.080 INFO:teuthology.orchestra.run.smithi097.stdout:3198 read 0x2a3ebbe thru 0x2a41ec5 (0x3308 bytes) 2024-09-18T00:18:54.080 INFO:teuthology.orchestra.run.smithi097.stdout:3199 writesame 0x232b45f thru 0x232e91e data_size 0x20(0x34c0 bytes) 2024-09-18T00:18:54.085 INFO:teuthology.orchestra.run.smithi097.stdout:3200 punch from 0x381e12a to 0x3822a7d, (0x4953 bytes) 2024-09-18T00:18:54.091 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:54.086+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:54.102 INFO:teuthology.orchestra.run.smithi097.stdout:3202 write 0xa5c11ee thru 0xa5c2cc9 (0x1adc bytes) 2024-09-18T00:18:54.108 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:54.108 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:54.108 INFO:teuthology.orchestra.run.smithi097.stdout:3206 read 0x3b9cec6 thru 0x3b9f82b (0x2966 bytes) 2024-09-18T00:18:54.108 INFO:teuthology.orchestra.run.smithi097.stdout:3207 write 0x51e5591 thru 0x51f1d93 (0xc803 bytes) 2024-09-18T00:18:54.114 INFO:teuthology.orchestra.run.smithi097.stdout:3208 read 0x76de733 thru 0x76df48f (0xd5d bytes) 2024-09-18T00:18:54.114 INFO:teuthology.orchestra.run.smithi097.stdout:3209 read 0x255dd46 thru 0x2562b10 (0x4dcb bytes) 2024-09-18T00:18:54.114 INFO:teuthology.orchestra.run.smithi097.stdout:3210 write 0x868d2fb thru 0x868e4dc (0x11e2 bytes) 2024-09-18T00:18:54.120 INFO:teuthology.orchestra.run.smithi097.stdout:3214 read 0x65a4af0 thru 0x65ae22d (0x973e bytes) 2024-09-18T00:18:54.120 INFO:teuthology.orchestra.run.smithi097.stdout:3216 writesame 0x8406294 thru 0x84107a3 data_size 0x4c(0xa510 bytes) 2024-09-18T00:18:54.127 INFO:teuthology.orchestra.run.smithi097.stdout:3217 punch from 0x8fe09ab to 0x8ff040f, (0xfa64 bytes) 2024-09-18T00:18:54.127 INFO:teuthology.orchestra.run.smithi097.stdout:3218 read 0x982e0aa thru 0x98327f2 (0x4749 bytes) 2024-09-18T00:18:54.141 INFO:teuthology.orchestra.run.smithi097.stdout:3219 trunc from 0xa5c2cca to 0x17aa62 2024-09-18T00:18:54.162 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:692 3.0 e29/23 3:03fab4ad:::rbd_data.112ae20746f5.0000000000000057:head v 29'1, mlcod=0'0)", 2024-09-18T00:18:54.162 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.918910740999998, 2024-09-18T00:18:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0074841819999999998, 2024-09-18T00:18:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238837+0000", 2024-09-18T00:18:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238838+0000", 2024-09-18T00:18:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.187e-06 2024-09-18T00:18:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238874+0000", 2024-09-18T00:18:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5380999999999999e-05 2024-09-18T00:18:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238893+0000", 2024-09-18T00:18:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9539999999999999e-05 2024-09-18T00:18:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238906+0000", 2024-09-18T00:18:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3011e-05 2024-09-18T00:18:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.238982+0000", 2024-09-18T00:18:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6018000000000003e-05 2024-09-18T00:18:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.239006+0000", 2024-09-18T00:18:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3683e-05 2024-09-18T00:18:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246227+0000", 2024-09-18T00:18:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072210989999999999 2024-09-18T00:18:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.246321+0000", 2024-09-18T00:18:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4263000000000002e-05 2024-09-18T00:18:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.166 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:5722 3.4 e41/23 3:20381d86:::rbd_data.112a3a39fc45.000000000000001d:head v 41'107, mlcod=40'105)", 2024-09-18T00:18:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:18:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.822252071000001, 2024-09-18T00:18:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01185172, 2024-09-18T00:18:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:18:54.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:18:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335497+0000", 2024-09-18T00:18:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.505e-06 2024-09-18T00:18:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335945+0000", 2024-09-18T00:18:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000447687 2024-09-18T00:18:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335961+0000", 2024-09-18T00:18:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6430999999999999e-05 2024-09-18T00:18:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335969+0000", 2024-09-18T00:18:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:18:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336030+0000", 2024-09-18T00:18:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1255999999999998e-05 2024-09-18T00:18:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336061+0000", 2024-09-18T00:18:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0579e-05 2024-09-18T00:18:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347296+0000", 2024-09-18T00:18:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011235663 2024-09-18T00:18:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347347+0000", 2024-09-18T00:18:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0914000000000001e-05 2024-09-18T00:18:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11457 3.0 3:0214e7b3:::rbd_data.112ab1fa102a.0000000000000150:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:18:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.897886133, 2024-09-18T00:18:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028499165, 2024-09-18T00:18:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.171 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11457 2024-09-18T00:18:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:18:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:18:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259865+0000", 2024-09-18T00:18:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7230000000000002e-06 2024-09-18T00:18:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259883+0000", 2024-09-18T00:18:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8067000000000002e-05 2024-09-18T00:18:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259924+0000", 2024-09-18T00:18:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0550999999999999e-05 2024-09-18T00:18:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259943+0000", 2024-09-18T00:18:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:18:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260054+0000", 2024-09-18T00:18:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011075 2024-09-18T00:18:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260138+0000", 2024-09-18T00:18:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4549000000000007e-05 2024-09-18T00:18:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.287926+0000", 2024-09-18T00:18:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027787642000000001 2024-09-18T00:18:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288090+0000", 2024-09-18T00:18:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016402499999999999 2024-09-18T00:18:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288361+0000", 2024-09-18T00:18:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027097800000000001 2024-09-18T00:18:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18661 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:18:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.2231182679999999, 2024-09-18T00:18:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020008037999999999, 2024-09-18T00:18:54.176 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18661 2024-09-18T00:18:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:18:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:18:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934633+0000", 2024-09-18T00:18:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8e-06 2024-09-18T00:18:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935511+0000", 2024-09-18T00:18:54.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087785700000000001 2024-09-18T00:18:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935561+0000", 2024-09-18T00:18:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9472000000000003e-05 2024-09-18T00:18:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935586+0000", 2024-09-18T00:18:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5616e-05 2024-09-18T00:18:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935838+0000", 2024-09-18T00:18:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025202699999999999 2024-09-18T00:18:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.936023+0000", 2024-09-18T00:18:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018480300000000001 2024-09-18T00:18:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953545+0000", 2024-09-18T00:18:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017522157 2024-09-18T00:18:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953723+0000", 2024-09-18T00:18:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017753800000000001 2024-09-18T00:18:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954637+0000", 2024-09-18T00:18:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091476800000000005 2024-09-18T00:18:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.181 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18662 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:18:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.2221133310000001, 2024-09-18T00:18:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019388843999999999, 2024-09-18T00:18:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18662 2024-09-18T00:18:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.182 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:18:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:18:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935636+0000", 2024-09-18T00:18:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.663e-06 2024-09-18T00:18:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935646+0000", 2024-09-18T00:18:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000006e-06 2024-09-18T00:18:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935672+0000", 2024-09-18T00:18:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5868999999999999e-05 2024-09-18T00:18:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935692+0000", 2024-09-18T00:18:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0604000000000001e-05 2024-09-18T00:18:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950738+0000", 2024-09-18T00:18:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015045868 2024-09-18T00:18:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950992+0000", 2024-09-18T00:18:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025368400000000001 2024-09-18T00:18:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954674+0000", 2024-09-18T00:18:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003682154 2024-09-18T00:18:54.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954806+0000", 2024-09-18T00:18:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013252000000000001 2024-09-18T00:18:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.955023+0000", 2024-09-18T00:18:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021681200000000001 2024-09-18T00:18:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19382 3.4 3:22f09d3a:::rbd_data.112a7fe29ab4.000000000000015a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:18:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.53570328099999998, 2024-09-18T00:18:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075119302999999998, 2024-09-18T00:18:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19382 2024-09-18T00:18:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:18:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:18:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622045+0000", 2024-09-18T00:18:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0020000000000001e-06 2024-09-18T00:18:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622053+0000", 2024-09-18T00:18:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2679999999999997e-06 2024-09-18T00:18:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622062+0000", 2024-09-18T00:18:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3749999999999992e-06 2024-09-18T00:18:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622068+0000", 2024-09-18T00:18:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5729999999999998e-06 2024-09-18T00:18:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622118+0000", 2024-09-18T00:18:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0696999999999999e-05 2024-09-18T00:18:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622222+0000", 2024-09-18T00:18:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103286 2024-09-18T00:18:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697061+0000", 2024-09-18T00:18:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074839141999999997 2024-09-18T00:18:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697102+0000", 2024-09-18T00:18:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1513e-05 2024-09-18T00:18:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697164+0000", 2024-09-18T00:18:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1446999999999996e-05 2024-09-18T00:18:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19896 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.191 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:18:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.21631914299999999, 2024-09-18T00:18:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097478370000000005, 2024-09-18T00:18:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19896 2024-09-18T00:18:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:18:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:18:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:18:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6000000000000002e-07 2024-09-18T00:18:54.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941434+0000", 2024-09-18T00:18:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4750000000000004e-06 2024-09-18T00:18:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941452+0000", 2024-09-18T00:18:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8603000000000001e-05 2024-09-18T00:18:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941461+0000", 2024-09-18T00:18:54.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5199999999999997e-06 2024-09-18T00:18:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.947391+0000", 2024-09-18T00:18:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059302399999999998 2024-09-18T00:18:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.947479+0000", 2024-09-18T00:18:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8002000000000002e-05 2024-09-18T00:18:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.951176+0000", 2024-09-18T00:18:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036971370000000001 2024-09-18T00:18:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19898 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941520+0000", 2024-09-18T00:18:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.216228067, 2024-09-18T00:18:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0065354799999999998, 2024-09-18T00:18:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.196 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19898 2024-09-18T00:18:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941520+0000", 2024-09-18T00:18:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941520+0000", 2024-09-18T00:18:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941521+0000", 2024-09-18T00:18:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.04e-06 2024-09-18T00:18:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941524+0000", 2024-09-18T00:18:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5420000000000001e-06 2024-09-18T00:18:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941534+0000", 2024-09-18T00:18:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3139999999999993e-06 2024-09-18T00:18:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941540+0000", 2024-09-18T00:18:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6540000000000002e-06 2024-09-18T00:18:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.947844+0000", 2024-09-18T00:18:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.006303889 2024-09-18T00:18:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.947944+0000", 2024-09-18T00:18:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010038 2024-09-18T00:18:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.948055+0000", 2024-09-18T00:18:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000110661 2024-09-18T00:18:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.200 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19899 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941549+0000", 2024-09-18T00:18:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.216198209, 2024-09-18T00:18:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067741060000000002, 2024-09-18T00:18:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19899 2024-09-18T00:18:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941549+0000", 2024-09-18T00:18:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941549+0000", 2024-09-18T00:18:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941550+0000", 2024-09-18T00:18:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8299999999999996e-07 2024-09-18T00:18:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941553+0000", 2024-09-18T00:18:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8619999999999999e-06 2024-09-18T00:18:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941562+0000", 2024-09-18T00:18:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.054e-06 2024-09-18T00:18:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941569+0000", 2024-09-18T00:18:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9140000000000002e-06 2024-09-18T00:18:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.948082+0000", 2024-09-18T00:18:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0065129560000000003 2024-09-18T00:18:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.948168+0000", 2024-09-18T00:18:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5589999999999999e-05 2024-09-18T00:18:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.948324+0000", 2024-09-18T00:18:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015604700000000001 2024-09-18T00:18:54.204 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19900 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941590+0000", 2024-09-18T00:18:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.216157765, 2024-09-18T00:18:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0069496820000000004, 2024-09-18T00:18:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.205 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19900 2024-09-18T00:18:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941590+0000", 2024-09-18T00:18:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941590+0000", 2024-09-18T00:18:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941591+0000", 2024-09-18T00:18:54.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5799999999999999e-07 2024-09-18T00:18:54.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941594+0000", 2024-09-18T00:18:54.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.179e-06 2024-09-18T00:18:54.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.207 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.207 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941603+0000", 2024-09-18T00:18:54.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1169999999999993e-06 2024-09-18T00:18:54.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941610+0000", 2024-09-18T00:18:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6209999999999998e-06 2024-09-18T00:18:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.948344+0000", 2024-09-18T00:18:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0067345629999999998 2024-09-18T00:18:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.948416+0000", 2024-09-18T00:18:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1674000000000005e-05 2024-09-18T00:18:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.948540+0000", 2024-09-18T00:18:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012386999999999999 2024-09-18T00:18:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19901 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941619+0000", 2024-09-18T00:18:54.209 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.21612852299999999, 2024-09-18T00:18:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072409249999999996, 2024-09-18T00:18:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19901 2024-09-18T00:18:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941619+0000", 2024-09-18T00:18:54.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941619+0000", 2024-09-18T00:18:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941620+0000", 2024-09-18T00:18:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7299999999999996e-07 2024-09-18T00:18:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941623+0000", 2024-09-18T00:18:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7750000000000001e-06 2024-09-18T00:18:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941631+0000", 2024-09-18T00:18:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8959999999999992e-06 2024-09-18T00:18:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941638+0000", 2024-09-18T00:18:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8310000000000003e-06 2024-09-18T00:18:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.948571+0000", 2024-09-18T00:18:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0069326020000000004 2024-09-18T00:18:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.948666+0000", 2024-09-18T00:18:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4963999999999994e-05 2024-09-18T00:18:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.948860+0000", 2024-09-18T00:18:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000194284 2024-09-18T00:18:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19902 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941648+0000", 2024-09-18T00:18:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.21609994099999999, 2024-09-18T00:18:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0075142890000000004, 2024-09-18T00:18:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.214 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19902 2024-09-18T00:18:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941648+0000", 2024-09-18T00:18:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941648+0000", 2024-09-18T00:18:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941648+0000", 2024-09-18T00:18:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7299999999999996e-07 2024-09-18T00:18:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941651+0000", 2024-09-18T00:18:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9500000000000001e-06 2024-09-18T00:18:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941661+0000", 2024-09-18T00:18:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6500000000000008e-06 2024-09-18T00:18:54.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941668+0000", 2024-09-18T00:18:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1770000000000001e-06 2024-09-18T00:18:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.948891+0000", 2024-09-18T00:18:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0072229520000000004 2024-09-18T00:18:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.948983+0000", 2024-09-18T00:18:54.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2140999999999998e-05 2024-09-18T00:18:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.949162+0000", 2024-09-18T00:18:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000178846 2024-09-18T00:18:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19903 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 401408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941676+0000", 2024-09-18T00:18:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.21607211700000001, 2024-09-18T00:18:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0078017750000000004, 2024-09-18T00:18:54.218 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19903 2024-09-18T00:18:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941676+0000", 2024-09-18T00:18:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941676+0000", 2024-09-18T00:18:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941676+0000", 2024-09-18T00:18:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0999999999999999e-07 2024-09-18T00:18:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941678+0000", 2024-09-18T00:18:54.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9630000000000001e-06 2024-09-18T00:18:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941683+0000", 2024-09-18T00:18:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1150000000000002e-06 2024-09-18T00:18:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941687+0000", 2024-09-18T00:18:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.693e-06 2024-09-18T00:18:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.949193+0000", 2024-09-18T00:18:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0075059840000000003 2024-09-18T00:18:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.949284+0000", 2024-09-18T00:18:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1036e-05 2024-09-18T00:18:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.949477+0000", 2024-09-18T00:18:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019357399999999999 2024-09-18T00:18:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19904 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941692+0000", 2024-09-18T00:18:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.21605611, 2024-09-18T00:18:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0081099009999999992, 2024-09-18T00:18:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19904 2024-09-18T00:18:54.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941692+0000", 2024-09-18T00:18:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941692+0000", 2024-09-18T00:18:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941692+0000", 2024-09-18T00:18:54.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2000000000000001e-07 2024-09-18T00:18:54.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941693+0000", 2024-09-18T00:18:54.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4699999999999999e-06 2024-09-18T00:18:54.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941698+0000", 2024-09-18T00:18:54.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7099999999999998e-06 2024-09-18T00:18:54.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941706+0000", 2024-09-18T00:18:54.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8380000000000002e-06 2024-09-18T00:18:54.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.949510+0000", 2024-09-18T00:18:54.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0078037089999999998 2024-09-18T00:18:54.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.949614+0000", 2024-09-18T00:18:54.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104849 2024-09-18T00:18:54.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.949801+0000", 2024-09-18T00:18:54.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018700499999999999 2024-09-18T00:18:54.227 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.227 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.227 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.227 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19905 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.227 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941712+0000", 2024-09-18T00:18:54.227 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.216035488, 2024-09-18T00:18:54.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083853710000000008, 2024-09-18T00:18:54.227 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.227 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.227 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.227 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.228 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.228 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.228 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19905 2024-09-18T00:18:54.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.228 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941712+0000", 2024-09-18T00:18:54.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941712+0000", 2024-09-18T00:18:54.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941713+0000", 2024-09-18T00:18:54.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0600000000000001e-07 2024-09-18T00:18:54.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941714+0000", 2024-09-18T00:18:54.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.564e-06 2024-09-18T00:18:54.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941724+0000", 2024-09-18T00:18:54.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7329999999999998e-06 2024-09-18T00:18:54.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941730+0000", 2024-09-18T00:18:54.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7899999999999996e-06 2024-09-18T00:18:54.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.949832+0000", 2024-09-18T00:18:54.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0081021540000000003 2024-09-18T00:18:54.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.949923+0000", 2024-09-18T00:18:54.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0847999999999995e-05 2024-09-18T00:18:54.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950098+0000", 2024-09-18T00:18:54.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000174976 2024-09-18T00:18:54.231 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.231 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.231 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.231 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19906 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.231 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941737+0000", 2024-09-18T00:18:54.232 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.216011067, 2024-09-18T00:18:54.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085440659999999995, 2024-09-18T00:18:54.232 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.232 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.232 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.232 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.232 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.232 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.232 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19906 2024-09-18T00:18:54.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.232 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941737+0000", 2024-09-18T00:18:54.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941737+0000", 2024-09-18T00:18:54.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941737+0000", 2024-09-18T00:18:54.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0299999999999999e-07 2024-09-18T00:18:54.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941740+0000", 2024-09-18T00:18:54.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6350000000000002e-06 2024-09-18T00:18:54.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941749+0000", 2024-09-18T00:18:54.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1260000000000004e-06 2024-09-18T00:18:54.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941756+0000", 2024-09-18T00:18:54.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7129999999999999e-06 2024-09-18T00:18:54.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950131+0000", 2024-09-18T00:18:54.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0083757950000000001 2024-09-18T00:18:54.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950184+0000", 2024-09-18T00:18:54.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2692000000000003e-05 2024-09-18T00:18:54.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950281+0000", 2024-09-18T00:18:54.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6601999999999994e-05 2024-09-18T00:18:54.235 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.236 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.236 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.236 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19907 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.236 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941771+0000", 2024-09-18T00:18:54.236 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.21597648899999999, 2024-09-18T00:18:54.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0087132449999999997, 2024-09-18T00:18:54.236 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.236 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.236 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.236 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.236 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.236 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.237 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19907 2024-09-18T00:18:54.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.237 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941771+0000", 2024-09-18T00:18:54.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941771+0000", 2024-09-18T00:18:54.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941772+0000", 2024-09-18T00:18:54.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9999999999999998e-07 2024-09-18T00:18:54.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941781+0000", 2024-09-18T00:18:54.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0210000000000005e-06 2024-09-18T00:18:54.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941792+0000", 2024-09-18T00:18:54.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0925e-05 2024-09-18T00:18:54.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941798+0000", 2024-09-18T00:18:54.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6749999999999996e-06 2024-09-18T00:18:54.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950307+0000", 2024-09-18T00:18:54.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085082860000000003 2024-09-18T00:18:54.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950390+0000", 2024-09-18T00:18:54.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3542999999999996e-05 2024-09-18T00:18:54.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950484+0000", 2024-09-18T00:18:54.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4295000000000004e-05 2024-09-18T00:18:54.240 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.240 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.240 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.240 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19908 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.240 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941809+0000", 2024-09-18T00:18:54.240 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.21593853900000001, 2024-09-18T00:18:54.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0088554250000000001, 2024-09-18T00:18:54.240 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.240 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.241 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.241 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.241 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.241 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.241 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19908 2024-09-18T00:18:54.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.241 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941809+0000", 2024-09-18T00:18:54.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941809+0000", 2024-09-18T00:18:54.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941810+0000", 2024-09-18T00:18:54.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2600000000000002e-07 2024-09-18T00:18:54.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941823+0000", 2024-09-18T00:18:54.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2784e-05 2024-09-18T00:18:54.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941832+0000", 2024-09-18T00:18:54.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6229999999999992e-06 2024-09-18T00:18:54.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941839+0000", 2024-09-18T00:18:54.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1520000000000003e-06 2024-09-18T00:18:54.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950515+0000", 2024-09-18T00:18:54.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0086754880000000003 2024-09-18T00:18:54.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950581+0000", 2024-09-18T00:18:54.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6398999999999999e-05 2024-09-18T00:18:54.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950665+0000", 2024-09-18T00:18:54.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3252999999999997e-05 2024-09-18T00:18:54.244 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.244 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.244 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.245 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19909 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.245 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941849+0000", 2024-09-18T00:18:54.245 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.21589825000000001, 2024-09-18T00:18:54.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0090299730000000002, 2024-09-18T00:18:54.245 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.245 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.245 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.245 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.245 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.245 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.245 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19909 2024-09-18T00:18:54.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.245 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941849+0000", 2024-09-18T00:18:54.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941849+0000", 2024-09-18T00:18:54.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941850+0000", 2024-09-18T00:18:54.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4899999999999995e-07 2024-09-18T00:18:54.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941853+0000", 2024-09-18T00:18:54.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.371e-06 2024-09-18T00:18:54.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941862+0000", 2024-09-18T00:18:54.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7749999999999998e-06 2024-09-18T00:18:54.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941869+0000", 2024-09-18T00:18:54.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6320000000000002e-06 2024-09-18T00:18:54.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950702+0000", 2024-09-18T00:18:54.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0088332930000000007 2024-09-18T00:18:54.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950778+0000", 2024-09-18T00:18:54.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5983999999999996e-05 2024-09-18T00:18:54.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950879+0000", 2024-09-18T00:18:54.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101269 2024-09-18T00:18:54.249 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.249 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.249 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.249 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19910 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.249 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941878+0000", 2024-09-18T00:18:54.249 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.21586941400000001, 2024-09-18T00:18:54.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092861479999999993, 2024-09-18T00:18:54.249 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.249 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.249 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.250 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.250 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.250 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.250 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19910 2024-09-18T00:18:54.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.250 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941878+0000", 2024-09-18T00:18:54.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941878+0000", 2024-09-18T00:18:54.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941879+0000", 2024-09-18T00:18:54.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6199999999999998e-07 2024-09-18T00:18:54.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941881+0000", 2024-09-18T00:18:54.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6390000000000001e-06 2024-09-18T00:18:54.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941890+0000", 2024-09-18T00:18:54.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4999999999999999e-06 2024-09-18T00:18:54.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941896+0000", 2024-09-18T00:18:54.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1299999999999998e-06 2024-09-18T00:18:54.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950898+0000", 2024-09-18T00:18:54.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0090014499999999994 2024-09-18T00:18:54.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950984+0000", 2024-09-18T00:18:54.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6933000000000003e-05 2024-09-18T00:18:54.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.951164+0000", 2024-09-18T00:18:54.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000179934 2024-09-18T00:18:54.253 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.253 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.253 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.253 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.253 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.253 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:54.255 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:18:54.256 INFO:teuthology.orchestra.run.smithi097.stdout:3221 write 0x57983a2 thru 0x579e694 (0x62f3 bytes) 2024-09-18T00:18:54.256 INFO:teuthology.orchestra.run.smithi097.stdout:3222 writesame 0xd2f7ef8 thru 0xd2fd571 data_size 0x1(0x567a bytes) 2024-09-18T00:18:54.256 INFO:teuthology.orchestra.run.smithi097.stdout:3223 read 0xa299fb2 thru 0xa2a692f (0xc97e bytes) 2024-09-18T00:18:54.256 INFO:teuthology.orchestra.run.smithi097.stdout:3225 write 0x9b20f93 thru 0x9b2524c (0x42ba bytes) 2024-09-18T00:18:54.256 INFO:teuthology.orchestra.run.smithi097.stdout:3226 punch from 0x8a95baf to 0x8aa2e47, (0xd298 bytes) 2024-09-18T00:18:54.256 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:54.250+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:54.261 INFO:teuthology.orchestra.run.smithi097.stdout:3227 trunc from 0xd2fd572 to 0xa87bc6 2024-09-18T00:18:54.274 INFO:teuthology.orchestra.run.smithi097.stdout:3228 read 0xa03b4b thru 0xa13a53 (0xff09 bytes) 2024-09-18T00:18:54.274 INFO:teuthology.orchestra.run.smithi097.stdout:3229 read 0x894944 thru 0x897768 (0x2e25 bytes) 2024-09-18T00:18:54.274 INFO:teuthology.orchestra.run.smithi097.stdout:3230 punch from 0x906fe8 to 0x9102f9, (0x9311 bytes) 2024-09-18T00:18:54.274 INFO:teuthology.orchestra.run.smithi097.stdout:3231 punch from 0x416550 to 0x41e1d4, (0x7c84 bytes) 2024-09-18T00:18:54.286 INFO:teuthology.orchestra.run.smithi097.stdout:3232 write 0x41bf6e2 thru 0x41c55d5 (0x5ef4 bytes) 2024-09-18T00:18:54.293 INFO:teuthology.orchestra.run.smithi097.stdout:3233 read 0x2a153e7 thru 0x2a18d1e (0x3938 bytes) 2024-09-18T00:18:54.293 INFO:teuthology.orchestra.run.smithi097.stdout:3234 read 0x1b1183e thru 0x1b1fab9 (0xe27c bytes) 2024-09-18T00:18:54.293 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:54.293 INFO:teuthology.orchestra.run.smithi097.stdout:3237 read 0x3e91f51 thru 0x3e9d6bb (0xb76b bytes) 2024-09-18T00:18:54.293 INFO:teuthology.orchestra.run.smithi097.stdout:3238 punch from 0x14b1785 to 0x14b649b, (0x4d16 bytes) 2024-09-18T00:18:54.293 INFO:teuthology.orchestra.run.smithi097.stdout:3239 read 0x287b0d7 thru 0x288ab29 (0xfa53 bytes) 2024-09-18T00:18:54.293 INFO:teuthology.orchestra.run.smithi097.stdout:3240 punch from 0x7f777b to 0x80425f, (0xcae4 bytes) 2024-09-18T00:18:54.297 INFO:teuthology.orchestra.run.smithi097.stdout:3241 trunc from 0x41c55d6 to 0x208d602 2024-09-18T00:18:54.308 INFO:teuthology.orchestra.run.smithi097.stdout:3242 write 0x1f63b80 thru 0x1f677ac (0x3c2d bytes) 2024-09-18T00:18:54.314 INFO:teuthology.orchestra.run.smithi097.stdout:3243 read 0xd1c22f thru 0xd1d3ff (0x11d1 bytes) 2024-09-18T00:18:54.315 INFO:teuthology.orchestra.run.smithi097.stdout:3245 trunc from 0x208d602 to 0x2787893 2024-09-18T00:18:54.329 INFO:teuthology.orchestra.run.smithi097.stdout:3246 write 0x2bafbd2 thru 0x2bb03cc (0x7fb bytes) 2024-09-18T00:18:54.357 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:54.350+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:54.357 INFO:teuthology.orchestra.run.smithi097.stdout:3247 writesame 0xc9541ef thru 0xc95805f data_size 0x17(0x3e71 bytes) 2024-09-18T00:18:54.358 INFO:teuthology.orchestra.run.smithi097.stdout:3248 read 0xbcc4ebe thru 0xbccbcef (0x6e32 bytes) 2024-09-18T00:18:54.358 INFO:teuthology.orchestra.run.smithi097.stdout:3249 read 0x76d97e5 thru 0x76e3346 (0x9b62 bytes) 2024-09-18T00:18:54.358 INFO:teuthology.orchestra.run.smithi097.stdout:3250 read 0x97c6f33 thru 0x97c8493 (0x1561 bytes) 2024-09-18T00:18:54.358 INFO:teuthology.orchestra.run.smithi097.stdout:3252 write 0x13ad505 thru 0x13b939f (0xbe9b bytes) 2024-09-18T00:18:54.376 INFO:teuthology.orchestra.run.smithi097.stdout:3254 trunc from 0xc958060 to 0x5c770fc 2024-09-18T00:18:54.385 INFO:teuthology.orchestra.run.smithi097.stdout:3256 read 0xd818bc thru 0xd8a27a (0x89bf bytes) 2024-09-18T00:18:54.390 INFO:teuthology.orchestra.run.smithi097.stdout:3258 trunc from 0x5c770fc to 0x26fafd9 2024-09-18T00:18:54.405 INFO:teuthology.orchestra.run.smithi097.stdout:3259 read 0x1dccfc6 thru 0x1dd86df (0xb71a bytes) 2024-09-18T00:18:54.405 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:54.406 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:54.406 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:54.406 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:54.424 INFO:teuthology.orchestra.run.smithi097.stdout:3260 writesame 0x8723cf0 thru 0x872f4ab data_size 0x16(0xb7bc bytes) 2024-09-18T00:18:54.424 INFO:teuthology.orchestra.run.smithi097.stdout:3262 read 0x7086224 thru 0x709004c (0x9e29 bytes) 2024-09-18T00:18:54.424 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:54.424 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:54.424 INFO:teuthology.orchestra.run.smithi097.stdout:3266 trunc from 0x872f4ac to 0x8b55989 2024-09-18T00:18:54.426 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:18:54.432 INFO:teuthology.orchestra.run.smithi097.stdout:3267 write 0x7221180 thru 0x722ca63 (0xb8e4 bytes) 2024-09-18T00:18:54.441 INFO:teuthology.orchestra.run.smithi097.stdout:3268 punch from 0x6cb4688 to 0x6cc03fa, (0xbd72 bytes) 2024-09-18T00:18:54.441 INFO:teuthology.orchestra.run.smithi097.stdout:3269 write 0x88ec279 thru 0x88f7372 (0xb0fa bytes) 2024-09-18T00:18:54.450 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:54.450 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:54.450 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:54.456 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:54.450+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:54.460 INFO:teuthology.orchestra.run.smithi097.stdout:3273 writesame 0x93c9fd7 thru 0x93d4ace data_size 0x4(0xaaf8 bytes) 2024-09-18T00:18:54.460 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:54.460 INFO:teuthology.orchestra.run.smithi097.stdout:3277 punch from 0xfb04cd to 0xfb4147, (0x3c7a bytes) 2024-09-18T00:18:54.465 INFO:teuthology.orchestra.run.smithi097.stdout:3278 trunc from 0x93d4acf to 0xbdb3361 2024-09-18T00:18:54.474 INFO:teuthology.orchestra.run.smithi097.stdout:3279 write 0x2345d10 thru 0x23505ca (0xa8bb bytes) 2024-09-18T00:18:54.482 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:54.482 INFO:teuthology.orchestra.run.smithi097.stdout:3281 punch from 0x402b459 to 0x402f000, (0x3ba7 bytes) 2024-09-18T00:18:54.482 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:54.498 INFO:teuthology.orchestra.run.smithi097.stdout:3283 trunc from 0xbdb3361 to 0x3947c88 2024-09-18T00:18:54.539 INFO:teuthology.orchestra.run.smithi097.stdout:3285 writesame 0xb46d27c thru 0xb47b247 data_size 0x2(0xdfcc bytes) 2024-09-18T00:18:54.539 INFO:teuthology.orchestra.run.smithi097.stdout:3286 write 0x8d7dd83 thru 0x8d8becc (0xe14a bytes) 2024-09-18T00:18:54.549 INFO:teuthology.orchestra.run.smithi097.stdout:3287 write 0x1252cb thru 0x129cb3 (0x49e9 bytes) 2024-09-18T00:18:54.553 INFO:teuthology.orchestra.run.smithi097.stdout:3288 punch from 0x80ab4b4 to 0x80b5366, (0x9eb2 bytes) 2024-09-18T00:18:54.553 INFO:teuthology.orchestra.run.smithi097.stdout:3289 punch from 0x90ff012 to 0x910e2ba, (0xf2a8 bytes) 2024-09-18T00:18:54.553 INFO:teuthology.orchestra.run.smithi097.stdout:3290 clone 15 order 25 su 131072 sc 10 2024-09-18T00:18:54.560 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:54.550+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:54.568 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:54.568 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:54.568 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:54.569 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:54.569 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:54.583 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:18:54.669 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:54.666+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:54.722 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:18:54.749 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:18:54.754 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:54.754 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.951445716999999, 2024-09-18T00:18:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:18:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.755 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:18:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:18:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:18:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:18:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:18:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:18:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:18:54.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:18:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:18:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:18:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:18:54.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:18:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:18:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:18:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:18:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:18:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:18:54.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:18:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:18:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:18:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:18:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:18:54.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:18:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:18:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.935237844, 2024-09-18T00:18:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:18:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:18:54.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:18:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:18:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:18:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:18:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:18:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:18:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:18:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:18:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:18:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:18:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:18:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:18:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:18:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.908196524000001, 2024-09-18T00:18:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:18:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.767 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:18:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:54.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:18:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:18:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:18:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:18:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:18:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:18:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:18:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:18:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:18:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:18:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:18:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:18:54.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:18:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:18:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:18:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:18:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:18:54.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:18:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:18:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:18:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:18:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:18:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:18:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.886919972000001, 2024-09-18T00:18:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:18:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:54.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:18:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:18:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:18:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:18:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:18:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:18:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:18:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:18:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:18:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:18:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:18:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:18:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:18:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:18:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11460 3.19 3:9e578b6c:::rbd_data.112ab1fa102a.0000000000000153:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:18:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.484548520000001, 2024-09-18T00:18:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041989011999999999, 2024-09-18T00:18:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11460 2024-09-18T00:18:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:18:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:18:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267067+0000", 2024-09-18T00:18:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:18:54.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267088+0000", 2024-09-18T00:18:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0908999999999999e-05 2024-09-18T00:18:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267133+0000", 2024-09-18T00:18:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5232000000000003e-05 2024-09-18T00:18:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267157+0000", 2024-09-18T00:18:54.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3833e-05 2024-09-18T00:18:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267300+0000", 2024-09-18T00:18:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142282 2024-09-18T00:18:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267447+0000", 2024-09-18T00:18:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147851 2024-09-18T00:18:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308889+0000", 2024-09-18T00:18:54.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041441779999999998 2024-09-18T00:18:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308956+0000", 2024-09-18T00:18:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7136000000000002e-05 2024-09-18T00:18:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.309053+0000", 2024-09-18T00:18:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6552999999999995e-05 2024-09-18T00:18:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.782 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11464 3.b 3:d331f8ac:::rbd_data.112ab1fa102a.0000000000000157:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:18:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.479558034, 2024-09-18T00:18:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034709218, 2024-09-18T00:18:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11464 2024-09-18T00:18:54.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:18:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:18:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272058+0000", 2024-09-18T00:18:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4070000000000002e-06 2024-09-18T00:18:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272074+0000", 2024-09-18T00:18:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5872999999999999e-05 2024-09-18T00:18:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272113+0000", 2024-09-18T00:18:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9224999999999998e-05 2024-09-18T00:18:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272135+0000", 2024-09-18T00:18:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2093e-05 2024-09-18T00:18:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272238+0000", 2024-09-18T00:18:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102939 2024-09-18T00:18:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272327+0000", 2024-09-18T00:18:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8746999999999996e-05 2024-09-18T00:18:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.273486+0000", 2024-09-18T00:18:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011588779999999999 2024-09-18T00:18:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306439+0000", 2024-09-18T00:18:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.032953142999999997 2024-09-18T00:18:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306491+0000", 2024-09-18T00:18:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2475000000000002e-05 2024-09-18T00:18:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306648+0000", 2024-09-18T00:18:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015725099999999999 2024-09-18T00:18:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306764+0000", 2024-09-18T00:18:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011518699999999999 2024-09-18T00:18:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:16947 3.18 3:18b58c7b:::rbd_data.112a6efb7564.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:18:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:18:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.1347708189999999, 2024-09-18T00:18:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.101692484, 2024-09-18T00:18:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.789 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 16947 2024-09-18T00:18:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:18:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:18:54.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616843+0000", 2024-09-18T00:18:54.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0780000000000001e-06 2024-09-18T00:18:54.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617785+0000", 2024-09-18T00:18:54.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094204100000000002 2024-09-18T00:18:54.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617801+0000", 2024-09-18T00:18:54.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6436999999999999e-05 2024-09-18T00:18:54.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617809+0000", 2024-09-18T00:18:54.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9049999999999997e-06 2024-09-18T00:18:54.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617904+0000", 2024-09-18T00:18:54.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5204999999999994e-05 2024-09-18T00:18:54.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.618071+0000", 2024-09-18T00:18:54.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016640000000000001 2024-09-18T00:18:54.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:54.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.624534+0000", 2024-09-18T00:18:54.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064634499999999999 2024-09-18T00:18:54.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718299+0000", 2024-09-18T00:18:54.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093764739 2024-09-18T00:18:54.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:54.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718375+0000", 2024-09-18T00:18:54.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6532999999999994e-05 2024-09-18T00:18:54.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718466+0000", 2024-09-18T00:18:54.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0581000000000006e-05 2024-09-18T00:18:54.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718534+0000", 2024-09-18T00:18:54.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8114999999999994e-05 2024-09-18T00:18:54.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.794 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.795 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:16947 3.18 e59/23 ondisk, result = 0)", 2024-09-18T00:18:54.795 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:18:54.795 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.1095541249999998, 2024-09-18T00:18:54.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076488069000000006, 2024-09-18T00:18:54.795 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.795 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.795 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.795 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:18:54.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:18:54.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642060+0000", 2024-09-18T00:18:54.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6640000000000001e-06 2024-09-18T00:18:54.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642071+0000", 2024-09-18T00:18:54.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1204e-05 2024-09-18T00:18:54.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642091+0000", 2024-09-18T00:18:54.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0313999999999999e-05 2024-09-18T00:18:54.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642103+0000", 2024-09-18T00:18:54.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1214e-05 2024-09-18T00:18:54.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718350+0000", 2024-09-18T00:18:54.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076247681999999997 2024-09-18T00:18:54.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718366+0000", 2024-09-18T00:18:54.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5420000000000001e-05 2024-09-18T00:18:54.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718546+0000", 2024-09-18T00:18:54.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000180571 2024-09-18T00:18:54.798 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.799 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.799 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.799 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:17323 3.18 3:18b58c7b:::rbd_data.112a6efb7564.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:18:54.799 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:49.402708+0000", 2024-09-18T00:18:54.799 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.3489039500000004, 2024-09-18T00:18:54.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034488817999999997, 2024-09-18T00:18:54.799 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.799 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.799 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.799 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.800 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.800 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 17323 2024-09-18T00:18:54.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.800 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.402708+0000", 2024-09-18T00:18:54.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.402708+0000", 2024-09-18T00:18:54.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.402711+0000", 2024-09-18T00:18:54.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2139999999999999e-06 2024-09-18T00:18:54.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.402722+0000", 2024-09-18T00:18:54.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1158999999999999e-05 2024-09-18T00:18:54.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.402752+0000", 2024-09-18T00:18:54.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9805999999999999e-05 2024-09-18T00:18:54.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.402766+0000", 2024-09-18T00:18:54.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4633e-05 2024-09-18T00:18:54.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.402852+0000", 2024-09-18T00:18:54.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5799999999999998e-05 2024-09-18T00:18:54.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.402943+0000", 2024-09-18T00:18:54.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0636000000000004e-05 2024-09-18T00:18:54.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:54.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.406990+0000", 2024-09-18T00:18:54.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040471109999999999 2024-09-18T00:18:54.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:54.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.437014+0000", 2024-09-18T00:18:54.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.03002442 2024-09-18T00:18:54.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.437036+0000", 2024-09-18T00:18:54.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1923999999999999e-05 2024-09-18T00:18:54.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.437124+0000", 2024-09-18T00:18:54.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8027000000000003e-05 2024-09-18T00:18:54.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.437197+0000", 2024-09-18T00:18:54.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3088000000000002e-05 2024-09-18T00:18:54.805 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.805 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.805 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.805 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:17335 3.7 3:e69124d0:::rbd_data.112a6efb7564.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:18:54.805 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:49.461417+0000", 2024-09-18T00:18:54.806 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.2901954230000001, 2024-09-18T00:18:54.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022323323999999999, 2024-09-18T00:18:54.806 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.806 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.806 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.806 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.806 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.806 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.806 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 17335 2024-09-18T00:18:54.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.806 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.461417+0000", 2024-09-18T00:18:54.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.461417+0000", 2024-09-18T00:18:54.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.461419+0000", 2024-09-18T00:18:54.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3240000000000001e-06 2024-09-18T00:18:54.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.461431+0000", 2024-09-18T00:18:54.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.217e-05 2024-09-18T00:18:54.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.461462+0000", 2024-09-18T00:18:54.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0672e-05 2024-09-18T00:18:54.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.461484+0000", 2024-09-18T00:18:54.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2107e-05 2024-09-18T00:18:54.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.461597+0000", 2024-09-18T00:18:54.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000113053 2024-09-18T00:18:54.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.461685+0000", 2024-09-18T00:18:54.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7923999999999994e-05 2024-09-18T00:18:54.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:54.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.464094+0000", 2024-09-18T00:18:54.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0024083220000000001 2024-09-18T00:18:54.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.483300+0000", 2024-09-18T00:18:54.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019206804000000001 2024-09-18T00:18:54.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:54.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.483444+0000", 2024-09-18T00:18:54.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000143669 2024-09-18T00:18:54.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.483614+0000", 2024-09-18T00:18:54.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000169786 2024-09-18T00:18:54.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.483740+0000", 2024-09-18T00:18:54.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012649300000000001 2024-09-18T00:18:54.810 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.811 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.811 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.811 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:17363 3.15 e59/32 3:ad4938b4:::rbd_object_map.112a6efb7564:head v 59'143, mlcod=59'141)", 2024-09-18T00:18:54.811 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:49.539843+0000", 2024-09-18T00:18:54.811 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.2117691529999997, 2024-09-18T00:18:54.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023088518999999998, 2024-09-18T00:18:54.811 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.811 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.811 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.811 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.539843+0000", 2024-09-18T00:18:54.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.539843+0000", 2024-09-18T00:18:54.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.539845+0000", 2024-09-18T00:18:54.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.81e-06 2024-09-18T00:18:54.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.539858+0000", 2024-09-18T00:18:54.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2894000000000001e-05 2024-09-18T00:18:54.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.539889+0000", 2024-09-18T00:18:54.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0664999999999999e-05 2024-09-18T00:18:54.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.539902+0000", 2024-09-18T00:18:54.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3203999999999999e-05 2024-09-18T00:18:54.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.539989+0000", 2024-09-18T00:18:54.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7318999999999995e-05 2024-09-18T00:18:54.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.540016+0000", 2024-09-18T00:18:54.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6735000000000001e-05 2024-09-18T00:18:54.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.562832+0000", 2024-09-18T00:18:54.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022816184999999999 2024-09-18T00:18:54.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.562932+0000", 2024-09-18T00:18:54.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9706999999999999e-05 2024-09-18T00:18:54.815 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.815 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.815 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.815 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18684 3.b 3:d2138ebe:::rbd_data.112a5ba20d45.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.815 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.947362+0000", 2024-09-18T00:18:54.815 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.8042501500000001, 2024-09-18T00:18:54.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022849864000000001, 2024-09-18T00:18:54.815 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.815 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.816 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.816 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.816 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.816 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.816 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18684 2024-09-18T00:18:54.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.816 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.947362+0000", 2024-09-18T00:18:54.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.947362+0000", 2024-09-18T00:18:54.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.947365+0000", 2024-09-18T00:18:54.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0630000000000002e-06 2024-09-18T00:18:54.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948505+0000", 2024-09-18T00:18:54.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011400679999999999 2024-09-18T00:18:54.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948548+0000", 2024-09-18T00:18:54.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2318999999999999e-05 2024-09-18T00:18:54.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948572+0000", 2024-09-18T00:18:54.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4053000000000002e-05 2024-09-18T00:18:54.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:18:54.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016425899999999999 2024-09-18T00:18:54.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949055+0000", 2024-09-18T00:18:54.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031904600000000002 2024-09-18T00:18:54.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:54.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949775+0000", 2024-09-18T00:18:54.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00072009500000000002 2024-09-18T00:18:54.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.963879+0000", 2024-09-18T00:18:54.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014103704 2024-09-18T00:18:54.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:54.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.970012+0000", 2024-09-18T00:18:54.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0061329369999999998 2024-09-18T00:18:54.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.970126+0000", 2024-09-18T00:18:54.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000114436 2024-09-18T00:18:54.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.970212+0000", 2024-09-18T00:18:54.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5884000000000006e-05 2024-09-18T00:18:54.820 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.820 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18686 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.802876755, 2024-09-18T00:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041132573999999998, 2024-09-18T00:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18686 2024-09-18T00:18:54.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:18:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:18:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948737+0000", 2024-09-18T00:18:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3769999999999999e-06 2024-09-18T00:18:54.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949968+0000", 2024-09-18T00:18:54.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001230493 2024-09-18T00:18:54.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949993+0000", 2024-09-18T00:18:54.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5282e-05 2024-09-18T00:18:54.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950011+0000", 2024-09-18T00:18:54.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8199999999999999e-05 2024-09-18T00:18:54.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950180+0000", 2024-09-18T00:18:54.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016867600000000001 2024-09-18T00:18:54.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950337+0000", 2024-09-18T00:18:54.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156902 2024-09-18T00:18:54.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:54.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950818+0000", 2024-09-18T00:18:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048192599999999998 2024-09-18T00:18:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978491+0000", 2024-09-18T00:18:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027672568000000002 2024-09-18T00:18:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989759+0000", 2024-09-18T00:18:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011268144000000001 2024-09-18T00:18:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989824+0000", 2024-09-18T00:18:54.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4556999999999999e-05 2024-09-18T00:18:54.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989868+0000", 2024-09-18T00:18:54.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4449000000000001e-05 2024-09-18T00:18:54.826 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.826 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.826 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.826 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18687 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.826 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:18:54.826 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.8015489900000001, 2024-09-18T00:18:54.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040034156000000001, 2024-09-18T00:18:54.826 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.827 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.827 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.827 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.827 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.827 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.827 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18687 2024-09-18T00:18:54.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.827 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:18:54.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:18:54.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950065+0000", 2024-09-18T00:18:54.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5820000000000001e-06 2024-09-18T00:18:54.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950072+0000", 2024-09-18T00:18:54.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1029999999999996e-06 2024-09-18T00:18:54.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950092+0000", 2024-09-18T00:18:54.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0288e-05 2024-09-18T00:18:54.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950107+0000", 2024-09-18T00:18:54.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4290999999999999e-05 2024-09-18T00:18:54.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975820+0000", 2024-09-18T00:18:54.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025713573999999999 2024-09-18T00:18:54.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975980+0000", 2024-09-18T00:18:54.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016002799999999999 2024-09-18T00:18:54.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:54.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.976299+0000", 2024-09-18T00:18:54.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031853300000000002 2024-09-18T00:18:54.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:54.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989936+0000", 2024-09-18T00:18:54.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013637046999999999 2024-09-18T00:18:54.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990025+0000", 2024-09-18T00:18:54.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9040000000000001e-05 2024-09-18T00:18:54.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990064+0000", 2024-09-18T00:18:54.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9394000000000003e-05 2024-09-18T00:18:54.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990098+0000", 2024-09-18T00:18:54.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3275999999999998e-05 2024-09-18T00:18:54.832 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.832 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.832 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.832 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18688 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.832 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:18:54.832 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.801447638, 2024-09-18T00:18:54.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040041259000000003, 2024-09-18T00:18:54.833 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.833 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.833 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.833 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.833 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18688 2024-09-18T00:18:54.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.833 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:18:54.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:18:54.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950166+0000", 2024-09-18T00:18:54.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:18:54.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951295+0000", 2024-09-18T00:18:54.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011293480000000001 2024-09-18T00:18:54.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951328+0000", 2024-09-18T00:18:54.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2524999999999998e-05 2024-09-18T00:18:54.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951346+0000", 2024-09-18T00:18:54.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8170999999999999e-05 2024-09-18T00:18:54.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978546+0000", 2024-09-18T00:18:54.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027200418000000001 2024-09-18T00:18:54.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978666+0000", 2024-09-18T00:18:54.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119913 2024-09-18T00:18:54.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:54.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.979098+0000", 2024-09-18T00:18:54.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00043169399999999999 2024-09-18T00:18:54.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:54.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990000+0000", 2024-09-18T00:18:54.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010901719000000001 2024-09-18T00:18:54.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990109+0000", 2024-09-18T00:18:54.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010964299999999999 2024-09-18T00:18:54.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990152+0000", 2024-09-18T00:18:54.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2877000000000003e-05 2024-09-18T00:18:54.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990206+0000", 2024-09-18T00:18:54.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3717000000000001e-05 2024-09-18T00:18:54.838 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.838 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.838 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.838 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18689 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.838 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:18:54.838 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.8002359399999999, 2024-09-18T00:18:54.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051022338, 2024-09-18T00:18:54.838 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.838 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.839 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.839 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.839 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.839 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.839 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18689 2024-09-18T00:18:54.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.839 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:18:54.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:18:54.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951378+0000", 2024-09-18T00:18:54.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:18:54.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951385+0000", 2024-09-18T00:18:54.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-18T00:18:54.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951402+0000", 2024-09-18T00:18:54.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6538999999999999e-05 2024-09-18T00:18:54.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951565+0000", 2024-09-18T00:18:54.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016307999999999999 2024-09-18T00:18:54.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988232+0000", 2024-09-18T00:18:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036667656999999999 2024-09-18T00:18:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988318+0000", 2024-09-18T00:18:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5348999999999999e-05 2024-09-18T00:18:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988574+0000", 2024-09-18T00:18:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025670800000000002 2024-09-18T00:18:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002254+0000", 2024-09-18T00:18:54.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013679274 2024-09-18T00:18:54.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002326+0000", 2024-09-18T00:18:54.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2720999999999997e-05 2024-09-18T00:18:54.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002364+0000", 2024-09-18T00:18:54.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7314000000000002e-05 2024-09-18T00:18:54.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002399+0000", 2024-09-18T00:18:54.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5162e-05 2024-09-18T00:18:54.844 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.844 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.844 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.844 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18690 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.844 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:18:54.844 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.800018922, 2024-09-18T00:18:54.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050872134999999999, 2024-09-18T00:18:54.844 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.845 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.845 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.845 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.845 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.845 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.845 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18690 2024-09-18T00:18:54.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.845 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:18:54.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:18:54.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951595+0000", 2024-09-18T00:18:54.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4479999999999999e-06 2024-09-18T00:18:54.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952167+0000", 2024-09-18T00:18:54.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00057188400000000004 2024-09-18T00:18:54.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952192+0000", 2024-09-18T00:18:54.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4936999999999999e-05 2024-09-18T00:18:54.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952206+0000", 2024-09-18T00:18:54.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4565e-05 2024-09-18T00:18:54.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990303+0000", 2024-09-18T00:18:54.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038096954000000002 2024-09-18T00:18:54.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990403+0000", 2024-09-18T00:18:54.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9486000000000003e-05 2024-09-18T00:18:54.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:54.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990773+0000", 2024-09-18T00:18:54.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037037799999999998 2024-09-18T00:18:54.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:54.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002316+0000", 2024-09-18T00:18:54.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011542581 2024-09-18T00:18:54.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002407+0000", 2024-09-18T00:18:54.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0968e-05 2024-09-18T00:18:54.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002442+0000", 2024-09-18T00:18:54.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4962000000000002e-05 2024-09-18T00:18:54.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002466+0000", 2024-09-18T00:18:54.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3972e-05 2024-09-18T00:18:54.850 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.850 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.850 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.850 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18691 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.850 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:18:54.850 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.7993860449999999, 2024-09-18T00:18:54.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051495530999999997, 2024-09-18T00:18:54.850 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.851 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.851 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.851 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.851 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.851 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.851 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18691 2024-09-18T00:18:54.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.851 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:18:54.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:18:54.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952228+0000", 2024-09-18T00:18:54.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:18:54.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952234+0000", 2024-09-18T00:18:54.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0310000000000004e-06 2024-09-18T00:18:54.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952248+0000", 2024-09-18T00:18:54.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4577e-05 2024-09-18T00:18:54.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952260+0000", 2024-09-18T00:18:54.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1659e-05 2024-09-18T00:18:54.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.000989+0000", 2024-09-18T00:18:54.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048729445000000003 2024-09-18T00:18:54.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001076+0000", 2024-09-18T00:18:54.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6323999999999995e-05 2024-09-18T00:18:54.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:54.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001295+0000", 2024-09-18T00:18:54.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021944299999999999 2024-09-18T00:18:54.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003502+0000", 2024-09-18T00:18:54.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022065209999999999 2024-09-18T00:18:54.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:54.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003581+0000", 2024-09-18T00:18:54.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9404000000000001e-05 2024-09-18T00:18:54.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003652+0000", 2024-09-18T00:18:54.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0506999999999994e-05 2024-09-18T00:18:54.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003722+0000", 2024-09-18T00:18:54.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0300000000000001e-05 2024-09-18T00:18:54.856 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.856 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.856 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.856 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18692 3.7 3:e69124d0:::rbd_data.112a6efb7564.0000000000000001:f [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.856 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:18:54.856 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.7993379840000001, 2024-09-18T00:18:54.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050418114999999999, 2024-09-18T00:18:54.856 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.857 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.857 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:54.857 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.857 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.857 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.857 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18692 2024-09-18T00:18:54.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.857 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:18:54.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:18:54.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952275+0000", 2024-09-18T00:18:54.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4e-07 2024-09-18T00:18:54.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952281+0000", 2024-09-18T00:18:54.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2000000000000002e-06 2024-09-18T00:18:54.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952295+0000", 2024-09-18T00:18:54.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4731e-05 2024-09-18T00:18:54.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952307+0000", 2024-09-18T00:18:54.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1877e-05 2024-09-18T00:18:54.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002506+0000", 2024-09-18T00:18:54.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050199230999999997 2024-09-18T00:18:54.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002574+0000", 2024-09-18T00:18:54.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8076000000000003e-05 2024-09-18T00:18:54.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002693+0000", 2024-09-18T00:18:54.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011806 2024-09-18T00:18:54.860 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.861 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.861 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.861 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19383 3.5 3:a1859bb0:::rbd_data.112a7fe29ab4.000000000000015b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:54.861 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:18:54.861 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.1293263869999999, 2024-09-18T00:18:54.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056760192000000001, 2024-09-18T00:18:54.861 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:54.861 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:54.861 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:54.861 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:54.861 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:54.861 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:54.861 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19383 2024-09-18T00:18:54.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.862 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:54.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:54.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:18:54.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:54.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:18:54.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:54.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:54.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622287+0000", 2024-09-18T00:18:54.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-18T00:18:54.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:54.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622292+0000", 2024-09-18T00:18:54.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6110000000000004e-06 2024-09-18T00:18:54.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:54.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622304+0000", 2024-09-18T00:18:54.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2106000000000001e-05 2024-09-18T00:18:54.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:54.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622309+0000", 2024-09-18T00:18:54.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.818e-06 2024-09-18T00:18:54.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:54.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622360+0000", 2024-09-18T00:18:54.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0430999999999999e-05 2024-09-18T00:18:54.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:54.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622438+0000", 2024-09-18T00:18:54.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8362999999999995e-05 2024-09-18T00:18:54.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:54.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678811+0000", 2024-09-18T00:18:54.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056372894 2024-09-18T00:18:54.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:54.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678906+0000", 2024-09-18T00:18:54.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5078e-05 2024-09-18T00:18:54.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:54.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:54.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:54.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.679046+0000", 2024-09-18T00:18:54.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139987 2024-09-18T00:18:54.866 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.866 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.866 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.866 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:54.866 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:54.866 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:54.868 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:18:54.868 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:54.862+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:54.875 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:54.875 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:54.876 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:18:54.878 INFO:teuthology.orchestra.run.smithi114.stdout:get_pg_num_history pg_num_history pg_num_history(e55 pg_nums {1={16=1},2={19=8},3={23=32}} deleted_pools ) 2024-09-18T00:18:54.878 INFO:teuthology.orchestra.run.smithi114.stdout:Importing pgid 2.4 2024-09-18T00:18:54.878 INFO:teuthology.orchestra.run.smithi114.stdout:write_pg epoch 24 info 2.4( empty local-lis/les=19/20 n=0 ec=19/19 lis/c=19/19 les/c/f=20/20/0 sis=19) 2024-09-18T00:18:54.884 INFO:teuthology.orchestra.run.smithi114.stdout:Import successful 2024-09-18T00:18:54.892 INFO:teuthology.orchestra.run.smithi097.stdout:truncating image image_client.0-clone14 from 0xb47b248 (overlap 0x17aa62) to 0xbd531 2024-09-18T00:18:54.910 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:54.910 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:54.937 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:54.937 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:54.968 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:54.962+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:54.972 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:54.973 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:54.973 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-18T00:18:54.994 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:54.994 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:54.997 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:54.997 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:55.009 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:18:55.017 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:55.017 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:55.022 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-18T00:18:55.023 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:18:55.024 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:18:55.024 INFO:teuthology.orchestra.run.smithi097.stdout:3291 write 0x114157 thru 0x11cdf2 (0x8c9c bytes) 2024-09-18T00:18:55.035 INFO:teuthology.orchestra.run.smithi097.stdout:3293 read 0x6aad561 thru 0x6ab818f (0xac2f bytes) 2024-09-18T00:18:55.035 INFO:teuthology.orchestra.run.smithi097.stdout:3295 write 0xa7080 thru 0xaae29 (0x3daa bytes) 2024-09-18T00:18:55.044 INFO:teuthology.orchestra.run.smithi097.stdout:3296 read 0x2a1a29b thru 0x2a1f39f (0x5105 bytes) 2024-09-18T00:18:55.044 INFO:teuthology.orchestra.run.smithi097.stdout:3297 write 0x84262fb thru 0x8430e63 (0xab69 bytes) 2024-09-18T00:18:55.062 INFO:teuthology.orchestra.run.smithi097.stdout:3299 writesame 0xd1fbd88 thru 0xd202f00 data_size 0x17(0x7179 bytes) 2024-09-18T00:18:55.069 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:55.066+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:55.070 INFO:teuthology.orchestra.run.smithi097.stdout:3300 read 0x4603874 thru 0x460922a (0x59b7 bytes) 2024-09-18T00:18:55.070 INFO:teuthology.orchestra.run.smithi097.stdout:3301 write 0x5b33f6e thru 0x5b3854e (0x45e1 bytes) 2024-09-18T00:18:55.072 INFO:teuthology.orchestra.run.smithi097.stdout:3303 read 0x1455e1 thru 0x152957 (0xd377 bytes) 2024-09-18T00:18:55.073 INFO:teuthology.orchestra.run.smithi097.stdout:3306 write 0x5fb447f thru 0x5fb45bb (0x13d bytes) 2024-09-18T00:18:55.074 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:55.089 INFO:teuthology.orchestra.run.smithi097.stdout:3308 trunc from 0xd202f01 to 0x2355587 2024-09-18T00:18:55.125 INFO:teuthology.orchestra.run.smithi097.stdout:3309 writesame 0x818bd0f thru 0x818fe8e data_size 0x40(0x4180 bytes) 2024-09-18T00:18:55.127 INFO:teuthology.orchestra.run.smithi097.stdout:3310 punch from 0x24b04a6 to 0x24b3b29, (0x3683 bytes) 2024-09-18T00:18:55.138 INFO:teuthology.orchestra.run.smithi097.stdout:3312 write 0xb3f0216 thru 0xb3ff7a5 (0xf590 bytes) 2024-09-18T00:18:55.140 INFO:teuthology.orchestra.run.smithi097.stdout:3313 write 0x797238e thru 0x797a664 (0x82d7 bytes) 2024-09-18T00:18:55.143 INFO:teuthology.orchestra.run.smithi097.stdout:3314 read 0x37e84ef thru 0x37f0854 (0x8366 bytes) 2024-09-18T00:18:55.143 INFO:teuthology.orchestra.run.smithi097.stdout:3315 read 0x282ef78 thru 0x283e5c9 (0xf652 bytes) 2024-09-18T00:18:55.144 INFO:teuthology.orchestra.run.smithi097.stdout:3317 read 0x18ec439 thru 0x18f8c87 (0xc84f bytes) 2024-09-18T00:18:55.158 INFO:teuthology.orchestra.run.smithi097.stdout:3318 trunc from 0xb3ff7a6 to 0x217ad65 2024-09-18T00:18:55.167 INFO:teuthology.orchestra.run.smithi097.stdout:3319 read 0x2137f82 thru 0x214172e (0x97ad bytes) 2024-09-18T00:18:55.170 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:55.166+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:55.171 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:55.171 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:55.186 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:18:55.189 INFO:teuthology.orchestra.run.smithi097.stdout:3321 writesame 0xc1c1b69 thru 0xc1c67fd data_size 0x5(0x4c95 bytes) 2024-09-18T00:18:55.197 INFO:teuthology.orchestra.run.smithi097.stdout:3322 trunc from 0xc1c67fe to 0x6a4d967 2024-09-18T00:18:55.217 INFO:teuthology.orchestra.run.smithi097.stdout:3325 trunc from 0x6a4d967 to 0x99902eb 2024-09-18T00:18:55.224 INFO:teuthology.orchestra.run.smithi097.stdout:3326 writesame 0x2dfd723 thru 0x2e048c5 data_size 0x1(0x71a3 bytes) 2024-09-18T00:18:55.271 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:55.266+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:55.291 INFO:teuthology.orchestra.run.smithi097.stdout:3329 write 0x60a652b thru 0x60ac42d (0x5f03 bytes) 2024-09-18T00:18:55.297 INFO:teuthology.orchestra.run.smithi097.stdout:3330 trunc from 0x99902eb to 0xa6d263e 2024-09-18T00:18:55.311 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:55.311 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:55.325 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:18:55.328 INFO:teuthology.orchestra.run.smithi097.stdout:3331 writesame 0xc3bda85 thru 0xc3c7716 data_size 0x31(0x9c92 bytes) 2024-09-18T00:18:55.331 INFO:teuthology.orchestra.run.smithi097.stdout:3337 write 0x5bd25d6 thru 0x5bda8f4 (0x831f bytes) 2024-09-18T00:18:55.335 INFO:teuthology.orchestra.run.smithi097.stdout:3338 write 0x56b9988 thru 0x56c876c (0xede5 bytes) 2024-09-18T00:18:55.340 INFO:teuthology.orchestra.run.smithi097.stdout:3339 write 0x63603b thru 0x63b1d3 (0x5199 bytes) 2024-09-18T00:18:55.345 INFO:teuthology.orchestra.run.smithi097.stdout:3341 read 0x5b1c465 thru 0x5b2661c (0xa1b8 bytes) 2024-09-18T00:18:55.376 INFO:teuthology.orchestra.run.smithi097.stdout:3342 trunc from 0xc3c7717 to 0x3bc28e 2024-09-18T00:18:55.409 INFO:teuthology.orchestra.run.smithi097.stdout:3344 trunc from 0x3bc28e to 0x7c7281b 2024-09-18T00:18:55.414 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:55.410+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:55.433 INFO:teuthology.orchestra.run.smithi097.stdout:3345 writesame 0x81a6866 thru 0x81b3570 data_size 0x1(0xcd0b bytes) 2024-09-18T00:18:55.449 INFO:teuthology.orchestra.run.smithi097.stdout:3348 write 0x8a91413 thru 0x8a96ed0 (0x5abe bytes) 2024-09-18T00:18:55.450 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /var/log/ceph/exp.2.4.1 2024-09-18T00:18:55.453 INFO:teuthology.orchestra.run.smithi097.stdout:3352 read 0x817c85f thru 0x8189316 (0xcab8 bytes) 2024-09-18T00:18:55.453 INFO:teuthology.orchestra.run.smithi097.stdout:3353 punch from 0x5cb8316 to 0x5cc4c2f, (0xc919 bytes) 2024-09-18T00:18:55.453 INFO:teuthology.orchestra.run.smithi097.stdout:3354 read 0x1b2761b thru 0x1b334f0 (0xbed6 bytes) 2024-09-18T00:18:55.461 INFO:teuthology.orchestra.run.smithi097.stdout:3356 trunc from 0x8a96ed1 to 0xb2a405c 2024-09-18T00:18:55.465 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f /var/log/ceph/exp.2.4.1 2024-09-18T00:18:55.473 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:55.473 INFO:teuthology.orchestra.run.smithi097.stdout:3360 read 0x7d91800 thru 0x7d9283c (0x103d bytes) 2024-09-18T00:18:55.474 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:55.470+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:55.491 INFO:teuthology.orchestra.run.smithi097.stdout:3361 write 0xd342710 thru 0xd350adf (0xe3d0 bytes) 2024-09-18T00:18:55.494 INFO:teuthology.orchestra.run.smithi097.stdout:3363 read 0x727c036 thru 0x728283b (0x6806 bytes) 2024-09-18T00:18:55.495 INFO:teuthology.orchestra.run.smithi097.stdout:3365 write 0x92133c7 thru 0x92187d0 (0x540a bytes) 2024-09-18T00:18:55.499 INFO:teuthology.orchestra.run.smithi097.stdout:3366 read 0x69f7c73 thru 0x69fe858 (0x6be6 bytes) 2024-09-18T00:18:55.500 INFO:teuthology.orchestra.run.smithi097.stdout:3367 write 0x1251b7a thru 0x1256027 (0x44ae bytes) 2024-09-18T00:18:55.503 INFO:teuthology.orchestra.run.smithi097.stdout:3368 write 0x2f61af9 thru 0x2f684bf (0x69c7 bytes) 2024-09-18T00:18:55.506 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:18:55.507 INFO:teuthology.orchestra.run.smithi097.stdout:3369 punch from 0xca3fc12 to 0xca4c70d, (0xcafb bytes) 2024-09-18T00:18:55.507 INFO:teuthology.orchestra.run.smithi097.stdout:3370 write 0x382ece2 thru 0x3833abe (0x4ddd bytes) 2024-09-18T00:18:55.527 INFO:teuthology.orchestra.run.smithi097.stdout:3371 write 0xeae0869 thru 0xeae45c7 (0x3d5f bytes) 2024-09-18T00:18:55.558 INFO:teuthology.orchestra.run.smithi097.stdout:3372 trunc from 0xeae45c8 to 0x546a4d 2024-09-18T00:18:55.593 INFO:teuthology.orchestra.run.smithi097.stdout:3373 write 0x67d182f thru 0x67d3d04 (0x24d6 bytes) 2024-09-18T00:18:55.597 INFO:teuthology.orchestra.run.smithi097.stdout:3374 punch from 0x2f3d1e3 to 0x2f4c470, (0xf28d bytes) 2024-09-18T00:18:55.597 INFO:teuthology.orchestra.run.smithi097.stdout:3375 read 0x4f0b8b1 thru 0x4f19123 (0xd873 bytes) 2024-09-18T00:18:55.598 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:55.598+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:55.608 INFO:teuthology.orchestra.run.smithi097.stdout:3376 writesame 0x6aff6fb thru 0x6affea7 data_size 0x5(0x7ad bytes) 2024-09-18T00:18:55.610 INFO:teuthology.orchestra.run.smithi097.stdout:3377 read 0x2ac0db4 thru 0x2ac3f97 (0x31e4 bytes) 2024-09-18T00:18:55.611 INFO:teuthology.orchestra.run.smithi097.stdout:3379 read 0x150e287 thru 0x150e769 (0x4e3 bytes) 2024-09-18T00:18:55.612 INFO:teuthology.orchestra.run.smithi097.stdout:3380 write 0x384fa12 thru 0x3858182 (0x8771 bytes) 2024-09-18T00:18:55.618 INFO:teuthology.orchestra.run.smithi097.stdout:3382 read 0x8c2133 thru 0x8c351f (0x13ed bytes) 2024-09-18T00:18:55.619 INFO:teuthology.orchestra.run.smithi097.stdout:3383 writesame 0x187b7d3 thru 0x1885de3 data_size 0x25(0xa611 bytes) 2024-09-18T00:18:55.623 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:55.648 INFO:teuthology.orchestra.run.smithi097.stdout:3386 writesame 0xb0509a7 thru 0xb054d82 data_size 0x4(0x43dc bytes) 2024-09-18T00:18:55.654 INFO:teuthology.orchestra.run.smithi097.stdout:3387 trunc from 0xb054d83 to 0xc879471 2024-09-18T00:18:55.666 INFO:teuthology.orchestra.run.smithi097.stdout:3388 read 0x33a05b1 thru 0x33abf0b (0xb95b bytes) 2024-09-18T00:18:55.674 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:55.674 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:55.674 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:55.670+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:55.685 INFO:teuthology.orchestra.run.smithi097.stdout:3389 writesame 0xe24243a thru 0xe24522c data_size 0x3(0x2df3 bytes) 2024-09-18T00:18:55.686 INFO:teuthology.orchestra.run.smithi097.stdout:3390 read 0xd705dcf thru 0xd70727d (0x14af bytes) 2024-09-18T00:18:55.688 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:18:55.689 INFO:teuthology.orchestra.run.smithi097.stdout:3391 writesame 0xdea387 thru 0xdf0b1d data_size 0x1(0x6797 bytes) 2024-09-18T00:18:55.736 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is set 2024-09-18T00:18:55.778 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:55.778+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:55.786 INFO:teuthology.orchestra.run.smithi097.stdout:3394 trunc from 0xe24522d to 0x2c9e6a2 2024-09-18T00:18:55.828 INFO:teuthology.orchestra.run.smithi097.stdout:3396 write 0xe50d897 thru 0xe51a6e9 (0xce53 bytes) 2024-09-18T00:18:55.830 INFO:teuthology.orchestra.run.smithi097.stdout:3397 write 0x1d59ae1 thru 0x1d5b38c (0x18ac bytes) 2024-09-18T00:18:55.842 INFO:teuthology.orchestra.run.smithi097.stdout:3401 write 0xe84ccd5 thru 0xe85b2aa (0xe5d6 bytes) 2024-09-18T00:18:55.853 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:55.853 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:55.863 INFO:teuthology.orchestra.run.smithi097.stdout:3402 trunc from 0xe85b2ab to 0x35907e3 2024-09-18T00:18:55.868 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:18:55.878 INFO:teuthology.orchestra.run.smithi097.stdout:3404 read 0x308eba thru 0x315cb3 (0xcdfa bytes) 2024-09-18T00:18:55.878 INFO:teuthology.orchestra.run.smithi097.stdout:3405 read 0x34e8d6a thru 0x34ebfc8 (0x325f bytes) 2024-09-18T00:18:55.879 INFO:teuthology.orchestra.run.smithi097.stdout:3409 read 0x246205b thru 0x246d440 (0xb3e6 bytes) 2024-09-18T00:18:55.880 INFO:teuthology.orchestra.run.smithi097.stdout:3411 read 0xd09318 thru 0xd0e2d4 (0x4fbd bytes) 2024-09-18T00:18:55.910 INFO:teuthology.orchestra.run.smithi097.stdout:3412 write 0xd5ea336 thru 0xd5f9766 (0xf431 bytes) 2024-09-18T00:18:55.914 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:55.914+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:55.920 INFO:teuthology.orchestra.run.smithi097.stdout:3413 read 0xb02140e thru 0xb030043 (0xec36 bytes) 2024-09-18T00:18:55.921 INFO:teuthology.orchestra.run.smithi097.stdout:3416 write 0xc524050 thru 0xc528778 (0x4729 bytes) 2024-09-18T00:18:55.925 INFO:teuthology.orchestra.run.smithi097.stdout:3417 read 0x3b38fa5 thru 0x3b3c222 (0x327e bytes) 2024-09-18T00:18:55.926 INFO:teuthology.orchestra.run.smithi097.stdout:3418 write 0x3629548 thru 0x362d050 (0x3b09 bytes) 2024-09-18T00:18:55.930 INFO:teuthology.orchestra.run.smithi097.stdout:3419 trunc from 0xd5f9767 to 0xd49bba1 2024-09-18T00:18:55.945 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:55.945 INFO:teuthology.orchestra.run.smithi097.stdout:3421 read 0xb69f2d1 thru 0xb6a04cc (0x11fc bytes) 2024-09-18T00:18:55.946 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:55.946 INFO:teuthology.orchestra.run.smithi097.stdout:3423 write 0x80e208b thru 0x80e2d37 (0xcad bytes) 2024-09-18T00:18:55.948 INFO:teuthology.orchestra.run.smithi097.stdout:3425 writesame 0xa12dba7 thru 0xa136218 data_size 0x1(0x8672 bytes) 2024-09-18T00:18:55.977 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:55.970+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:56.039 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:56.039 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:56.055 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:18:56.089 INFO:teuthology.orchestra.run.smithi097.stdout:3427 trunc from 0xd49bba1 to 0x4760f05 2024-09-18T00:18:56.098 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:56.094+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:56.143 INFO:teuthology.orchestra.run.smithi097.stdout:3428 writesame 0xe3c5669 thru 0xe3d152c data_size 0x4(0xbec4 bytes) 2024-09-18T00:18:56.145 INFO:teuthology.orchestra.run.smithi097.stdout:3429 writesame 0xc024ef8 thru 0xc027140 data_size 0x1(0x2249 bytes) 2024-09-18T00:18:56.179 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:56.174+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:56.184 INFO:teuthology.orchestra.run.smithi097.stdout:3430 read 0x205489 thru 0x20ad65 (0x58dd bytes) 2024-09-18T00:18:56.186 INFO:teuthology.orchestra.run.smithi097.stdout:3431 read 0x1d4bfa thru 0x1d56e9 (0xaf0 bytes) 2024-09-18T00:18:56.188 INFO:teuthology.orchestra.run.smithi097.stdout:3432 trunc from 0xe3d152d to 0xdaaa040 2024-09-18T00:18:56.205 INFO:teuthology.orchestra.run.smithi097.stdout:3433 write 0x8534dde thru 0x853e847 (0x9a6a bytes) 2024-09-18T00:18:56.212 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:56.222 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:56.222 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:56.233 INFO:teuthology.orchestra.run.smithi097.stdout:3438 trunc from 0xdaaa040 to 0x55f0573 2024-09-18T00:18:56.236 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:18:56.251 INFO:teuthology.orchestra.run.smithi097.stdout:3439 write 0x35ca0a2 thru 0x35d21cd (0x812c bytes) 2024-09-18T00:18:56.279 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:56.278+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:56.284 INFO:teuthology.orchestra.run.smithi097.stdout:3440 write 0xbaa6cab thru 0xbab07d2 (0x9b28 bytes) 2024-09-18T00:18:56.287 INFO:teuthology.orchestra.run.smithi097.stdout:3441 read 0x439ba22 thru 0x43a34ee (0x7acd bytes) 2024-09-18T00:18:56.295 INFO:teuthology.orchestra.run.smithi097.stdout:3443 trunc from 0xbab07d3 to 0xea27b63 2024-09-18T00:18:56.307 INFO:teuthology.orchestra.run.smithi097.stdout:3444 write 0xa60fa5b thru 0xa6193d9 (0x997f bytes) 2024-09-18T00:18:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:3448 punch from 0x21e5eda to 0x21e5fb9, (0xdf bytes) 2024-09-18T00:18:56.314 INFO:teuthology.orchestra.run.smithi097.stdout:3449 read 0x42ec41f thru 0x42f3386 (0x6f68 bytes) 2024-09-18T00:18:56.315 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:56.315 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:56.316 INFO:teuthology.orchestra.run.smithi097.stdout:3452 write 0x3f9886b thru 0x3f9f525 (0x6cbb bytes) 2024-09-18T00:18:56.320 INFO:teuthology.orchestra.run.smithi097.stdout:3453 write 0xb1d7470 thru 0xb1e44e2 (0xd073 bytes) 2024-09-18T00:18:56.326 INFO:teuthology.orchestra.run.smithi097.stdout:3454 write 0xd56e707 thru 0xd57799a (0x9294 bytes) 2024-09-18T00:18:56.333 INFO:teuthology.orchestra.run.smithi097.stdout:3456 flatten 2024-09-18T00:18:56.374 INFO:teuthology.orchestra.run.smithi097.stdout:3457 trunc from 0xea27b63 to 0x563e13e 2024-09-18T00:18:56.380 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:56.374+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:56.391 INFO:teuthology.orchestra.run.smithi097.stdout:3458 writesame 0x4d51fdd thru 0x4d5bc15 data_size 0x1(0x9c39 bytes) 2024-09-18T00:18:56.403 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:56.404 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:56.418 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:18:56.481 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:56.478+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:56.519 INFO:teuthology.orchestra.run.smithi097.stdout:3459 read 0x452acf thru 0x4609ff (0xdf31 bytes) 2024-09-18T00:18:56.537 INFO:teuthology.orchestra.run.smithi097.stdout:3462 trunc from 0x563e13e to 0xd0f9750 2024-09-18T00:18:56.547 INFO:teuthology.orchestra.run.smithi097.stdout:3463 write 0x4c4b595 thru 0x4c5adfd (0xf869 bytes) 2024-09-18T00:18:56.553 INFO:teuthology.orchestra.run.smithi097.stdout:3464 write 0x47b9e7d thru 0x47c5e68 (0xbfec bytes) 2024-09-18T00:18:56.556 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:56.556 INFO:teuthology.orchestra.run.smithi097.stdout:3466 punch from 0x706d247 to 0x707b6df, (0xe498 bytes) 2024-09-18T00:18:56.556 INFO:teuthology.orchestra.run.smithi097.stdout:3467 write 0x7d852f4 thru 0x7d85c7e (0x98b bytes) 2024-09-18T00:18:56.560 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:56.561 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:56.569 INFO:teuthology.orchestra.run.smithi097.stdout:3469 trunc from 0xd0f9750 to 0x4aac989 2024-09-18T00:18:56.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:18:56.582 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:56.578+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:56.583 INFO:teuthology.orchestra.run.smithi097.stdout:3471 read 0x22bd841 thru 0x22c8c21 (0xb3e1 bytes) 2024-09-18T00:18:56.585 INFO:teuthology.orchestra.run.smithi097.stdout:3472 trunc from 0x4aac989 to 0x4e3418b 2024-09-18T00:18:56.607 INFO:teuthology.orchestra.run.smithi097.stdout:3473 trunc from 0x4e3418b to 0xe03d82a 2024-09-18T00:18:56.617 INFO:teuthology.orchestra.run.smithi097.stdout:3474 write 0x6e289f9 thru 0x6e38865 (0xfe6d bytes) 2024-09-18T00:18:56.631 INFO:teuthology.orchestra.run.smithi097.stdout:3475 trunc from 0xe03d82a to 0x6eb52f9 2024-09-18T00:18:56.658 INFO:teuthology.orchestra.run.smithi097.stdout:3477 write 0xce4f552 thru 0xce5f456 (0xff05 bytes) 2024-09-18T00:18:56.660 INFO:teuthology.orchestra.run.smithi097.stdout:3478 write 0x37d1d96 thru 0x37d76ea (0x5955 bytes) 2024-09-18T00:18:56.666 INFO:teuthology.orchestra.run.smithi097.stdout:3479 writesame 0x774e04f thru 0x774fff2 data_size 0x2d(0x1fa4 bytes) 2024-09-18T00:18:56.667 INFO:teuthology.orchestra.run.smithi097.stdout:3481 read 0x4a0bfca thru 0x4a11e35 (0x5e6c bytes) 2024-09-18T00:18:56.668 INFO:teuthology.orchestra.run.smithi097.stdout:3483 punch from 0x30bf2d4 to 0x30c583b, (0x6567 bytes) 2024-09-18T00:18:56.674 INFO:teuthology.orchestra.run.smithi097.stdout:3484 trunc from 0xce5f457 to 0x87b6079 2024-09-18T00:18:56.682 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:56.682+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:56.684 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:56.684 INFO:teuthology.orchestra.run.smithi097.stdout:3486 write 0x1721247 thru 0x1727bfd (0x69b7 bytes) 2024-09-18T00:18:56.686 INFO:teuthology.orchestra.run.smithi097.stdout:3487 read 0x36b3bac thru 0x36b8875 (0x4cca bytes) 2024-09-18T00:18:56.687 INFO:teuthology.orchestra.run.smithi097.stdout:3488 write 0xbcf618 thru 0xbd31ac (0x3b95 bytes) 2024-09-18T00:18:56.689 INFO:teuthology.orchestra.run.smithi097.stdout:3489 writesame 0x6638de2 thru 0x6647706 data_size 0x17(0xe925 bytes) 2024-09-18T00:18:56.691 INFO:teuthology.orchestra.run.smithi097.stdout:3490 writesame 0x42d00c8 thru 0x42d9e12 data_size 0x1(0x9d4b bytes) 2024-09-18T00:18:56.736 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:56.737 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:56.765 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:18:56.783 INFO:teuthology.orchestra.run.smithi097.stdout:3494 write 0x25ace2c thru 0x25b6ae1 (0x9cb6 bytes) 2024-09-18T00:18:56.784 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:56.778+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:56.787 INFO:teuthology.orchestra.run.smithi097.stdout:3495 write 0x6b8ef26 thru 0x6b9c0a0 (0xd17b bytes) 2024-09-18T00:18:56.791 INFO:teuthology.orchestra.run.smithi097.stdout:3496 read 0x52cf3a1 thru 0x52d6cec (0x794c bytes) 2024-09-18T00:18:56.791 INFO:teuthology.orchestra.run.smithi097.stdout:3497 read 0xa9f3d5 thru 0xaa5aca (0x66f6 bytes) 2024-09-18T00:18:56.792 INFO:teuthology.orchestra.run.smithi097.stdout:3498 writesame 0x86515cd thru 0x8657214 data_size 0x4(0x5c48 bytes) 2024-09-18T00:18:56.798 INFO:teuthology.orchestra.run.smithi097.stdout:3499 punch from 0x30db00e to 0x30db3dc, (0x3ce bytes) 2024-09-18T00:18:56.798 INFO:teuthology.orchestra.run.smithi097.stdout:3500 punch from 0x57e9714 to 0x57f4d30, (0xb61c bytes) 2024-09-18T00:18:56.798 INFO:teuthology.orchestra.run.smithi097.stdout:3505 punch from 0x3f88e5d to 0x3f8e8c0, (0x5a63 bytes) 2024-09-18T00:18:56.798 INFO:teuthology.orchestra.run.smithi097.stdout:3506 read 0x423aaff thru 0x423b96f (0xe71 bytes) 2024-09-18T00:18:56.798 INFO:teuthology.orchestra.run.smithi097.stdout:3507 write 0x5811830 thru 0x581f168 (0xd939 bytes) 2024-09-18T00:18:56.802 INFO:teuthology.orchestra.run.smithi097.stdout:3508 write 0x25c78bf thru 0x25d18ba (0x9ffc bytes) 2024-09-18T00:18:56.807 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:56.807 INFO:teuthology.orchestra.run.smithi097.stdout:3510 write 0x63bed5c thru 0x63c1de6 (0x308b bytes) 2024-09-18T00:18:56.810 INFO:teuthology.orchestra.run.smithi097.stdout:3511 trunc from 0x87b6079 to 0x6f1c3bb 2024-09-18T00:18:56.820 INFO:teuthology.orchestra.run.smithi097.stdout:3512 read 0x1a1a15b thru 0x1a1bfc6 (0x1e6c bytes) 2024-09-18T00:18:56.830 INFO:teuthology.orchestra.run.smithi097.stdout:3516 writesame 0x84bc911 thru 0x84bd818 data_size 0x25(0xf08 bytes) 2024-09-18T00:18:56.831 INFO:teuthology.orchestra.run.smithi097.stdout:3517 write 0x6b4f462 thru 0x6b59da3 (0xa942 bytes) 2024-09-18T00:18:56.833 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:56.833 INFO:teuthology.orchestra.run.smithi097.stdout:3519 punch from 0x10d2a00 to 0x10d8bef, (0x61ef bytes) 2024-09-18T00:18:56.834 INFO:teuthology.orchestra.run.smithi097.stdout:3520 write 0x7b24704 thru 0x7b2c055 (0x7952 bytes) 2024-09-18T00:18:56.836 INFO:teuthology.orchestra.run.smithi097.stdout:3521 read 0x3a53373 thru 0x3a5c30b (0x8f99 bytes) 2024-09-18T00:18:56.838 INFO:teuthology.orchestra.run.smithi097.stdout:3523 trunc from 0x84bd819 to 0x95bc498 2024-09-18T00:18:56.846 INFO:teuthology.orchestra.run.smithi097.stdout:3526 write 0x49d1b12 thru 0x49d976b (0x7c5a bytes) 2024-09-18T00:18:56.848 INFO:teuthology.orchestra.run.smithi097.stdout:3528 read 0x1c08f9b thru 0x1c18b74 (0xfbda bytes) 2024-09-18T00:18:56.863 INFO:teuthology.orchestra.run.smithi097.stdout:3530 write 0xd85bcf1 thru 0xd866872 (0xab82 bytes) 2024-09-18T00:18:56.866 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:56.866 INFO:teuthology.orchestra.run.smithi097.stdout:3533 punch from 0x58c32dc to 0x58ccf1b, (0x9c3f bytes) 2024-09-18T00:18:56.866 INFO:teuthology.orchestra.run.smithi097.stdout:3534 read 0xcc6e6b1 thru 0xcc6f463 (0xdb3 bytes) 2024-09-18T00:18:56.866 INFO:teuthology.orchestra.run.smithi097.stdout:3535 read 0x1fff260 thru 0x200675e (0x74ff bytes) 2024-09-18T00:18:56.867 INFO:teuthology.orchestra.run.smithi097.stdout:3536 punch from 0x4679fae to 0x467b576, (0x15c8 bytes) 2024-09-18T00:18:56.867 INFO:teuthology.orchestra.run.smithi097.stdout:3537 write 0x936d081 thru 0x9378bf7 (0xbb77 bytes) 2024-09-18T00:18:56.877 INFO:teuthology.orchestra.run.smithi097.stdout:3538 trunc from 0xd866873 to 0xc20751f 2024-09-18T00:18:56.884 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:56.878+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:56.913 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:56.913 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:56.928 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:18:56.985 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:56.982+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:57.025 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:18:57.085 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:57.082+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:57.089 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:57.089 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:57.090 INFO:teuthology.orchestra.run.smithi097.stdout:3539 read 0x6d883cf thru 0x6d8f269 (0x6e9b bytes) 2024-09-18T00:18:57.091 INFO:teuthology.orchestra.run.smithi097.stdout:3540 write 0x2db7ea3 thru 0x2dbf8ec (0x7a4a bytes) 2024-09-18T00:18:57.094 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:57.096 INFO:teuthology.orchestra.run.smithi097.stdout:3542 trunc from 0xc20751f to 0xa738469 2024-09-18T00:18:57.110 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:18:57.110 INFO:teuthology.orchestra.run.smithi097.stdout:3543 trunc from 0xa738469 to 0xc3819a7 2024-09-18T00:18:57.127 INFO:teuthology.orchestra.run.smithi097.stdout:3544 write 0xd7abc7b thru 0xd7b58be (0x9c44 bytes) 2024-09-18T00:18:57.129 INFO:teuthology.orchestra.run.smithi097.stdout:3546 write 0x2287839 thru 0x228ef40 (0x7708 bytes) 2024-09-18T00:18:57.135 INFO:teuthology.orchestra.run.smithi097.stdout:3547 punch from 0x785d51f to 0x7869739, (0xc21a bytes) 2024-09-18T00:18:57.135 INFO:teuthology.orchestra.run.smithi097.stdout:3548 read 0xcf0dc78 thru 0xcf1db15 (0xfe9e bytes) 2024-09-18T00:18:57.136 INFO:teuthology.orchestra.run.smithi097.stdout:3549 read 0x14f73a3 thru 0x14ff3b4 (0x8012 bytes) 2024-09-18T00:18:57.136 INFO:teuthology.orchestra.run.smithi097.stdout:3550 punch from 0xd4dcd75 to 0xd4e564d, (0x88d8 bytes) 2024-09-18T00:18:57.137 INFO:teuthology.orchestra.run.smithi097.stdout:3551 read 0x6a17c9d thru 0x6a27c19 (0xff7d bytes) 2024-09-18T00:18:57.147 INFO:teuthology.orchestra.run.smithi097.stdout:3552 write 0xe8dcc1f thru 0xe8e9de9 (0xd1cb bytes) 2024-09-18T00:18:57.149 INFO:teuthology.orchestra.run.smithi097.stdout:3553 write 0x9dc02f5 thru 0x9dcddb9 (0xdac5 bytes) 2024-09-18T00:18:57.152 INFO:teuthology.orchestra.run.smithi097.stdout:3554 write 0x1b2b35d thru 0x1b355d0 (0xa274 bytes) 2024-09-18T00:18:57.155 INFO:teuthology.orchestra.run.smithi097.stdout:3555 writesame 0x6a75b35 thru 0x6a7c8f1 data_size 0x1(0x6dbd bytes) 2024-09-18T00:18:57.186 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:57.186 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:57.186 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:57.182+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:57.187 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:18:57.204 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:57.204 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:57.208 INFO:teuthology.orchestra.run.smithi097.stdout:3557 read 0x834c4ad thru 0x83576d1 (0xb225 bytes) 2024-09-18T00:18:57.216 INFO:teuthology.orchestra.run.smithi097.stdout:3558 writesame 0xea2fc2f thru 0xea38f35 data_size 0x13(0x9307 bytes) 2024-09-18T00:18:57.217 INFO:teuthology.orchestra.run.smithi097.stdout:3559 write 0x61e53ce thru 0x61f4d33 (0xf966 bytes) 2024-09-18T00:18:57.220 INFO:teuthology.orchestra.run.smithi097.stdout:3560 write 0x4df1db6 thru 0x4df41ff (0x244a bytes) 2024-09-18T00:18:57.222 INFO:teuthology.orchestra.run.smithi097.stdout:3561 write 0xab3fcfa thru 0xab41e25 (0x212c bytes) 2024-09-18T00:18:57.223 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:57.223 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:57.224 INFO:teuthology.orchestra.run.smithi097.stdout:3562 write 0x82ea97b thru 0x82f6670 (0xbcf6 bytes) 2024-09-18T00:18:57.228 INFO:teuthology.orchestra.run.smithi097.stdout:3563 read 0xa4e3cba thru 0xa4f062d (0xc974 bytes) 2024-09-18T00:18:57.232 INFO:teuthology.orchestra.run.smithi097.stdout:3564 trunc from 0xea38f36 to 0xcef9c6a 2024-09-18T00:18:57.242 INFO:teuthology.orchestra.run.smithi097.stdout:3566 write 0x6a9855d thru 0x6a9d9e9 (0x548d bytes) 2024-09-18T00:18:57.244 INFO:teuthology.orchestra.run.smithi097.stdout:3567 read 0xa53e69f thru 0xa54d6a3 (0xf005 bytes) 2024-09-18T00:18:57.245 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:57.245 INFO:teuthology.orchestra.run.smithi097.stdout:3569 write 0x48dcae2 thru 0x48ec288 (0xf7a7 bytes) 2024-09-18T00:18:57.245 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:57.245 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:57.245 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:57.245 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:57.245 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:57.247 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:57.247 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:57.247 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-18T00:18:57.249 INFO:teuthology.orchestra.run.smithi097.stdout:3570 punch from 0x355d261 to 0x356b915, (0xe6b4 bytes) 2024-09-18T00:18:57.257 INFO:teuthology.orchestra.run.smithi097.stdout:3571 trunc from 0xcef9c6a to 0x896c2a5 2024-09-18T00:18:57.269 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:57.270 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:18:57.270 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:57.271 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:57.282 INFO:teuthology.orchestra.run.smithi097.stdout:3573 write 0xcd60ec7 thru 0xcd62e7e (0x1fb8 bytes) 2024-09-18T00:18:57.283 INFO:teuthology.orchestra.run.smithi097.stdout:3574 write 0x60ceca5 thru 0x60d620b (0x7567 bytes) 2024-09-18T00:18:57.285 INFO:teuthology.orchestra.run.smithi097.stdout:3575 punch from 0x5c79bde to 0x5c864c7, (0xc8e9 bytes) 2024-09-18T00:18:57.285 INFO:teuthology.orchestra.run.smithi097.stdout:3576 write 0x2ae94ec thru 0x2af0f2a (0x7a3f bytes) 2024-09-18T00:18:57.286 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:57.282+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:57.287 INFO:teuthology.orchestra.run.smithi097.stdout:3578 writesame 0x68d5e3f thru 0x68e31b1 data_size 0x25(0xd373 bytes) 2024-09-18T00:18:57.290 INFO:teuthology.orchestra.run.smithi097.stdout:3580 read 0x19ea3d7 thru 0x19f64d1 (0xc0fb bytes) 2024-09-18T00:18:57.291 INFO:teuthology.orchestra.run.smithi097.stdout:3581 write 0x89c2913 thru 0x89c3534 (0xc22 bytes) 2024-09-18T00:18:57.292 INFO:teuthology.orchestra.run.smithi097.stdout:3582 writesame 0xc2ae2aa thru 0xc2bdd68 data_size 0x1(0xfabf bytes) 2024-09-18T00:18:57.295 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:57.295 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:57.298 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:18:57.299 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:18:57.387 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:57.386+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:57.391 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:57.391 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:57.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:57.391 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:57.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.392 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:5722 3.4 e41/23 3:20381d86:::rbd_data.112a3a39fc45.000000000000001d:head v 41'107, mlcod=40'105)", 2024-09-18T00:18:57.392 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:18:57.392 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.054798140999999, 2024-09-18T00:18:57.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01185172, 2024-09-18T00:18:57.392 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.392 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.392 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:57.392 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:18:57.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:18:57.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335497+0000", 2024-09-18T00:18:57.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.505e-06 2024-09-18T00:18:57.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335945+0000", 2024-09-18T00:18:57.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000447687 2024-09-18T00:18:57.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335961+0000", 2024-09-18T00:18:57.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6430999999999999e-05 2024-09-18T00:18:57.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335969+0000", 2024-09-18T00:18:57.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:18:57.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.394 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.394 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336030+0000", 2024-09-18T00:18:57.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1255999999999998e-05 2024-09-18T00:18:57.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336061+0000", 2024-09-18T00:18:57.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0579e-05 2024-09-18T00:18:57.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:57.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347296+0000", 2024-09-18T00:18:57.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011235663 2024-09-18T00:18:57.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347347+0000", 2024-09-18T00:18:57.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0914000000000001e-05 2024-09-18T00:18:57.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.396 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.396 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11457 3.0 3:0214e7b3:::rbd_data.112ab1fa102a.0000000000000150:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:57.396 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:18:57.396 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.130432203000002, 2024-09-18T00:18:57.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028499165, 2024-09-18T00:18:57.396 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.396 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.396 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:57.397 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.397 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.397 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.397 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11457 2024-09-18T00:18:57.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.397 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:18:57.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:18:57.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259865+0000", 2024-09-18T00:18:57.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7230000000000002e-06 2024-09-18T00:18:57.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259883+0000", 2024-09-18T00:18:57.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8067000000000002e-05 2024-09-18T00:18:57.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259924+0000", 2024-09-18T00:18:57.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0550999999999999e-05 2024-09-18T00:18:57.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259943+0000", 2024-09-18T00:18:57.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:18:57.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260054+0000", 2024-09-18T00:18:57.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011075 2024-09-18T00:18:57.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260138+0000", 2024-09-18T00:18:57.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4549000000000007e-05 2024-09-18T00:18:57.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:57.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.287926+0000", 2024-09-18T00:18:57.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027787642000000001 2024-09-18T00:18:57.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:57.400 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288090+0000", 2024-09-18T00:18:57.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016402499999999999 2024-09-18T00:18:57.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.400 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.400 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288361+0000", 2024-09-18T00:18:57.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027097800000000001 2024-09-18T00:18:57.401 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.401 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.401 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.401 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18661 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.401 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:18:57.401 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4556643380000001, 2024-09-18T00:18:57.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020008037999999999, 2024-09-18T00:18:57.401 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.401 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.401 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:57.402 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.402 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.402 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.402 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18661 2024-09-18T00:18:57.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.402 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:18:57.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:18:57.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934633+0000", 2024-09-18T00:18:57.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8e-06 2024-09-18T00:18:57.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935511+0000", 2024-09-18T00:18:57.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087785700000000001 2024-09-18T00:18:57.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935561+0000", 2024-09-18T00:18:57.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9472000000000003e-05 2024-09-18T00:18:57.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935586+0000", 2024-09-18T00:18:57.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5616e-05 2024-09-18T00:18:57.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935838+0000", 2024-09-18T00:18:57.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025202699999999999 2024-09-18T00:18:57.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.936023+0000", 2024-09-18T00:18:57.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018480300000000001 2024-09-18T00:18:57.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:57.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953545+0000", 2024-09-18T00:18:57.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017522157 2024-09-18T00:18:57.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:57.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953723+0000", 2024-09-18T00:18:57.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017753800000000001 2024-09-18T00:18:57.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.406 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.406 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954637+0000", 2024-09-18T00:18:57.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091476800000000005 2024-09-18T00:18:57.406 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.406 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.406 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.406 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18662 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.406 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:18:57.406 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.4546594009999998, 2024-09-18T00:18:57.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019388843999999999, 2024-09-18T00:18:57.406 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.406 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.407 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:57.407 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.407 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.407 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.407 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18662 2024-09-18T00:18:57.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.407 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:18:57.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:18:57.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935636+0000", 2024-09-18T00:18:57.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.663e-06 2024-09-18T00:18:57.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935646+0000", 2024-09-18T00:18:57.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000006e-06 2024-09-18T00:18:57.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935672+0000", 2024-09-18T00:18:57.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5868999999999999e-05 2024-09-18T00:18:57.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935692+0000", 2024-09-18T00:18:57.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0604000000000001e-05 2024-09-18T00:18:57.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950738+0000", 2024-09-18T00:18:57.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015045868 2024-09-18T00:18:57.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950992+0000", 2024-09-18T00:18:57.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025368400000000001 2024-09-18T00:18:57.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:57.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954674+0000", 2024-09-18T00:18:57.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003682154 2024-09-18T00:18:57.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:57.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954806+0000", 2024-09-18T00:18:57.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013252000000000001 2024-09-18T00:18:57.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.955023+0000", 2024-09-18T00:18:57.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021681200000000001 2024-09-18T00:18:57.411 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.411 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.411 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.411 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19382 3.4 3:22f09d3a:::rbd_data.112a7fe29ab4.000000000000015a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.411 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:18:57.411 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.7682493510000001, 2024-09-18T00:18:57.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075119302999999998, 2024-09-18T00:18:57.411 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.412 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.412 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:57.412 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.412 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.412 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.412 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19382 2024-09-18T00:18:57.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.412 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:18:57.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:18:57.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622045+0000", 2024-09-18T00:18:57.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0020000000000001e-06 2024-09-18T00:18:57.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622053+0000", 2024-09-18T00:18:57.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2679999999999997e-06 2024-09-18T00:18:57.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622062+0000", 2024-09-18T00:18:57.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3749999999999992e-06 2024-09-18T00:18:57.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622068+0000", 2024-09-18T00:18:57.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5729999999999998e-06 2024-09-18T00:18:57.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622118+0000", 2024-09-18T00:18:57.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0696999999999999e-05 2024-09-18T00:18:57.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622222+0000", 2024-09-18T00:18:57.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103286 2024-09-18T00:18:57.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:57.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697061+0000", 2024-09-18T00:18:57.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074839141999999997 2024-09-18T00:18:57.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:57.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697102+0000", 2024-09-18T00:18:57.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1513e-05 2024-09-18T00:18:57.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697164+0000", 2024-09-18T00:18:57.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1446999999999996e-05 2024-09-18T00:18:57.416 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.416 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.416 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.416 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19896 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.416 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:18:57.416 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.4488652129999999, 2024-09-18T00:18:57.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097478370000000005, 2024-09-18T00:18:57.417 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.417 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.417 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:57.417 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.417 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.417 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.417 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19896 2024-09-18T00:18:57.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.417 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:18:57.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:18:57.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:18:57.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6000000000000002e-07 2024-09-18T00:18:57.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941434+0000", 2024-09-18T00:18:57.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4750000000000004e-06 2024-09-18T00:18:57.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941452+0000", 2024-09-18T00:18:57.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8603000000000001e-05 2024-09-18T00:18:57.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941461+0000", 2024-09-18T00:18:57.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5199999999999997e-06 2024-09-18T00:18:57.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.947391+0000", 2024-09-18T00:18:57.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059302399999999998 2024-09-18T00:18:57.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.947479+0000", 2024-09-18T00:18:57.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8002000000000002e-05 2024-09-18T00:18:57.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.951176+0000", 2024-09-18T00:18:57.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036971370000000001 2024-09-18T00:18:57.420 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.420 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.420 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.420 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19907 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.421 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941771+0000", 2024-09-18T00:18:57.421 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.4485225590000002, 2024-09-18T00:18:57.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0087132449999999997, 2024-09-18T00:18:57.421 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.421 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.421 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:57.421 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.421 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.421 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.421 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19907 2024-09-18T00:18:57.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.421 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941771+0000", 2024-09-18T00:18:57.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941771+0000", 2024-09-18T00:18:57.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941772+0000", 2024-09-18T00:18:57.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9999999999999998e-07 2024-09-18T00:18:57.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941781+0000", 2024-09-18T00:18:57.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0210000000000005e-06 2024-09-18T00:18:57.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941792+0000", 2024-09-18T00:18:57.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0925e-05 2024-09-18T00:18:57.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941798+0000", 2024-09-18T00:18:57.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6749999999999996e-06 2024-09-18T00:18:57.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950307+0000", 2024-09-18T00:18:57.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0085082860000000003 2024-09-18T00:18:57.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950390+0000", 2024-09-18T00:18:57.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3542999999999996e-05 2024-09-18T00:18:57.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950484+0000", 2024-09-18T00:18:57.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4295000000000004e-05 2024-09-18T00:18:57.424 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.424 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.425 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.425 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19908 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.425 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941809+0000", 2024-09-18T00:18:57.425 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.4484846089999999, 2024-09-18T00:18:57.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0088554250000000001, 2024-09-18T00:18:57.425 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.425 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.425 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:57.425 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.425 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.425 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.425 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19908 2024-09-18T00:18:57.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.426 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941809+0000", 2024-09-18T00:18:57.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941809+0000", 2024-09-18T00:18:57.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941810+0000", 2024-09-18T00:18:57.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2600000000000002e-07 2024-09-18T00:18:57.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941823+0000", 2024-09-18T00:18:57.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2784e-05 2024-09-18T00:18:57.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941832+0000", 2024-09-18T00:18:57.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6229999999999992e-06 2024-09-18T00:18:57.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941839+0000", 2024-09-18T00:18:57.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1520000000000003e-06 2024-09-18T00:18:57.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950515+0000", 2024-09-18T00:18:57.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0086754880000000003 2024-09-18T00:18:57.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950581+0000", 2024-09-18T00:18:57.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6398999999999999e-05 2024-09-18T00:18:57.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950665+0000", 2024-09-18T00:18:57.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3252999999999997e-05 2024-09-18T00:18:57.429 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.429 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.429 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.432 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19909 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.432 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941849+0000", 2024-09-18T00:18:57.432 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.4484443200000001, 2024-09-18T00:18:57.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0090299730000000002, 2024-09-18T00:18:57.432 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.432 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.433 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:57.433 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.433 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.433 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.433 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19909 2024-09-18T00:18:57.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.433 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941849+0000", 2024-09-18T00:18:57.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941849+0000", 2024-09-18T00:18:57.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941850+0000", 2024-09-18T00:18:57.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4899999999999995e-07 2024-09-18T00:18:57.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941853+0000", 2024-09-18T00:18:57.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.371e-06 2024-09-18T00:18:57.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941862+0000", 2024-09-18T00:18:57.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7749999999999998e-06 2024-09-18T00:18:57.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941869+0000", 2024-09-18T00:18:57.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6320000000000002e-06 2024-09-18T00:18:57.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950702+0000", 2024-09-18T00:18:57.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0088332930000000007 2024-09-18T00:18:57.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950778+0000", 2024-09-18T00:18:57.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5983999999999996e-05 2024-09-18T00:18:57.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950879+0000", 2024-09-18T00:18:57.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101269 2024-09-18T00:18:57.436 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.436 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.436 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.436 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19910 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.436 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941878+0000", 2024-09-18T00:18:57.437 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.4484154839999999, 2024-09-18T00:18:57.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092861479999999993, 2024-09-18T00:18:57.437 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.437 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.437 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:57.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.437 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19910 2024-09-18T00:18:57.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.437 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941878+0000", 2024-09-18T00:18:57.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941878+0000", 2024-09-18T00:18:57.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941879+0000", 2024-09-18T00:18:57.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6199999999999998e-07 2024-09-18T00:18:57.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941881+0000", 2024-09-18T00:18:57.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6390000000000001e-06 2024-09-18T00:18:57.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941890+0000", 2024-09-18T00:18:57.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4999999999999999e-06 2024-09-18T00:18:57.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941896+0000", 2024-09-18T00:18:57.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1299999999999998e-06 2024-09-18T00:18:57.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950898+0000", 2024-09-18T00:18:57.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0090014499999999994 2024-09-18T00:18:57.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950984+0000", 2024-09-18T00:18:57.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6933000000000003e-05 2024-09-18T00:18:57.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.951164+0000", 2024-09-18T00:18:57.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000179934 2024-09-18T00:18:57.440 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.440 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.440 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.441 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20240 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.441 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155573+0000", 2024-09-18T00:18:57.441 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.234720636, 2024-09-18T00:18:57.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0088073579999999995, 2024-09-18T00:18:57.441 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.441 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.441 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:57.441 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.441 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.441 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.441 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20240 2024-09-18T00:18:57.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.442 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155573+0000", 2024-09-18T00:18:57.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155573+0000", 2024-09-18T00:18:57.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155574+0000", 2024-09-18T00:18:57.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3799999999999998e-07 2024-09-18T00:18:57.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155576+0000", 2024-09-18T00:18:57.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.125e-06 2024-09-18T00:18:57.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155582+0000", 2024-09-18T00:18:57.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4819999999999999e-06 2024-09-18T00:18:57.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155587+0000", 2024-09-18T00:18:57.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.831e-06 2024-09-18T00:18:57.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.163695+0000", 2024-09-18T00:18:57.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0081077949999999992 2024-09-18T00:18:57.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.163971+0000", 2024-09-18T00:18:57.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027655399999999998 2024-09-18T00:18:57.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164380+0000", 2024-09-18T00:18:57.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000409133 2024-09-18T00:18:57.445 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.445 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.445 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.445 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20241 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.445 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155593+0000", 2024-09-18T00:18:57.445 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.2347003540000001, 2024-09-18T00:18:57.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0090827070000000006, 2024-09-18T00:18:57.445 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.445 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.445 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:57.445 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.446 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20241 2024-09-18T00:18:57.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.446 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155593+0000", 2024-09-18T00:18:57.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155593+0000", 2024-09-18T00:18:57.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155594+0000", 2024-09-18T00:18:57.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9700000000000002e-07 2024-09-18T00:18:57.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155596+0000", 2024-09-18T00:18:57.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1210000000000001e-06 2024-09-18T00:18:57.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155602+0000", 2024-09-18T00:18:57.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.144e-06 2024-09-18T00:18:57.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155607+0000", 2024-09-18T00:18:57.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7539999999999997e-06 2024-09-18T00:18:57.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164450+0000", 2024-09-18T00:18:57.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0088436079999999993 2024-09-18T00:18:57.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164537+0000", 2024-09-18T00:18:57.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6329000000000006e-05 2024-09-18T00:18:57.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164676+0000", 2024-09-18T00:18:57.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013935400000000001 2024-09-18T00:18:57.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.449 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.449 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20242 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.449 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155613+0000", 2024-09-18T00:18:57.449 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.2346806940000001, 2024-09-18T00:18:57.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00933551, 2024-09-18T00:18:57.450 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.450 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.450 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:57.450 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.450 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.450 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.450 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20242 2024-09-18T00:18:57.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.450 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155613+0000", 2024-09-18T00:18:57.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155613+0000", 2024-09-18T00:18:57.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155613+0000", 2024-09-18T00:18:57.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1e-07 2024-09-18T00:18:57.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155615+0000", 2024-09-18T00:18:57.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1110000000000002e-06 2024-09-18T00:18:57.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155621+0000", 2024-09-18T00:18:57.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:18:57.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155628+0000", 2024-09-18T00:18:57.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7259999999999996e-06 2024-09-18T00:18:57.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164725+0000", 2024-09-18T00:18:57.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0090965009999999999 2024-09-18T00:18:57.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164832+0000", 2024-09-18T00:18:57.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106956 2024-09-18T00:18:57.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164949+0000", 2024-09-18T00:18:57.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011703 2024-09-18T00:18:57.453 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.453 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.453 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.454 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20243 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.454 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155635+0000", 2024-09-18T00:18:57.454 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.2346586510000002, 2024-09-18T00:18:57.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.009569358, 2024-09-18T00:18:57.454 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.454 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.454 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:57.454 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.454 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.454 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.454 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20243 2024-09-18T00:18:57.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.454 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155635+0000", 2024-09-18T00:18:57.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155635+0000", 2024-09-18T00:18:57.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155635+0000", 2024-09-18T00:18:57.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8000000000000001e-07 2024-09-18T00:18:57.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155639+0000", 2024-09-18T00:18:57.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0809999999999998e-06 2024-09-18T00:18:57.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155645+0000", 2024-09-18T00:18:57.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0469999999999999e-06 2024-09-18T00:18:57.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155649+0000", 2024-09-18T00:18:57.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7419999999999997e-06 2024-09-18T00:18:57.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164990+0000", 2024-09-18T00:18:57.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093401370000000001 2024-09-18T00:18:57.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165066+0000", 2024-09-18T00:18:57.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6861999999999997e-05 2024-09-18T00:18:57.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165204+0000", 2024-09-18T00:18:57.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138109 2024-09-18T00:18:57.457 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.458 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.458 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.458 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20244 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.458 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155679+0000", 2024-09-18T00:18:57.458 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.2346146949999999, 2024-09-18T00:18:57.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097580089999999998, 2024-09-18T00:18:57.458 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.458 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.458 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:57.458 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.458 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.458 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.458 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20244 2024-09-18T00:18:57.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.459 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155679+0000", 2024-09-18T00:18:57.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155679+0000", 2024-09-18T00:18:57.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155680+0000", 2024-09-18T00:18:57.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0900000000000002e-07 2024-09-18T00:18:57.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155682+0000", 2024-09-18T00:18:57.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6589999999999999e-06 2024-09-18T00:18:57.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155690+0000", 2024-09-18T00:18:57.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3139999999999998e-06 2024-09-18T00:18:57.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155695+0000", 2024-09-18T00:18:57.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0950000000000003e-06 2024-09-18T00:18:57.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165243+0000", 2024-09-18T00:18:57.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095483519999999995 2024-09-18T00:18:57.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165321+0000", 2024-09-18T00:18:57.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8219000000000004e-05 2024-09-18T00:18:57.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165437+0000", 2024-09-18T00:18:57.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011586099999999999 2024-09-18T00:18:57.462 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.462 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.462 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.462 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20245 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.462 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:18:57.462 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.234591022, 2024-09-18T00:18:57.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099981159999999996, 2024-09-18T00:18:57.462 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.462 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.462 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:57.463 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.463 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.463 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.463 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20245 2024-09-18T00:18:57.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.463 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:18:57.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:18:57.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:18:57.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.65e-07 2024-09-18T00:18:57.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155711+0000", 2024-09-18T00:18:57.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:18:57.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155720+0000", 2024-09-18T00:18:57.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6580000000000008e-06 2024-09-18T00:18:57.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155725+0000", 2024-09-18T00:18:57.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3430000000000003e-06 2024-09-18T00:18:57.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165479+0000", 2024-09-18T00:18:57.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097538239999999995 2024-09-18T00:18:57.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165556+0000", 2024-09-18T00:18:57.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6724000000000006e-05 2024-09-18T00:18:57.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165701+0000", 2024-09-18T00:18:57.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145129 2024-09-18T00:18:57.466 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.466 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.466 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.466 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20246 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.466 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:18:57.466 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.2345611590000001, 2024-09-18T00:18:57.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01018784, 2024-09-18T00:18:57.467 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.467 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.467 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:57.467 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.467 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.467 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.467 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20246 2024-09-18T00:18:57.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.467 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:18:57.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:18:57.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:18:57.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2599999999999998e-07 2024-09-18T00:18:57.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155735+0000", 2024-09-18T00:18:57.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2699999999999999e-06 2024-09-18T00:18:57.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155744+0000", 2024-09-18T00:18:57.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8629999999999997e-06 2024-09-18T00:18:57.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155749+0000", 2024-09-18T00:18:57.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0359999999999997e-06 2024-09-18T00:18:57.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165748+0000", 2024-09-18T00:18:57.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099991449999999992 2024-09-18T00:18:57.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165819+0000", 2024-09-18T00:18:57.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0964999999999996e-05 2024-09-18T00:18:57.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165920+0000", 2024-09-18T00:18:57.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101135 2024-09-18T00:18:57.470 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.470 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.471 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.471 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20247 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.471 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:18:57.471 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.2345378170000001, 2024-09-18T00:18:57.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010443987, 2024-09-18T00:18:57.471 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.471 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.471 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:57.471 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.471 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.471 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.471 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20247 2024-09-18T00:18:57.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.472 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:18:57.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:18:57.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:18:57.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1899999999999998e-07 2024-09-18T00:18:57.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155760+0000", 2024-09-18T00:18:57.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6459999999999999e-06 2024-09-18T00:18:57.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155767+0000", 2024-09-18T00:18:57.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0350000000000004e-06 2024-09-18T00:18:57.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155774+0000", 2024-09-18T00:18:57.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:18:57.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165960+0000", 2024-09-18T00:18:57.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186051999999999 2024-09-18T00:18:57.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166048+0000", 2024-09-18T00:18:57.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8848e-05 2024-09-18T00:18:57.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166200+0000", 2024-09-18T00:18:57.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015147000000000001 2024-09-18T00:18:57.475 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.475 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.475 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.475 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20248 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.475 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:18:57.475 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.2345126529999999, 2024-09-18T00:18:57.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010617721, 2024-09-18T00:18:57.475 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.475 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.476 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:57.476 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.476 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.476 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.476 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20248 2024-09-18T00:18:57.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.476 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:18:57.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:18:57.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155782+0000", 2024-09-18T00:18:57.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.84e-07 2024-09-18T00:18:57.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155784+0000", 2024-09-18T00:18:57.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2800000000000002e-06 2024-09-18T00:18:57.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155790+0000", 2024-09-18T00:18:57.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5679999999999996e-06 2024-09-18T00:18:57.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155795+0000", 2024-09-18T00:18:57.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6929999999999998e-06 2024-09-18T00:18:57.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166245+0000", 2024-09-18T00:18:57.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010449847999999999 2024-09-18T00:18:57.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166307+0000", 2024-09-18T00:18:57.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2287999999999997e-05 2024-09-18T00:18:57.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166399+0000", 2024-09-18T00:18:57.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.166e-05 2024-09-18T00:18:57.479 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.479 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.479 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.479 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20249 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:57.479 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:18:57.480 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 3.2344922770000002, 2024-09-18T00:18:57.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010811983000000001, 2024-09-18T00:18:57.480 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.480 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.480 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:57.480 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.480 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.480 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.480 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20249 2024-09-18T00:18:57.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.480 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:18:57.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:18:57.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155802+0000", 2024-09-18T00:18:57.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:18:57.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155804+0000", 2024-09-18T00:18:57.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:18:57.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155810+0000", 2024-09-18T00:18:57.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0580000000000003e-06 2024-09-18T00:18:57.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155817+0000", 2024-09-18T00:18:57.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3980000000000004e-06 2024-09-18T00:18:57.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166451+0000", 2024-09-18T00:18:57.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010634825000000001 2024-09-18T00:18:57.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166520+0000", 2024-09-18T00:18:57.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8317000000000003e-05 2024-09-18T00:18:57.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166613+0000", 2024-09-18T00:18:57.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3801000000000005e-05 2024-09-18T00:18:57.483 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.483 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.483 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.483 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.484 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.484 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:57.484 INFO:teuthology.orchestra.run.smithi097.stdout:3583 writesame 0xdf8002e thru 0xdf8dd88 data_size 0xd(0xdd5b bytes) 2024-09-18T00:18:57.485 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:57.485 INFO:teuthology.orchestra.run.smithi097.stdout:3587 trunc from 0xdf8dd89 to 0xe886e29 2024-09-18T00:18:57.485 INFO:teuthology.orchestra.run.smithi097.stdout:3591 read 0xa2bc554 thru 0xa2c6f74 (0xaa21 bytes) 2024-09-18T00:18:57.485 INFO:teuthology.orchestra.run.smithi097.stdout:3592 read 0x1e7772e thru 0x1e86f0b (0xf7de bytes) 2024-09-18T00:18:57.486 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:18:57.488 INFO:teuthology.orchestra.run.smithi097.stdout:3593 writesame 0x282b495 thru 0x282beea data_size 0x12(0xa56 bytes) 2024-09-18T00:18:57.488 INFO:teuthology.orchestra.run.smithi097.stdout:3594 write 0xb5c190 thru 0xb60d72 (0x4be3 bytes) 2024-09-18T00:18:57.488 INFO:teuthology.orchestra.run.smithi097.stdout:3595 read 0x954e14d thru 0x954e22f (0xe3 bytes) 2024-09-18T00:18:57.488 INFO:teuthology.orchestra.run.smithi097.stdout:3596 read 0x9f226e0 thru 0x9f2959e (0x6ebf bytes) 2024-09-18T00:18:57.488 INFO:teuthology.orchestra.run.smithi097.stdout:3597 write 0x8a1971d thru 0x8a22ceb (0x95cf bytes) 2024-09-18T00:18:57.488 INFO:teuthology.orchestra.run.smithi097.stdout:3598 trunc from 0xe886e29 to 0xb491e23 2024-09-18T00:18:57.488 INFO:teuthology.orchestra.run.smithi097.stdout:3601 trunc from 0xb491e23 to 0x86f5e23 2024-09-18T00:18:57.488 INFO:teuthology.orchestra.run.smithi097.stdout:3602 read 0x4d047b4 thru 0x4d0a18b (0x59d8 bytes) 2024-09-18T00:18:57.488 INFO:teuthology.orchestra.run.smithi097.stdout:3603 writesame 0x4be6d6f thru 0x4beb223 data_size 0x29(0x44b5 bytes) 2024-09-18T00:18:57.488 INFO:teuthology.orchestra.run.smithi097.stdout:3604 read 0x5e5d292 thru 0x5e60f76 (0x3ce5 bytes) 2024-09-18T00:18:57.488 INFO:teuthology.orchestra.run.smithi097.stdout:3605 read 0x254b215 thru 0x254db09 (0x28f5 bytes) 2024-09-18T00:18:57.488 INFO:teuthology.orchestra.run.smithi097.stdout:3606 write 0xbabb6b4 thru 0xbac021a (0x4b67 bytes) 2024-09-18T00:18:57.488 INFO:teuthology.orchestra.run.smithi097.stdout:3608 write 0xe7155ea thru 0xe725257 (0xfc6e bytes) 2024-09-18T00:18:57.489 INFO:teuthology.orchestra.run.smithi097.stdout:3611 write 0x3505b02 thru 0x350c966 (0x6e65 bytes) 2024-09-18T00:18:57.489 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:57.489 INFO:teuthology.orchestra.run.smithi097.stdout:3613 read 0xbcade74 thru 0xbcb2ab1 (0x4c3e bytes) 2024-09-18T00:18:57.489 INFO:teuthology.orchestra.run.smithi097.stdout:3615 write 0xbc24296 thru 0xbc2eb75 (0xa8e0 bytes) 2024-09-18T00:18:57.489 INFO:teuthology.orchestra.run.smithi097.stdout:3617 writesame 0xd8567a thru 0xd8a64f data_size 0xb(0x4fd6 bytes) 2024-09-18T00:18:57.490 INFO:teuthology.orchestra.run.smithi097.stdout:3618 punch from 0x4202151 to 0x420613a, (0x3fe9 bytes) 2024-09-18T00:18:57.490 INFO:teuthology.orchestra.run.smithi097.stdout:3620 write 0x1371937 thru 0x1379781 (0x7e4b bytes) 2024-09-18T00:18:57.490 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:57.486+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:57.493 INFO:teuthology.orchestra.run.smithi097.stdout:3621 read 0x7022057 thru 0x7022a90 (0xa3a bytes) 2024-09-18T00:18:57.493 INFO:teuthology.orchestra.run.smithi097.stdout:3622 read 0xad79d27 thru 0xad83afb (0x9dd5 bytes) 2024-09-18T00:18:57.494 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:57.494 INFO:teuthology.orchestra.run.smithi097.stdout:3624 read 0x17f2aff thru 0x17fb778 (0x8c7a bytes) 2024-09-18T00:18:57.494 INFO:teuthology.orchestra.run.smithi097.stdout:3625 write 0xae51d2e thru 0xae5f2f9 (0xd5cc bytes) 2024-09-18T00:18:57.498 INFO:teuthology.orchestra.run.smithi097.stdout:3626 writesame 0xb423ff thru 0xb4c79f data_size 0x1(0xa3a1 bytes) 2024-09-18T00:18:57.537 INFO:teuthology.orchestra.run.smithi097.stdout:3627 write 0x3de302 thru 0x3e1310 (0x300f bytes) 2024-09-18T00:18:57.539 INFO:teuthology.orchestra.run.smithi097.stdout:3628 write 0x1c0d3 thru 0x22eb5 (0x6de3 bytes) 2024-09-18T00:18:57.541 INFO:teuthology.orchestra.run.smithi097.stdout:3629 writesame 0x9644faa thru 0x964abd9 data_size 0xa(0x5c30 bytes) 2024-09-18T00:18:57.556 INFO:teuthology.orchestra.run.smithi097.stdout:3631 trunc from 0xe725258 to 0x8d34331 2024-09-18T00:18:57.567 INFO:teuthology.orchestra.run.smithi097.stdout:3633 write 0x16363bb thru 0x163beda (0x5b20 bytes) 2024-09-18T00:18:57.569 INFO:teuthology.orchestra.run.smithi097.stdout:3634 punch from 0x390aba to 0x398130, (0x7676 bytes) 2024-09-18T00:18:57.569 INFO:teuthology.orchestra.run.smithi097.stdout:3635 write 0x71b7e83 thru 0x71c185e (0x99dc bytes) 2024-09-18T00:18:57.575 INFO:teuthology.orchestra.run.smithi097.stdout:3637 read 0x582b32d thru 0x58371d8 (0xbeac bytes) 2024-09-18T00:18:57.575 INFO:teuthology.orchestra.run.smithi097.stdout:3638 punch from 0x8c0aa0 to 0x8c336e, (0x28ce bytes) 2024-09-18T00:18:57.575 INFO:teuthology.orchestra.run.smithi097.stdout:3639 write 0x7e34d95 thru 0x7e429eb (0xdc57 bytes) 2024-09-18T00:18:57.584 INFO:teuthology.orchestra.run.smithi097.stdout:3640 trunc from 0x8d34331 to 0x8886f0e 2024-09-18T00:18:57.591 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:57.586+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:57.595 INFO:teuthology.orchestra.run.smithi097.stdout:3641 write 0x6eac9f7 thru 0x6eb6954 (0x9f5e bytes) 2024-09-18T00:18:57.598 INFO:teuthology.orchestra.run.smithi097.stdout:3642 punch from 0x5e0ced6 to 0x5e1622b, (0x9355 bytes) 2024-09-18T00:18:57.598 INFO:teuthology.orchestra.run.smithi097.stdout:3643 read 0x61a5ec4 thru 0x61aa356 (0x4493 bytes) 2024-09-18T00:18:57.607 INFO:teuthology.orchestra.run.smithi097.stdout:3644 trunc from 0x8886f0e to 0x27189de 2024-09-18T00:18:57.627 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:57.627 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:57.627 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:18:57.627 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:57.641 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:18:57.647 INFO:teuthology.orchestra.run.smithi097.stdout:3645 writesame 0x6602a57 thru 0x660fead data_size 0x1(0xd457 bytes) 2024-09-18T00:18:57.656 INFO:teuthology.orchestra.run.smithi097.stdout:3646 trunc from 0x660feae to 0x1cff9bc 2024-09-18T00:18:57.688 INFO:teuthology.orchestra.run.smithi097.stdout:3648 writesame 0x9d68aca thru 0x9d68d9b data_size 0x1(0x2d2 bytes) 2024-09-18T00:18:57.692 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:57.686+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:57.702 INFO:teuthology.orchestra.run.smithi097.stdout:3649 write 0xb85ef99 thru 0xb86dff4 (0xf05c bytes) 2024-09-18T00:18:57.711 INFO:teuthology.orchestra.run.smithi097.stdout:3650 trunc from 0xb86dff5 to 0x74dae6f 2024-09-18T00:18:57.721 INFO:teuthology.orchestra.run.smithi097.stdout:3651 punch from 0x516cafe to 0x51717f4, (0x4cf6 bytes) 2024-09-18T00:18:57.721 INFO:teuthology.orchestra.run.smithi097.stdout:3652 read 0x2408628 thru 0x24127fe (0xa1d7 bytes) 2024-09-18T00:18:57.729 INFO:teuthology.orchestra.run.smithi097.stdout:3653 trunc from 0x74dae6f to 0xc4d2597 2024-09-18T00:18:57.740 INFO:teuthology.orchestra.run.smithi097.stdout:3654 trunc from 0xc4d2597 to 0xa07e37e 2024-09-18T00:18:57.766 INFO:teuthology.orchestra.run.smithi097.stdout:3655 write 0xd3d7be1 thru 0xd3e29eb (0xae0b bytes) 2024-09-18T00:18:57.768 INFO:teuthology.orchestra.run.smithi097.stdout:3656 writesame 0xb1340ef thru 0xb143c25 data_size 0x61(0xfb37 bytes) 2024-09-18T00:18:57.768 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:18:57.771 INFO:teuthology.orchestra.run.smithi097.stdout:3657 punch from 0x5370377 to 0x5376aab, (0x6734 bytes) 2024-09-18T00:18:57.780 INFO:teuthology.orchestra.run.smithi097.stdout:3658 write 0xea84b8c thru 0xea920ef (0xd564 bytes) 2024-09-18T00:18:57.783 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:18:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:18:57.783 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:18:57.783 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:57.783 INFO:teuthology.orchestra.run.smithi097.stdout:3660 read 0x2820bed thru 0x2821688 (0xa9c bytes) 2024-09-18T00:18:57.784 INFO:teuthology.orchestra.run.smithi097.stdout:3662 writesame 0xe137cdb thru 0xe13ff96 data_size 0x6(0x82bc bytes) 2024-09-18T00:18:57.788 INFO:teuthology.orchestra.run.smithi097.stdout:3664 writesame 0x60341fc thru 0x603762c data_size 0x1(0x3431 bytes) 2024-09-18T00:18:57.793 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:57.790+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:57.799 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:18:57.834 INFO:teuthology.orchestra.run.smithi097.stdout:3665 write 0x9d16fb1 thru 0x9d17c92 (0xce2 bytes) 2024-09-18T00:18:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:3666 read 0x1bb6910 thru 0x1bb6a43 (0x134 bytes) 2024-09-18T00:18:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:57.839 INFO:teuthology.orchestra.run.smithi097.stdout:3670 read 0x64c5b3e thru 0x64cbae8 (0x5fab bytes) 2024-09-18T00:18:57.840 INFO:teuthology.orchestra.run.smithi097.stdout:3671 read 0xe4fd599 thru 0xe50225c (0x4cc4 bytes) 2024-09-18T00:18:57.840 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:57.843 INFO:teuthology.orchestra.run.smithi097.stdout:3675 trunc from 0xea920f0 to 0xc1b36be 2024-09-18T00:18:57.853 INFO:teuthology.orchestra.run.smithi097.stdout:3678 write 0x60376c8 thru 0x60382d1 (0xc0a bytes) 2024-09-18T00:18:57.867 INFO:teuthology.orchestra.run.smithi097.stdout:3682 write 0xeaf74ab thru 0xeaf86f0 (0x1246 bytes) 2024-09-18T00:18:57.869 INFO:teuthology.orchestra.run.smithi097.stdout:3684 read 0x1039fb1 thru 0x10418cd (0x791d bytes) 2024-09-18T00:18:57.870 INFO:teuthology.orchestra.run.smithi097.stdout:3685 punch from 0x632e809 to 0x6335690, (0x6e87 bytes) 2024-09-18T00:18:57.870 INFO:teuthology.orchestra.run.smithi097.stdout:3686 read 0x7baf1b9 thru 0x7bba068 (0xaeb0 bytes) 2024-09-18T00:18:57.871 INFO:teuthology.orchestra.run.smithi097.stdout:3687 write 0xf8ccb thru 0x104cb3 (0xbfe9 bytes) 2024-09-18T00:18:57.873 INFO:teuthology.orchestra.run.smithi097.stdout:3689 punch from 0x455f551 to 0x4567fea, (0x8a99 bytes) 2024-09-18T00:18:57.873 INFO:teuthology.orchestra.run.smithi097.stdout:3690 punch from 0x5319895 to 0x53248e3, (0xb04e bytes) 2024-09-18T00:18:57.874 INFO:teuthology.orchestra.run.smithi097.stdout:3691 writesame 0xd42a21a thru 0xd4315c9 data_size 0x18(0x73b0 bytes) 2024-09-18T00:18:57.875 INFO:teuthology.orchestra.run.smithi097.stdout:3692 write 0x3fa17e4 thru 0x3fa4aea (0x3307 bytes) 2024-09-18T00:18:57.878 INFO:teuthology.orchestra.run.smithi097.stdout:3695 write 0xba04e08 thru 0xba0c3f7 (0x75f0 bytes) 2024-09-18T00:18:57.880 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:57.880 INFO:teuthology.orchestra.run.smithi097.stdout:3699 write 0x82bc02b thru 0x82c0974 (0x494a bytes) 2024-09-18T00:18:57.884 INFO:teuthology.orchestra.run.smithi097.stdout:3701 write 0xc003e3d thru 0xc006924 (0x2ae8 bytes) 2024-09-18T00:18:57.888 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:57.888 INFO:teuthology.orchestra.run.smithi097.stdout:3703 write 0xc0b5baf thru 0xc0c33a9 (0xd7fb bytes) 2024-09-18T00:18:57.891 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:57.891 INFO:teuthology.orchestra.run.smithi097.stdout:3706 write 0x5e57931 thru 0x5e5bb02 (0x41d2 bytes) 2024-09-18T00:18:57.893 INFO:teuthology.orchestra.run.smithi097.stdout:3707 write 0x67a6c53 thru 0x67b6a74 (0xfe22 bytes) 2024-09-18T00:18:57.894 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:57.890+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:57.902 INFO:teuthology.orchestra.run.smithi097.stdout:3708 write 0x4216a58 thru 0x421fcc9 (0x9272 bytes) 2024-09-18T00:18:57.905 INFO:teuthology.orchestra.run.smithi097.stdout:3709 write 0xe1e51b4 thru 0xe1f0bc2 (0xba0f bytes) 2024-09-18T00:18:57.908 INFO:teuthology.orchestra.run.smithi097.stdout:3710 writesame 0x173a356 thru 0x17452fb data_size 0x1(0xafa6 bytes) 2024-09-18T00:18:57.990 INFO:teuthology.orchestra.run.smithi097.stdout:3711 writesame 0xa636900 thru 0xa64322e data_size 0x1(0xc92f bytes) 2024-09-18T00:18:57.993 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:18:57.993 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:18:57.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:18:57.993 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:18:57.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.993 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:18:57.993 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:57.993 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.189261444000003, 2024-09-18T00:18:57.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:18:57.993 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.993 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.993 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:57.994 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:57.994 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:57.994 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:57.994 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:18:57.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.994 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:57.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:57.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:57.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:18:57.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:57.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:57.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:18:57.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:18:57.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:57.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:18:57.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:18:57.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:57.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:18:57.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:18:57.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:57.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:18:57.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:18:57.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:57.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:18:57.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:18:57.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:57.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:18:57.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:18:57.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:57.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:18:57.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:18:57.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:57.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:18:57.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:18:57.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:57.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:18:57.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:18:57.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:57.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:18:57.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:18:57.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:57.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:18:57.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:18:57.999 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.999 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:57.999 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:57.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:57.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:57.999 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:18:57.999 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:57.999 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.173053570999997, 2024-09-18T00:18:57.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:18:57.999 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:57.999 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:57.999 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:57.999 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:57.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:58.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:18:58.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:18:58.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:18:58.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:18:58.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:18:58.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:18:58.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:18:58.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:18:58.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:18:58.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:18:58.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:18:58.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:18:58.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:18:58.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:18:58.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:18:58.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.003 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.003 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:58.003 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:58.003 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.146012251000002, 2024-09-18T00:18:58.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:18:58.003 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.004 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.004 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.004 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:58.004 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:58.004 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.004 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:18:58.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.004 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.004 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.004 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:58.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:18:58.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:18:58.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:18:58.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:18:58.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:18:58.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:18:58.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:18:58.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:18:58.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:18:58.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:18:58.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:18:58.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:18:58.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:18:58.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:58.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:18:58.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:18:58.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:58.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:18:58.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:18:58.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:58.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:18:58.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:18:58.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:58.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:18:58.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:18:58.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:18:58.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:18:58.009 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.009 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.009 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.009 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:18:58.009 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:58.009 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.124735698999999, 2024-09-18T00:18:58.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:18:58.009 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.009 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.009 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:58.009 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:58.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:18:58.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:18:58.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:18:58.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:18:58.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:18:58.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:18:58.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:18:58.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:18:58.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:18:58.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:18:58.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:18:58.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:18:58.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:18:58.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:18:58.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:18:58.013 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.013 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.013 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.013 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11460 3.19 3:9e578b6c:::rbd_data.112ab1fa102a.0000000000000153:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:58.013 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:18:58.013 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.722364247000002, 2024-09-18T00:18:58.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041989011999999999, 2024-09-18T00:18:58.013 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.013 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.013 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.014 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:58.014 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:58.014 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.014 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11460 2024-09-18T00:18:58.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.014 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:18:58.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:18:58.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267067+0000", 2024-09-18T00:18:58.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:18:58.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267088+0000", 2024-09-18T00:18:58.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0908999999999999e-05 2024-09-18T00:18:58.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267133+0000", 2024-09-18T00:18:58.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5232000000000003e-05 2024-09-18T00:18:58.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267157+0000", 2024-09-18T00:18:58.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3833e-05 2024-09-18T00:18:58.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267300+0000", 2024-09-18T00:18:58.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142282 2024-09-18T00:18:58.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267447+0000", 2024-09-18T00:18:58.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147851 2024-09-18T00:18:58.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:58.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308889+0000", 2024-09-18T00:18:58.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041441779999999998 2024-09-18T00:18:58.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:58.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308956+0000", 2024-09-18T00:18:58.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7136000000000002e-05 2024-09-18T00:18:58.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.309053+0000", 2024-09-18T00:18:58.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6552999999999995e-05 2024-09-18T00:18:58.018 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.018 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.018 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.018 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11464 3.b 3:d331f8ac:::rbd_data.112ab1fa102a.0000000000000157:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:18:58.018 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:18:58.018 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.717373761000001, 2024-09-18T00:18:58.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034709218, 2024-09-18T00:18:58.018 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.019 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.019 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.019 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:58.019 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:58.019 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.019 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11464 2024-09-18T00:18:58.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.019 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:18:58.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:18:58.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272058+0000", 2024-09-18T00:18:58.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4070000000000002e-06 2024-09-18T00:18:58.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272074+0000", 2024-09-18T00:18:58.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5872999999999999e-05 2024-09-18T00:18:58.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272113+0000", 2024-09-18T00:18:58.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9224999999999998e-05 2024-09-18T00:18:58.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272135+0000", 2024-09-18T00:18:58.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2093e-05 2024-09-18T00:18:58.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272238+0000", 2024-09-18T00:18:58.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102939 2024-09-18T00:18:58.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272327+0000", 2024-09-18T00:18:58.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8746999999999996e-05 2024-09-18T00:18:58.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:58.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.273486+0000", 2024-09-18T00:18:58.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011588779999999999 2024-09-18T00:18:58.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:58.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306439+0000", 2024-09-18T00:18:58.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.032953142999999997 2024-09-18T00:18:58.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:58.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306491+0000", 2024-09-18T00:18:58.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2475000000000002e-05 2024-09-18T00:18:58.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:58.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306648+0000", 2024-09-18T00:18:58.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015725099999999999 2024-09-18T00:18:58.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306764+0000", 2024-09-18T00:18:58.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011518699999999999 2024-09-18T00:18:58.024 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.024 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.024 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.024 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:16947 3.18 3:18b58c7b:::rbd_data.112a6efb7564.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:18:58.024 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:18:58.024 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.3725865460000009, 2024-09-18T00:18:58.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.101692484, 2024-09-18T00:18:58.024 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.024 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.024 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.025 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:58.025 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:58.025 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.025 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 16947 2024-09-18T00:18:58.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.025 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:18:58.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:18:58.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616843+0000", 2024-09-18T00:18:58.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0780000000000001e-06 2024-09-18T00:18:58.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617785+0000", 2024-09-18T00:18:58.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094204100000000002 2024-09-18T00:18:58.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617801+0000", 2024-09-18T00:18:58.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6436999999999999e-05 2024-09-18T00:18:58.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617809+0000", 2024-09-18T00:18:58.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9049999999999997e-06 2024-09-18T00:18:58.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617904+0000", 2024-09-18T00:18:58.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5204999999999994e-05 2024-09-18T00:18:58.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.618071+0000", 2024-09-18T00:18:58.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016640000000000001 2024-09-18T00:18:58.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:58.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.624534+0000", 2024-09-18T00:18:58.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064634499999999999 2024-09-18T00:18:58.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:58.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718299+0000", 2024-09-18T00:18:58.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093764739 2024-09-18T00:18:58.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:58.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718375+0000", 2024-09-18T00:18:58.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6532999999999994e-05 2024-09-18T00:18:58.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:58.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718466+0000", 2024-09-18T00:18:58.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0581000000000006e-05 2024-09-18T00:18:58.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718534+0000", 2024-09-18T00:18:58.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8114999999999994e-05 2024-09-18T00:18:58.029 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.029 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.030 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.030 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:16947 3.18 e59/23 ondisk, result = 0)", 2024-09-18T00:18:58.030 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:18:58.030 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.3473698519999999, 2024-09-18T00:18:58.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076488069000000006, 2024-09-18T00:18:58.030 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.030 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.030 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:58.030 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:18:58.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:18:58.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642060+0000", 2024-09-18T00:18:58.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6640000000000001e-06 2024-09-18T00:18:58.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642071+0000", 2024-09-18T00:18:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1204e-05 2024-09-18T00:18:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642091+0000", 2024-09-18T00:18:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0313999999999999e-05 2024-09-18T00:18:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642103+0000", 2024-09-18T00:18:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1214e-05 2024-09-18T00:18:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718350+0000", 2024-09-18T00:18:58.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076247681999999997 2024-09-18T00:18:58.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718366+0000", 2024-09-18T00:18:58.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5420000000000001e-05 2024-09-18T00:18:58.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718546+0000", 2024-09-18T00:18:58.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000180571 2024-09-18T00:18:58.033 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.033 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.034 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.034 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:17323 3.18 3:18b58c7b:::rbd_data.112a6efb7564.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:18:58.034 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:49.402708+0000", 2024-09-18T00:18:58.034 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.5867196769999996, 2024-09-18T00:18:58.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034488817999999997, 2024-09-18T00:18:58.034 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.034 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.034 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.034 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:58.034 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:58.034 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.034 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 17323 2024-09-18T00:18:58.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.035 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.402708+0000", 2024-09-18T00:18:58.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.402708+0000", 2024-09-18T00:18:58.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.402711+0000", 2024-09-18T00:18:58.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2139999999999999e-06 2024-09-18T00:18:58.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.402722+0000", 2024-09-18T00:18:58.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1158999999999999e-05 2024-09-18T00:18:58.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.402752+0000", 2024-09-18T00:18:58.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9805999999999999e-05 2024-09-18T00:18:58.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.402766+0000", 2024-09-18T00:18:58.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4633e-05 2024-09-18T00:18:58.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.402852+0000", 2024-09-18T00:18:58.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5799999999999998e-05 2024-09-18T00:18:58.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.402943+0000", 2024-09-18T00:18:58.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0636000000000004e-05 2024-09-18T00:18:58.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:58.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.406990+0000", 2024-09-18T00:18:58.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0040471109999999999 2024-09-18T00:18:58.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:58.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.437014+0000", 2024-09-18T00:18:58.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.03002442 2024-09-18T00:18:58.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:58.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.437036+0000", 2024-09-18T00:18:58.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1923999999999999e-05 2024-09-18T00:18:58.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:58.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.437124+0000", 2024-09-18T00:18:58.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8027000000000003e-05 2024-09-18T00:18:58.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.437197+0000", 2024-09-18T00:18:58.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3088000000000002e-05 2024-09-18T00:18:58.043 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.043 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.043 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.043 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:17363 3.15 e59/32 3:ad4938b4:::rbd_object_map.112a6efb7564:head v 59'143, mlcod=59'141)", 2024-09-18T00:18:58.043 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:49.539843+0000", 2024-09-18T00:18:58.044 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.4495848799999997, 2024-09-18T00:18:58.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.023088518999999998, 2024-09-18T00:18:58.044 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.044 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.044 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.044 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.539843+0000", 2024-09-18T00:18:58.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.539843+0000", 2024-09-18T00:18:58.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.539845+0000", 2024-09-18T00:18:58.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.81e-06 2024-09-18T00:18:58.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.539858+0000", 2024-09-18T00:18:58.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2894000000000001e-05 2024-09-18T00:18:58.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.539889+0000", 2024-09-18T00:18:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0664999999999999e-05 2024-09-18T00:18:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.539902+0000", 2024-09-18T00:18:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3203999999999999e-05 2024-09-18T00:18:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.539989+0000", 2024-09-18T00:18:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7318999999999995e-05 2024-09-18T00:18:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.540016+0000", 2024-09-18T00:18:58.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6735000000000001e-05 2024-09-18T00:18:58.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:58.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.562832+0000", 2024-09-18T00:18:58.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022816184999999999 2024-09-18T00:18:58.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:49.562932+0000", 2024-09-18T00:18:58.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9706999999999999e-05 2024-09-18T00:18:58.047 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.047 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.048 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.048 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18684 3.b 3:d2138ebe:::rbd_data.112a5ba20d45.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:58.048 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.947362+0000", 2024-09-18T00:18:58.048 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.0420658769999998, 2024-09-18T00:18:58.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.022849864000000001, 2024-09-18T00:18:58.048 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.048 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.048 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:58.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:58.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.048 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18684 2024-09-18T00:18:58.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.049 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.947362+0000", 2024-09-18T00:18:58.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.947362+0000", 2024-09-18T00:18:58.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.947365+0000", 2024-09-18T00:18:58.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0630000000000002e-06 2024-09-18T00:18:58.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948505+0000", 2024-09-18T00:18:58.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011400679999999999 2024-09-18T00:18:58.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948548+0000", 2024-09-18T00:18:58.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2318999999999999e-05 2024-09-18T00:18:58.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948572+0000", 2024-09-18T00:18:58.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4053000000000002e-05 2024-09-18T00:18:58.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:18:58.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016425899999999999 2024-09-18T00:18:58.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949055+0000", 2024-09-18T00:18:58.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031904600000000002 2024-09-18T00:18:58.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:58.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949775+0000", 2024-09-18T00:18:58.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00072009500000000002 2024-09-18T00:18:58.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:58.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.963879+0000", 2024-09-18T00:18:58.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.014103704 2024-09-18T00:18:58.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:58.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.970012+0000", 2024-09-18T00:18:58.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0061329369999999998 2024-09-18T00:18:58.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:58.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.970126+0000", 2024-09-18T00:18:58.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000114436 2024-09-18T00:18:58.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.970212+0000", 2024-09-18T00:18:58.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5884000000000006e-05 2024-09-18T00:18:58.053 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.053 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.053 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.053 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18686 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:58.053 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:18:58.054 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.0406924819999999, 2024-09-18T00:18:58.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041132573999999998, 2024-09-18T00:18:58.054 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.054 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.054 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.054 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:58.054 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:58.054 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.054 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18686 2024-09-18T00:18:58.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.054 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:18:58.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:18:58.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948737+0000", 2024-09-18T00:18:58.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3769999999999999e-06 2024-09-18T00:18:58.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949968+0000", 2024-09-18T00:18:58.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001230493 2024-09-18T00:18:58.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949993+0000", 2024-09-18T00:18:58.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5282e-05 2024-09-18T00:18:58.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950011+0000", 2024-09-18T00:18:58.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8199999999999999e-05 2024-09-18T00:18:58.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950180+0000", 2024-09-18T00:18:58.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016867600000000001 2024-09-18T00:18:58.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950337+0000", 2024-09-18T00:18:58.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156902 2024-09-18T00:18:58.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:58.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950818+0000", 2024-09-18T00:18:58.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048192599999999998 2024-09-18T00:18:58.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:58.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978491+0000", 2024-09-18T00:18:58.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027672568000000002 2024-09-18T00:18:58.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:58.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989759+0000", 2024-09-18T00:18:58.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011268144000000001 2024-09-18T00:18:58.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:58.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989824+0000", 2024-09-18T00:18:58.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4556999999999999e-05 2024-09-18T00:18:58.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989868+0000", 2024-09-18T00:18:58.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4449000000000001e-05 2024-09-18T00:18:58.059 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.059 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.059 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.059 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18687 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:58.059 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:18:58.059 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.0393647169999998, 2024-09-18T00:18:58.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040034156000000001, 2024-09-18T00:18:58.059 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.059 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.059 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.060 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:58.060 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:58.060 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.060 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18687 2024-09-18T00:18:58.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.060 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:18:58.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:18:58.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950065+0000", 2024-09-18T00:18:58.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5820000000000001e-06 2024-09-18T00:18:58.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950072+0000", 2024-09-18T00:18:58.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1029999999999996e-06 2024-09-18T00:18:58.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950092+0000", 2024-09-18T00:18:58.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0288e-05 2024-09-18T00:18:58.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950107+0000", 2024-09-18T00:18:58.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4290999999999999e-05 2024-09-18T00:18:58.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975820+0000", 2024-09-18T00:18:58.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025713573999999999 2024-09-18T00:18:58.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975980+0000", 2024-09-18T00:18:58.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016002799999999999 2024-09-18T00:18:58.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:58.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.976299+0000", 2024-09-18T00:18:58.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031853300000000002 2024-09-18T00:18:58.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:58.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989936+0000", 2024-09-18T00:18:58.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013637046999999999 2024-09-18T00:18:58.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:58.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990025+0000", 2024-09-18T00:18:58.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9040000000000001e-05 2024-09-18T00:18:58.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:58.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990064+0000", 2024-09-18T00:18:58.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9394000000000003e-05 2024-09-18T00:18:58.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990098+0000", 2024-09-18T00:18:58.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3275999999999998e-05 2024-09-18T00:18:58.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.065 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.065 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.065 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18688 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:58.065 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:18:58.065 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.0392633650000001, 2024-09-18T00:18:58.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040041259000000003, 2024-09-18T00:18:58.065 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.065 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.065 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:58.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:58.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.066 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18688 2024-09-18T00:18:58.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.066 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:18:58.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:18:58.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950166+0000", 2024-09-18T00:18:58.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:18:58.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951295+0000", 2024-09-18T00:18:58.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011293480000000001 2024-09-18T00:18:58.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951328+0000", 2024-09-18T00:18:58.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2524999999999998e-05 2024-09-18T00:18:58.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951346+0000", 2024-09-18T00:18:58.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8170999999999999e-05 2024-09-18T00:18:58.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978546+0000", 2024-09-18T00:18:58.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027200418000000001 2024-09-18T00:18:58.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978666+0000", 2024-09-18T00:18:58.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119913 2024-09-18T00:18:58.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:58.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.979098+0000", 2024-09-18T00:18:58.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00043169399999999999 2024-09-18T00:18:58.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:58.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990000+0000", 2024-09-18T00:18:58.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010901719000000001 2024-09-18T00:18:58.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:58.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990109+0000", 2024-09-18T00:18:58.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010964299999999999 2024-09-18T00:18:58.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:58.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990152+0000", 2024-09-18T00:18:58.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2877000000000003e-05 2024-09-18T00:18:58.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990206+0000", 2024-09-18T00:18:58.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3717000000000001e-05 2024-09-18T00:18:58.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.070 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.071 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18689 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:58.071 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:18:58.071 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.0380516670000004, 2024-09-18T00:18:58.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051022338, 2024-09-18T00:18:58.071 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.071 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.071 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:58.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:58.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.071 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18689 2024-09-18T00:18:58.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.071 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:18:58.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:18:58.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951378+0000", 2024-09-18T00:18:58.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:18:58.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951385+0000", 2024-09-18T00:18:58.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-18T00:18:58.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951402+0000", 2024-09-18T00:18:58.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6538999999999999e-05 2024-09-18T00:18:58.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951565+0000", 2024-09-18T00:18:58.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016307999999999999 2024-09-18T00:18:58.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988232+0000", 2024-09-18T00:18:58.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036667656999999999 2024-09-18T00:18:58.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988318+0000", 2024-09-18T00:18:58.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5348999999999999e-05 2024-09-18T00:18:58.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:58.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988574+0000", 2024-09-18T00:18:58.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025670800000000002 2024-09-18T00:18:58.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:58.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002254+0000", 2024-09-18T00:18:58.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013679274 2024-09-18T00:18:58.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:58.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002326+0000", 2024-09-18T00:18:58.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2720999999999997e-05 2024-09-18T00:18:58.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:58.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002364+0000", 2024-09-18T00:18:58.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7314000000000002e-05 2024-09-18T00:18:58.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002399+0000", 2024-09-18T00:18:58.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5162e-05 2024-09-18T00:18:58.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.076 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.076 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18690 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:58.076 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:18:58.076 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.0378346489999997, 2024-09-18T00:18:58.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050872134999999999, 2024-09-18T00:18:58.077 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.077 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.077 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:58.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:58.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.077 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18690 2024-09-18T00:18:58.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.077 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:18:58.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:18:58.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951595+0000", 2024-09-18T00:18:58.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4479999999999999e-06 2024-09-18T00:18:58.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952167+0000", 2024-09-18T00:18:58.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00057188400000000004 2024-09-18T00:18:58.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952192+0000", 2024-09-18T00:18:58.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4936999999999999e-05 2024-09-18T00:18:58.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952206+0000", 2024-09-18T00:18:58.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4565e-05 2024-09-18T00:18:58.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990303+0000", 2024-09-18T00:18:58.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038096954000000002 2024-09-18T00:18:58.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990403+0000", 2024-09-18T00:18:58.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9486000000000003e-05 2024-09-18T00:18:58.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:58.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990773+0000", 2024-09-18T00:18:58.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037037799999999998 2024-09-18T00:18:58.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:58.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002316+0000", 2024-09-18T00:18:58.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011542581 2024-09-18T00:18:58.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:58.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002407+0000", 2024-09-18T00:18:58.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0968e-05 2024-09-18T00:18:58.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:58.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002442+0000", 2024-09-18T00:18:58.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4962000000000002e-05 2024-09-18T00:18:58.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002466+0000", 2024-09-18T00:18:58.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3972e-05 2024-09-18T00:18:58.082 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.082 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.082 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.082 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18691 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:58.082 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:18:58.082 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.0372017720000004, 2024-09-18T00:18:58.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051495530999999997, 2024-09-18T00:18:58.082 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.082 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.082 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.083 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:58.083 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:58.083 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.083 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18691 2024-09-18T00:18:58.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.083 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:18:58.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:18:58.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952228+0000", 2024-09-18T00:18:58.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:18:58.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952234+0000", 2024-09-18T00:18:58.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0310000000000004e-06 2024-09-18T00:18:58.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952248+0000", 2024-09-18T00:18:58.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4577e-05 2024-09-18T00:18:58.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952260+0000", 2024-09-18T00:18:58.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1659e-05 2024-09-18T00:18:58.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.000989+0000", 2024-09-18T00:18:58.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048729445000000003 2024-09-18T00:18:58.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001076+0000", 2024-09-18T00:18:58.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6323999999999995e-05 2024-09-18T00:18:58.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:58.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001295+0000", 2024-09-18T00:18:58.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021944299999999999 2024-09-18T00:18:58.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:58.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003502+0000", 2024-09-18T00:18:58.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022065209999999999 2024-09-18T00:18:58.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:58.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003581+0000", 2024-09-18T00:18:58.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9404000000000001e-05 2024-09-18T00:18:58.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:58.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003652+0000", 2024-09-18T00:18:58.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0506999999999994e-05 2024-09-18T00:18:58.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003722+0000", 2024-09-18T00:18:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0300000000000001e-05 2024-09-18T00:18:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18692 3.7 3:e69124d0:::rbd_data.112a6efb7564.0000000000000001:f [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:18:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.0371537110000002, 2024-09-18T00:18:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050418114999999999, 2024-09-18T00:18:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:18:58.089 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:58.089 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:58.089 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.089 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18692 2024-09-18T00:18:58.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.089 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:18:58.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:18:58.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952275+0000", 2024-09-18T00:18:58.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4e-07 2024-09-18T00:18:58.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952281+0000", 2024-09-18T00:18:58.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2000000000000002e-06 2024-09-18T00:18:58.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952295+0000", 2024-09-18T00:18:58.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4731e-05 2024-09-18T00:18:58.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952307+0000", 2024-09-18T00:18:58.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1877e-05 2024-09-18T00:18:58.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002506+0000", 2024-09-18T00:18:58.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050199230999999997 2024-09-18T00:18:58.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002574+0000", 2024-09-18T00:18:58.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8076000000000003e-05 2024-09-18T00:18:58.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002693+0000", 2024-09-18T00:18:58.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011806 2024-09-18T00:18:58.092 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.092 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.092 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.092 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19383 3.5 3:a1859bb0:::rbd_data.112a7fe29ab4.000000000000015b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:58.092 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:18:58.093 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.367142114, 2024-09-18T00:18:58.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056760192000000001, 2024-09-18T00:18:58.093 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.093 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.093 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.093 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:58.093 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:58.093 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.093 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19383 2024-09-18T00:18:58.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.093 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:18:58.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:18:58.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622287+0000", 2024-09-18T00:18:58.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-18T00:18:58.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622292+0000", 2024-09-18T00:18:58.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6110000000000004e-06 2024-09-18T00:18:58.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622304+0000", 2024-09-18T00:18:58.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2106000000000001e-05 2024-09-18T00:18:58.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622309+0000", 2024-09-18T00:18:58.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.818e-06 2024-09-18T00:18:58.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622360+0000", 2024-09-18T00:18:58.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0430999999999999e-05 2024-09-18T00:18:58.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622438+0000", 2024-09-18T00:18:58.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8362999999999995e-05 2024-09-18T00:18:58.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:58.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678811+0000", 2024-09-18T00:18:58.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056372894 2024-09-18T00:18:58.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:58.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678906+0000", 2024-09-18T00:18:58.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5078e-05 2024-09-18T00:18:58.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.679046+0000", 2024-09-18T00:18:58.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139987 2024-09-18T00:18:58.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.097 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.098 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22019 3.c 3:37b77e94:::rbd_data.112a948d5956.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:18:58.098 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:18:58.098 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.49016728500000001, 2024-09-18T00:18:58.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.039176065000000003, 2024-09-18T00:18:58.098 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:18:58.098 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:18:58.098 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:18:58.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:18:58.098 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.098 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22019 2024-09-18T00:18:58.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.098 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:18:58.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:18:58.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:18:58.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:18:58.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:18:58.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:18:58.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:18:58.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499262+0000", 2024-09-18T00:18:58.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:18:58.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:18:58.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499268+0000", 2024-09-18T00:18:58.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8780000000000003e-06 2024-09-18T00:18:58.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:18:58.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499283+0000", 2024-09-18T00:18:58.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5184000000000001e-05 2024-09-18T00:18:58.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499291+0000", 2024-09-18T00:18:58.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5959999999999998e-06 2024-09-18T00:18:58.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:18:58.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499364+0000", 2024-09-18T00:18:58.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2855000000000006e-05 2024-09-18T00:18:58.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:18:58.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499427+0000", 2024-09-18T00:18:58.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3645999999999993e-05 2024-09-18T00:18:58.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:18:58.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538252+0000", 2024-09-18T00:18:58.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038825207 2024-09-18T00:18:58.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:18:58.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538328+0000", 2024-09-18T00:18:58.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5938000000000004e-05 2024-09-18T00:18:58.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:18:58.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:18:58.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:18:58.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538437+0000", 2024-09-18T00:18:58.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108527 2024-09-18T00:18:58.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.102 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.103 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:18:58.103 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:18:58.103 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:18:58.104 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:18:58.105 INFO:teuthology.orchestra.run.smithi097.stdout:3712 write 0xe1d295d thru 0xe1d7290 (0x4934 bytes) 2024-09-18T00:18:58.105 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:58.105 INFO:teuthology.orchestra.run.smithi097.stdout:3715 writesame 0x8e292fa thru 0x8e31c17 data_size 0x1(0x891e bytes) 2024-09-18T00:18:58.105 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:58.098+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:58.174 INFO:teuthology.orchestra.run.smithi097.stdout:3717 punch from 0x4b751d2 to 0x4b7ffd3, (0xae01 bytes) 2024-09-18T00:18:58.174 INFO:teuthology.orchestra.run.smithi097.stdout:3718 write 0x23d76bb thru 0x23d9852 (0x2198 bytes) 2024-09-18T00:18:58.177 INFO:teuthology.orchestra.run.smithi097.stdout:3719 write 0x148465e thru 0x148632b (0x1cce bytes) 2024-09-18T00:18:58.179 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:58.179 INFO:teuthology.orchestra.run.smithi097.stdout:3722 write 0x8d88653 thru 0x8d8d53e (0x4eec bytes) 2024-09-18T00:18:58.181 INFO:teuthology.orchestra.run.smithi097.stdout:3724 write 0xdfd0401 thru 0xdfdd755 (0xd355 bytes) 2024-09-18T00:18:58.185 INFO:teuthology.orchestra.run.smithi097.stdout:3726 read 0xc061bfc thru 0xc06ece1 (0xd0e6 bytes) 2024-09-18T00:18:58.186 INFO:teuthology.orchestra.run.smithi097.stdout:3728 punch from 0x96765c0 to 0x967db02, (0x7542 bytes) 2024-09-18T00:18:58.186 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:58.187 INFO:teuthology.orchestra.run.smithi097.stdout:3731 write 0xa18e3ad thru 0xa199a8e (0xb6e2 bytes) 2024-09-18T00:18:58.189 INFO:teuthology.orchestra.run.smithi097.stdout:3732 write 0xe14785e thru 0xe152d30 (0xb4d3 bytes) 2024-09-18T00:18:58.192 INFO:teuthology.orchestra.run.smithi097.stdout:3735 write 0x737dbe9 thru 0x73805c5 (0x29dd bytes) 2024-09-18T00:18:58.205 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:58.202+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:58.219 INFO:teuthology.orchestra.run.smithi097.stdout:3737 trunc from 0xeaf86f1 to 0x408c1c9 2024-09-18T00:18:58.231 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:58.231 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:58.231 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:18:58.232 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:58.246 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:18:58.268 INFO:teuthology.orchestra.run.smithi097.stdout:3738 write 0xe076da7 thru 0xe07ca3d (0x5c97 bytes) 2024-09-18T00:18:58.270 INFO:teuthology.orchestra.run.smithi097.stdout:3739 punch from 0x79e3d25 to 0x79ee8ec, (0xabc7 bytes) 2024-09-18T00:18:58.293 INFO:teuthology.orchestra.run.smithi097.stdout:3740 trunc from 0xe07ca3e to 0x47d602e 2024-09-18T00:18:58.306 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:58.302+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:58.307 INFO:teuthology.orchestra.run.smithi097.stdout:3743 read 0x2b96225 thru 0x2ba5bcf (0xf9ab bytes) 2024-09-18T00:18:58.308 INFO:teuthology.orchestra.run.smithi097.stdout:3744 read 0x3cfaedf thru 0x3d088e0 (0xda02 bytes) 2024-09-18T00:18:58.345 INFO:teuthology.orchestra.run.smithi097.stdout:3746 write 0xd586dea thru 0xd590203 (0x941a bytes) 2024-09-18T00:18:58.347 INFO:teuthology.orchestra.run.smithi097.stdout:3747 read 0x82a8b4c thru 0x82b4bd7 (0xc08c bytes) 2024-09-18T00:18:58.348 INFO:teuthology.orchestra.run.smithi097.stdout:3748 read 0xd135930 thru 0xd144fee (0xf6bf bytes) 2024-09-18T00:18:58.349 INFO:teuthology.orchestra.run.smithi097.stdout:3750 write 0xbd67a65 thru 0xbd745d2 (0xcb6e bytes) 2024-09-18T00:18:58.357 INFO:teuthology.orchestra.run.smithi097.stdout:3751 write 0xbfb80cd thru 0xbfb9c61 (0x1b95 bytes) 2024-09-18T00:18:58.359 INFO:teuthology.orchestra.run.smithi097.stdout:3753 punch from 0x9cbb534 to 0x9cca636, (0xf102 bytes) 2024-09-18T00:18:58.361 INFO:teuthology.orchestra.run.smithi097.stdout:3756 trunc from 0xd590204 to 0xdee9451 2024-09-18T00:18:58.368 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:58.368 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:58.368 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:58.368 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:58.369 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:58.373 INFO:teuthology.orchestra.run.smithi097.stdout:3757 punch from 0x78e4ab6 to 0x78e4c87, (0x1d1 bytes) 2024-09-18T00:18:58.382 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:18:58.385 INFO:teuthology.orchestra.run.smithi097.stdout:3758 trunc from 0xdee9451 to 0x995dbd3 2024-09-18T00:18:58.402 INFO:teuthology.orchestra.run.smithi097.stdout:3759 writesame 0x122b3f6 thru 0x12381e8 data_size 0x1(0xcdf3 bytes) 2024-09-18T00:18:58.406 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:58.402+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:58.508 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:58.506+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:58.528 INFO:teuthology.orchestra.run.smithi097.stdout:3760 write 0x9bc951 thru 0x9c6222 (0x98d2 bytes) 2024-09-18T00:18:58.533 INFO:teuthology.orchestra.run.smithi097.stdout:3761 read 0x6b6e346 thru 0x6b76f01 (0x8bbc bytes) 2024-09-18T00:18:58.546 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:58.546 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:58.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:58.547 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:58.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.547 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2418 3.1a 3:59a8b40d:::rbd_data.112aa7ba4578.00000000000001b0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:18:58.547 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:18:58.547 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.808749691000003, 2024-09-18T00:18:58.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10488562999999999, 2024-09-18T00:18:58.547 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.547 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.547 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.547 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.547 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.547 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.547 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2418 2024-09-18T00:18:58.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.548 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:18:58.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:18:58.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735592+0000", 2024-09-18T00:18:58.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5229999999999998e-06 2024-09-18T00:18:58.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735604+0000", 2024-09-18T00:18:58.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2143e-05 2024-09-18T00:18:58.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735629+0000", 2024-09-18T00:18:58.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.497e-05 2024-09-18T00:18:58.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735648+0000", 2024-09-18T00:18:58.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8896999999999999e-05 2024-09-18T00:18:58.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735775+0000", 2024-09-18T00:18:58.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126935 2024-09-18T00:18:58.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.736118+0000", 2024-09-18T00:18:58.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034330800000000001 2024-09-18T00:18:58.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-18T00:18:58.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.741633+0000", 2024-09-18T00:18:58.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055144139999999996 2024-09-18T00:18:58.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:58.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840134+0000", 2024-09-18T00:18:58.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.098501618999999999 2024-09-18T00:18:58.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:58.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840184+0000", 2024-09-18T00:18:58.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9953000000000001e-05 2024-09-18T00:18:58.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:58.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840336+0000", 2024-09-18T00:18:58.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151476 2024-09-18T00:18:58.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840475+0000", 2024-09-18T00:18:58.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013939200000000001 2024-09-18T00:18:58.552 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.552 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.552 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.553 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2418 3.1a e37/23 ondisk, result = 0)", 2024-09-18T00:18:58.553 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:18:58.553 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.790159848000002, 2024-09-18T00:18:58.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.085979978999999998, 2024-09-18T00:18:58.553 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.553 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.553 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:58.553 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:18:58.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:18:58.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754181+0000", 2024-09-18T00:18:58.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8530000000000001e-06 2024-09-18T00:18:58.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754194+0000", 2024-09-18T00:18:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.292e-05 2024-09-18T00:18:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754230+0000", 2024-09-18T00:18:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5602999999999998e-05 2024-09-18T00:18:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754249+0000", 2024-09-18T00:18:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8850000000000001e-05 2024-09-18T00:18:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840070+0000", 2024-09-18T00:18:58.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.085821123999999999 2024-09-18T00:18:58.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840098+0000", 2024-09-18T00:18:58.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8546000000000001e-05 2024-09-18T00:18:58.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840159+0000", 2024-09-18T00:18:58.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1082999999999998e-05 2024-09-18T00:18:58.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.556 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.557 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12544 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:58.557 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395178+0000", 2024-09-18T00:18:58.557 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.149160744, 2024-09-18T00:18:58.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045040958999999998, 2024-09-18T00:18:58.557 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.557 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.557 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:58.557 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.557 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.557 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.557 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12544 2024-09-18T00:18:58.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.558 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395178+0000", 2024-09-18T00:18:58.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395178+0000", 2024-09-18T00:18:58.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395180+0000", 2024-09-18T00:18:58.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.28e-06 2024-09-18T00:18:58.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395186+0000", 2024-09-18T00:18:58.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3829999999999997e-06 2024-09-18T00:18:58.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395207+0000", 2024-09-18T00:18:58.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0696e-05 2024-09-18T00:18:58.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395221+0000", 2024-09-18T00:18:58.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3876000000000001e-05 2024-09-18T00:18:58.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.439891+0000", 2024-09-18T00:18:58.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044670407000000002 2024-09-18T00:18:58.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440037+0000", 2024-09-18T00:18:58.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014583700000000001 2024-09-18T00:18:58.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440219+0000", 2024-09-18T00:18:58.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018248 2024-09-18T00:18:58.561 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.561 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.561 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.561 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12545 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:58.561 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:18:58.561 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.149097086000001, 2024-09-18T00:18:58.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045427642999999997, 2024-09-18T00:18:58.562 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.562 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.562 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:58.562 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.562 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.562 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.562 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12545 2024-09-18T00:18:58.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.562 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:18:58.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:18:58.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395243+0000", 2024-09-18T00:18:58.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.206e-06 2024-09-18T00:18:58.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395249+0000", 2024-09-18T00:18:58.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:18:58.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395270+0000", 2024-09-18T00:18:58.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1223e-05 2024-09-18T00:18:58.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395285+0000", 2024-09-18T00:18:58.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4725999999999999e-05 2024-09-18T00:18:58.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440296+0000", 2024-09-18T00:18:58.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045011402999999998 2024-09-18T00:18:58.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440453+0000", 2024-09-18T00:18:58.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015742299999999999 2024-09-18T00:18:58.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440670+0000", 2024-09-18T00:18:58.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000216302 2024-09-18T00:18:58.565 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.565 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.566 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.566 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12546 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:58.566 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:18:58.566 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.149034877999998, 2024-09-18T00:18:58.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045915171999999997, 2024-09-18T00:18:58.566 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.566 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.566 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:58.566 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.566 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.566 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.567 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12546 2024-09-18T00:18:58.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.567 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:18:58.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:18:58.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395306+0000", 2024-09-18T00:18:58.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.206e-06 2024-09-18T00:18:58.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395330+0000", 2024-09-18T00:18:58.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4686e-05 2024-09-18T00:18:58.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395356+0000", 2024-09-18T00:18:58.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5418999999999998e-05 2024-09-18T00:18:58.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395387+0000", 2024-09-18T00:18:58.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0942000000000003e-05 2024-09-18T00:18:58.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440743+0000", 2024-09-18T00:18:58.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045356592000000001 2024-09-18T00:18:58.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440880+0000", 2024-09-18T00:18:58.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013641700000000001 2024-09-18T00:18:58.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441219+0000", 2024-09-18T00:18:58.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033991000000000002 2024-09-18T00:18:58.570 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.570 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.570 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.570 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12547 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 212992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:58.570 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:18:58.570 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.148925545000001, 2024-09-18T00:18:58.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046163691999999999, 2024-09-18T00:18:58.571 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.571 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.571 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:58.571 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.571 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.571 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.571 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12547 2024-09-18T00:18:58.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.571 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:18:58.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:18:58.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395415+0000", 2024-09-18T00:18:58.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.553e-06 2024-09-18T00:18:58.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395426+0000", 2024-09-18T00:18:58.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0682999999999999e-05 2024-09-18T00:18:58.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395449+0000", 2024-09-18T00:18:58.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3082999999999999e-05 2024-09-18T00:18:58.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395469+0000", 2024-09-18T00:18:58.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0173000000000002e-05 2024-09-18T00:18:58.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441289+0000", 2024-09-18T00:18:58.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045820195000000001 2024-09-18T00:18:58.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441426+0000", 2024-09-18T00:18:58.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013712699999999999 2024-09-18T00:18:58.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441577+0000", 2024-09-18T00:18:58.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015087799999999999 2024-09-18T00:18:58.574 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.575 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.575 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.575 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12548 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:58.575 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:18:58.575 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.148851784000001, 2024-09-18T00:18:58.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046293932000000003, 2024-09-18T00:18:58.575 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.575 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.575 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:58.575 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.575 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.576 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.576 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12548 2024-09-18T00:18:58.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.576 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:18:58.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:18:58.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395489+0000", 2024-09-18T00:18:58.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.22e-06 2024-09-18T00:18:58.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395512+0000", 2024-09-18T00:18:58.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3312e-05 2024-09-18T00:18:58.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395621+0000", 2024-09-18T00:18:58.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010859 2024-09-18T00:18:58.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395689+0000", 2024-09-18T00:18:58.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8232000000000007e-05 2024-09-18T00:18:58.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441610+0000", 2024-09-18T00:18:58.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045920751000000003 2024-09-18T00:18:58.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441695+0000", 2024-09-18T00:18:58.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5379999999999999e-05 2024-09-18T00:18:58.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441781+0000", 2024-09-18T00:18:58.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6446999999999994e-05 2024-09-18T00:18:58.579 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.579 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.579 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.579 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12549 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 294912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:58.579 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:18:58.580 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.148591132, 2024-09-18T00:18:58.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046410264999999999, 2024-09-18T00:18:58.580 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.580 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.580 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:58.580 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.580 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.580 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.580 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12549 2024-09-18T00:18:58.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.580 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:18:58.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:18:58.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395750+0000", 2024-09-18T00:18:58.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:18:58.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395758+0000", 2024-09-18T00:18:58.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5259999999999993e-06 2024-09-18T00:18:58.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395793+0000", 2024-09-18T00:18:58.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4459000000000003e-05 2024-09-18T00:18:58.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395809+0000", 2024-09-18T00:18:58.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6433e-05 2024-09-18T00:18:58.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441851+0000", 2024-09-18T00:18:58.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046041493000000003 2024-09-18T00:18:58.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442002+0000", 2024-09-18T00:18:58.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151433 2024-09-18T00:18:58.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442158+0000", 2024-09-18T00:18:58.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000156358 2024-09-18T00:18:58.583 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.584 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.584 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.584 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12550 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:58.584 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:18:58.584 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.148479643000002, 2024-09-18T00:18:58.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046498398000000003, 2024-09-18T00:18:58.584 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.584 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.584 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:58.584 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.584 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.584 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.585 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12550 2024-09-18T00:18:58.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.585 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:18:58.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:18:58.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395861+0000", 2024-09-18T00:18:58.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.234e-06 2024-09-18T00:18:58.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395872+0000", 2024-09-18T00:18:58.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1409e-05 2024-09-18T00:18:58.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395892+0000", 2024-09-18T00:18:58.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9680000000000001e-05 2024-09-18T00:18:58.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395911+0000", 2024-09-18T00:18:58.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9283000000000002e-05 2024-09-18T00:18:58.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442210+0000", 2024-09-18T00:18:58.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046298372999999997 2024-09-18T00:18:58.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442279+0000", 2024-09-18T00:18:58.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9055999999999997e-05 2024-09-18T00:18:58.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442358+0000", 2024-09-18T00:18:58.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9363000000000006e-05 2024-09-18T00:18:58.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.588 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.588 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12551 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:58.588 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:18:58.588 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.147807190999998, 2024-09-18T00:18:58.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046066902999999999, 2024-09-18T00:18:58.588 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.589 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.589 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:58.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.589 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12551 2024-09-18T00:18:58.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.589 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:18:58.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:18:58.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396534+0000", 2024-09-18T00:18:58.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6869999999999999e-06 2024-09-18T00:18:58.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396543+0000", 2024-09-18T00:18:58.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3430000000000002e-06 2024-09-18T00:18:58.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396563+0000", 2024-09-18T00:18:58.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0063000000000001e-05 2024-09-18T00:18:58.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396579+0000", 2024-09-18T00:18:58.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5610000000000001e-05 2024-09-18T00:18:58.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442434+0000", 2024-09-18T00:18:58.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045855094999999998 2024-09-18T00:18:58.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442515+0000", 2024-09-18T00:18:58.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0988000000000005e-05 2024-09-18T00:18:58.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442599+0000", 2024-09-18T00:18:58.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4116999999999994e-05 2024-09-18T00:18:58.592 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.592 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.592 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.593 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12552 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:58.593 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:18:58.593 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.147732389000002, 2024-09-18T00:18:58.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046210804000000001, 2024-09-18T00:18:58.593 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.593 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.593 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:58.593 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.593 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.593 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.593 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12552 2024-09-18T00:18:58.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.594 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:18:58.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:18:58.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396608+0000", 2024-09-18T00:18:58.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4729999999999999e-06 2024-09-18T00:18:58.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396614+0000", 2024-09-18T00:18:58.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1e-06 2024-09-18T00:18:58.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396633+0000", 2024-09-18T00:18:58.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9009e-05 2024-09-18T00:18:58.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396650+0000", 2024-09-18T00:18:58.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7045999999999999e-05 2024-09-18T00:18:58.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442662+0000", 2024-09-18T00:18:58.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046011170999999997 2024-09-18T00:18:58.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442737+0000", 2024-09-18T00:18:58.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5759000000000004e-05 2024-09-18T00:18:58.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442818+0000", 2024-09-18T00:18:58.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0246000000000004e-05 2024-09-18T00:18:58.597 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.597 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.597 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.597 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12553 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:58.597 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:18:58.597 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.147669685, 2024-09-18T00:18:58.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046339454000000002, 2024-09-18T00:18:58.598 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.598 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.598 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:58.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.598 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12553 2024-09-18T00:18:58.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.598 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:18:58.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:18:58.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396671+0000", 2024-09-18T00:18:58.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.314e-06 2024-09-18T00:18:58.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396676+0000", 2024-09-18T00:18:58.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6659999999999996e-06 2024-09-18T00:18:58.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396695+0000", 2024-09-18T00:18:58.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8383e-05 2024-09-18T00:18:58.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396709+0000", 2024-09-18T00:18:58.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4309e-05 2024-09-18T00:18:58.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442871+0000", 2024-09-18T00:18:58.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046162295999999998 2024-09-18T00:18:58.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442939+0000", 2024-09-18T00:18:58.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7607999999999994e-05 2024-09-18T00:18:58.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443009+0000", 2024-09-18T00:18:58.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9877999999999997e-05 2024-09-18T00:18:58.601 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.601 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.601 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.602 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12554 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:58.602 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:18:58.602 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.147611763, 2024-09-18T00:18:58.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046504050999999998, 2024-09-18T00:18:58.602 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.602 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.602 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:58.602 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.602 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.602 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.602 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12554 2024-09-18T00:18:58.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.602 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:18:58.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:18:58.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396729+0000", 2024-09-18T00:18:58.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2559999999999999e-06 2024-09-18T00:18:58.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396736+0000", 2024-09-18T00:18:58.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6140000000000003e-06 2024-09-18T00:18:58.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396755+0000", 2024-09-18T00:18:58.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8221999999999999e-05 2024-09-18T00:18:58.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396769+0000", 2024-09-18T00:18:58.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4372999999999999e-05 2024-09-18T00:18:58.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443078+0000", 2024-09-18T00:18:58.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046308850999999998 2024-09-18T00:18:58.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443152+0000", 2024-09-18T00:18:58.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4083000000000002e-05 2024-09-18T00:18:58.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443231+0000", 2024-09-18T00:18:58.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9652000000000002e-05 2024-09-18T00:18:58.606 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.606 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.606 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.606 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12632 3.3 3:c21fac02:::rbd_data.112a7b1589a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1490944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:58.606 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:18:58.606 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.141053478, 2024-09-18T00:18:58.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045843916999999998, 2024-09-18T00:18:58.606 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.606 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.607 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.607 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.607 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.607 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.607 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12632 2024-09-18T00:18:58.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.607 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:18:58.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:18:58.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403288+0000", 2024-09-18T00:18:58.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3630000000000001e-06 2024-09-18T00:18:58.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403886+0000", 2024-09-18T00:18:58.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00059840100000000003 2024-09-18T00:18:58.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403913+0000", 2024-09-18T00:18:58.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6376e-05 2024-09-18T00:18:58.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403927+0000", 2024-09-18T00:18:58.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4188e-05 2024-09-18T00:18:58.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443268+0000", 2024-09-18T00:18:58.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.039340591000000001 2024-09-18T00:18:58.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443488+0000", 2024-09-18T00:18:58.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000220545 2024-09-18T00:18:58.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:58.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443918+0000", 2024-09-18T00:18:58.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043021800000000001 2024-09-18T00:18:58.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:58.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.448558+0000", 2024-09-18T00:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046391449999999999 2024-09-18T00:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449015+0000", 2024-09-18T00:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045766600000000002 2024-09-18T00:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449085+0000", 2024-09-18T00:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0191999999999995e-05 2024-09-18T00:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449130+0000", 2024-09-18T00:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4231999999999999e-05 2024-09-18T00:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:19386 3.1d 3:bcd89fc1:::rbd_data.112a7fe29ab4.00000000000000c8:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:53.699994+0000", 2024-09-18T00:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.8443453359999999, 2024-09-18T00:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050613197999999998, 2024-09-18T00:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.612 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 19386 2024-09-18T00:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.699994+0000", 2024-09-18T00:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.699994+0000", 2024-09-18T00:18:58.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.699996+0000", 2024-09-18T00:18:58.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2330000000000001e-06 2024-09-18T00:18:58.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700007+0000", 2024-09-18T00:18:58.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1199000000000001e-05 2024-09-18T00:18:58.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700039+0000", 2024-09-18T00:18:58.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1362999999999998e-05 2024-09-18T00:18:58.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700054+0000", 2024-09-18T00:18:58.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4970000000000001e-05 2024-09-18T00:18:58.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700229+0000", 2024-09-18T00:18:58.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017526600000000001 2024-09-18T00:18:58.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700488+0000", 2024-09-18T00:18:58.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025918099999999998 2024-09-18T00:18:58.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:58.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750393+0000", 2024-09-18T00:18:58.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049904733 2024-09-18T00:18:58.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:58.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750476+0000", 2024-09-18T00:18:58.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3413999999999998e-05 2024-09-18T00:18:58.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750607+0000", 2024-09-18T00:18:58.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130839 2024-09-18T00:18:58.617 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.617 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.617 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.617 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:19387 3.1d 3:bc19744b:::rbd_data.112a7fe29ab4.00000000000000c9:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:18:58.617 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:53.700113+0000", 2024-09-18T00:18:58.617 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.844225926, 2024-09-18T00:18:58.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050693338999999997, 2024-09-18T00:18:58.618 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.618 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.618 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.618 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 19387 2024-09-18T00:18:58.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.618 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700113+0000", 2024-09-18T00:18:58.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700113+0000", 2024-09-18T00:18:58.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700115+0000", 2024-09-18T00:18:58.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3599999999999999e-06 2024-09-18T00:18:58.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700130+0000", 2024-09-18T00:18:58.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5396999999999998e-05 2024-09-18T00:18:58.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700147+0000", 2024-09-18T00:18:58.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7255e-05 2024-09-18T00:18:58.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700158+0000", 2024-09-18T00:18:58.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.079e-05 2024-09-18T00:18:58.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.707161+0000", 2024-09-18T00:18:58.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070025820000000003 2024-09-18T00:18:58.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.707407+0000", 2024-09-18T00:18:58.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024667100000000001 2024-09-18T00:18:58.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:58.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750638+0000", 2024-09-18T00:18:58.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043230365 2024-09-18T00:18:58.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:58.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750704+0000", 2024-09-18T00:18:58.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6806999999999999e-05 2024-09-18T00:18:58.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750807+0000", 2024-09-18T00:18:58.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102112 2024-09-18T00:18:58.622 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.622 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.622 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.623 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:22270 3.14 3:2badd31c:::rbd_data.112a948d5956.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:18:58.623 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.908651+0000", 2024-09-18T00:18:58.623 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.63568821900000005, 2024-09-18T00:18:58.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.081875197999999996, 2024-09-18T00:18:58.623 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.623 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.623 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.623 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.623 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.623 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.623 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 22270 2024-09-18T00:18:58.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.623 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908651+0000", 2024-09-18T00:18:58.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908651+0000", 2024-09-18T00:18:58.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908653+0000", 2024-09-18T00:18:58.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2290000000000002e-06 2024-09-18T00:18:58.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908665+0000", 2024-09-18T00:18:58.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.225e-05 2024-09-18T00:18:58.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908697+0000", 2024-09-18T00:18:58.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1306000000000001e-05 2024-09-18T00:18:58.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908712+0000", 2024-09-18T00:18:58.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5466000000000001e-05 2024-09-18T00:18:58.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908889+0000", 2024-09-18T00:18:58.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017653999999999999 2024-09-18T00:18:58.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.909023+0000", 2024-09-18T00:18:58.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013473700000000001 2024-09-18T00:18:58.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-18T00:18:58.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.913637+0000", 2024-09-18T00:18:58.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046132600000000001 2024-09-18T00:18:58.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:58.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990060+0000", 2024-09-18T00:18:58.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.076422834999999995 2024-09-18T00:18:58.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:58.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990206+0000", 2024-09-18T00:18:58.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014591499999999999 2024-09-18T00:18:58.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:58.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990391+0000", 2024-09-18T00:18:58.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018554300000000001 2024-09-18T00:18:58.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990526+0000", 2024-09-18T00:18:58.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000135117 2024-09-18T00:18:58.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.628 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.628 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:22270 3.14 e64/32 ondisk, result = 0)", 2024-09-18T00:18:58.628 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.926819+0000", 2024-09-18T00:18:58.628 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.61751995500000001, 2024-09-18T00:18:58.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.063725764000000004, 2024-09-18T00:18:58.629 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.629 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.629 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:58.629 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926819+0000", 2024-09-18T00:18:58.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926819+0000", 2024-09-18T00:18:58.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926821+0000", 2024-09-18T00:18:58.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5540000000000001e-06 2024-09-18T00:18:58.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926828+0000", 2024-09-18T00:18:58.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9260000000000003e-06 2024-09-18T00:18:58.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926857+0000", 2024-09-18T00:18:58.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9357e-05 2024-09-18T00:18:58.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926878+0000", 2024-09-18T00:18:58.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0571999999999999e-05 2024-09-18T00:18:58.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990155+0000", 2024-09-18T00:18:58.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.063277618999999993 2024-09-18T00:18:58.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990187+0000", 2024-09-18T00:18:58.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1322999999999998e-05 2024-09-18T00:18:58.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990545+0000", 2024-09-18T00:18:58.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00035841300000000003 2024-09-18T00:18:58.632 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.632 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.632 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.632 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:22272 3.3 3:c617f123:::rbd_data.112a948d5956.0000000000000009:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:18:58.632 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:18:58.632 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.55324149199999995, 2024-09-18T00:18:58.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10752577300000001, 2024-09-18T00:18:58.633 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.633 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.633 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:58.633 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:58.633 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:58.633 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:58.633 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 22272 2024-09-18T00:18:58.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.633 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:18:58.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:18:58.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991102+0000", 2024-09-18T00:18:58.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9369999999999997e-06 2024-09-18T00:18:58.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991118+0000", 2024-09-18T00:18:58.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6396e-05 2024-09-18T00:18:58.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991150+0000", 2024-09-18T00:18:58.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2132999999999999e-05 2024-09-18T00:18:58.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991170+0000", 2024-09-18T00:18:58.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0085000000000002e-05 2024-09-18T00:18:58.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991370+0000", 2024-09-18T00:18:58.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019963700000000001 2024-09-18T00:18:58.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991518+0000", 2024-09-18T00:18:58.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000148424 2024-09-18T00:18:58.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:58.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.998850+0000", 2024-09-18T00:18:58.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073315480000000002 2024-09-18T00:18:58.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:58.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098178+0000", 2024-09-18T00:18:58.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.099328067000000006 2024-09-18T00:18:58.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:58.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098314+0000", 2024-09-18T00:18:58.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000135678 2024-09-18T00:18:58.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:58.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098498+0000", 2024-09-18T00:18:58.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018479600000000001 2024-09-18T00:18:58.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098623+0000", 2024-09-18T00:18:58.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000125072 2024-09-18T00:18:58.638 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.638 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.638 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.638 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:22272 3.3 e64/23 ondisk, result = 0)", 2024-09-18T00:18:58.638 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:58.023226+0000", 2024-09-18T00:18:58.638 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.52111319700000003, 2024-09-18T00:18:58.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.075416677000000001, 2024-09-18T00:18:58.638 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:58.638 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:58.638 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:58.639 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:58.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:58.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023226+0000", 2024-09-18T00:18:58.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:58.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023226+0000", 2024-09-18T00:18:58.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:58.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:58.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023228+0000", 2024-09-18T00:18:58.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0600000000000002e-06 2024-09-18T00:18:58.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:58.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023237+0000", 2024-09-18T00:18:58.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6430000000000001e-06 2024-09-18T00:18:58.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:58.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023271+0000", 2024-09-18T00:18:58.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4091999999999998e-05 2024-09-18T00:18:58.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:58.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023293+0000", 2024-09-18T00:18:58.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2103000000000002e-05 2024-09-18T00:18:58.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:58.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098272+0000", 2024-09-18T00:18:58.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.074979263000000004 2024-09-18T00:18:58.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:58.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098297+0000", 2024-09-18T00:18:58.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5162999999999999e-05 2024-09-18T00:18:58.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:58.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:58.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:58.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098643+0000", 2024-09-18T00:18:58.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034535300000000001 2024-09-18T00:18:58.642 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.642 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.642 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.642 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:58.642 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:58.642 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:58.644 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:18:58.645 INFO:teuthology.orchestra.run.smithi097.stdout:3762 writesame 0xae275d7 thru 0xae2f121 data_size 0x3f(0x7b4b bytes) 2024-09-18T00:18:58.645 INFO:teuthology.orchestra.run.smithi097.stdout:3763 read 0xa37ec0b thru 0xa3878f3 (0x8ce9 bytes) 2024-09-18T00:18:58.645 INFO:teuthology.orchestra.run.smithi097.stdout:3764 read 0x822b085 thru 0x822e9ba (0x3936 bytes) 2024-09-18T00:18:58.645 INFO:teuthology.orchestra.run.smithi097.stdout:3765 read 0x1193f75 thru 0x119f502 (0xb58e bytes) 2024-09-18T00:18:58.645 INFO:teuthology.orchestra.run.smithi097.stdout:3766 writesame 0xe88a00d thru 0xe8970cf data_size 0x1(0xd0c3 bytes) 2024-09-18T00:18:58.645 INFO:teuthology.orchestra.run.smithi097.stdout:3767 punch from 0x311256f to 0x31134cd, (0xf5e bytes) 2024-09-18T00:18:58.645 INFO:teuthology.orchestra.run.smithi097.stdout:3768 write 0xe989278 thru 0xe98a462 (0x11eb bytes) 2024-09-18T00:18:58.646 INFO:teuthology.orchestra.run.smithi097.stdout:3769 read 0xca2d9ae thru 0xca34020 (0x6673 bytes) 2024-09-18T00:18:58.646 INFO:teuthology.orchestra.run.smithi097.stdout:3770 read 0x79a03b3 thru 0x79ad4f2 (0xd140 bytes) 2024-09-18T00:18:58.646 INFO:teuthology.orchestra.run.smithi097.stdout:3771 write 0x74e1063 thru 0x74e5cd2 (0x4c70 bytes) 2024-09-18T00:18:58.646 INFO:teuthology.orchestra.run.smithi097.stdout:3773 trunc from 0xe98a463 to 0x1694bcc 2024-09-18T00:18:58.646 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:58.638+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:58.668 INFO:teuthology.orchestra.run.smithi097.stdout:3774 punch from 0x97a067 to 0x97a4a2, (0x43b bytes) 2024-09-18T00:18:58.668 INFO:teuthology.orchestra.run.smithi097.stdout:3775 punch from 0x158b138 to 0x158b513, (0x3db bytes) 2024-09-18T00:18:58.668 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:58.696 INFO:teuthology.orchestra.run.smithi097.stdout:3777 writesame 0x6f0d0c2 thru 0x6f110a9 data_size 0xa(0x3fe8 bytes) 2024-09-18T00:18:58.698 INFO:teuthology.orchestra.run.smithi097.stdout:3779 read 0x59a5718 thru 0x59ab839 (0x6122 bytes) 2024-09-18T00:18:58.699 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:58.699 INFO:teuthology.orchestra.run.smithi097.stdout:3782 writesame 0x27f8b63 thru 0x27fdc03 data_size 0x1(0x50a1 bytes) 2024-09-18T00:18:58.744 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:18:58.738+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:18:58.750 INFO:teuthology.orchestra.run.smithi097.stdout:3783 writesame 0xc7fc86c thru 0xc801433 data_size 0x32(0x4bc8 bytes) 2024-09-18T00:18:58.753 INFO:teuthology.orchestra.run.smithi097.stdout:3784 write 0x1d52aef thru 0x1d52f07 (0x419 bytes) 2024-09-18T00:18:58.756 INFO:teuthology.orchestra.run.smithi097.stdout:3785 read 0xd81925 thru 0xd887be (0x6e9a bytes) 2024-09-18T00:18:58.759 INFO:teuthology.orchestra.run.smithi097.stdout:3786 write 0xc154e97 thru 0xc15c6fe (0x7868 bytes) 2024-09-18T00:18:58.768 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:58.768 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:58.768 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:58.768 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:58.769 INFO:teuthology.orchestra.run.smithi097.stdout:3793 read 0x60bee3c thru 0x60c0ba5 (0x1d6a bytes) 2024-09-18T00:18:58.770 INFO:teuthology.orchestra.run.smithi097.stdout:3796 writesame 0x83ba92f thru 0x83c2cb5 data_size 0x1(0x8387 bytes) 2024-09-18T00:18:58.776 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is unset 2024-09-18T00:18:58.818 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:18:58.819 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:58.826 INFO:teuthology.orchestra.run.smithi097.stdout:3798 writesame 0x95026a thru 0x95cf78 data_size 0x1(0xcd0f bytes) 2024-09-18T00:18:58.833 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:18:58.878 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:18:58.878+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:18:58.915 INFO:teuthology.orchestra.run.smithi097.stdout:3800 read 0x57551cd thru 0x5757c0f (0x2a43 bytes) 2024-09-18T00:18:58.916 INFO:teuthology.orchestra.run.smithi097.stdout:3801 punch from 0x4c3cf51 to 0x4c4027b, (0x332a bytes) 2024-09-18T00:18:58.916 INFO:teuthology.orchestra.run.smithi097.stdout:3802 write 0xb4c70b3 thru 0xb4cb29d (0x41eb bytes) 2024-09-18T00:18:58.921 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:58.921 INFO:teuthology.orchestra.run.smithi097.stdout:3807 read 0x4aceaa6 thru 0x4add530 (0xea8b bytes) 2024-09-18T00:18:58.921 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:58.921 INFO:teuthology.orchestra.run.smithi097.stdout:3809 write 0x9386d7 thru 0x9402f3 (0x7c1d bytes) 2024-09-18T00:18:58.925 INFO:teuthology.orchestra.run.smithi097.stdout:3811 write 0xa650da2 thru 0xa653374 (0x25d3 bytes) 2024-09-18T00:18:58.929 INFO:teuthology.orchestra.run.smithi097.stdout:3812 read 0xc0c34c7 thru 0xc0d3242 (0xfd7c bytes) 2024-09-18T00:18:58.930 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:58.930 INFO:teuthology.orchestra.run.smithi097.stdout:3814 read 0xc291d8f thru 0xc2a1d05 (0xff77 bytes) 2024-09-18T00:18:58.931 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:58.931 INFO:teuthology.orchestra.run.smithi097.stdout:3816 writesame 0x8d0f75d thru 0x8d1888e data_size 0x69(0x9132 bytes) 2024-09-18T00:18:58.933 INFO:teuthology.orchestra.run.smithi097.stdout:3817 read 0x71843f1 thru 0x7190840 (0xc450 bytes) 2024-09-18T00:18:58.934 INFO:teuthology.orchestra.run.smithi097.stdout:3819 write 0x36a73bb thru 0x36b6493 (0xf0d9 bytes) 2024-09-18T00:18:58.943 INFO:teuthology.orchestra.run.smithi097.stdout:3822 write 0x15626d8 thru 0x156f35a (0xcc83 bytes) 2024-09-18T00:18:58.946 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:58.942+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:58.951 INFO:teuthology.orchestra.run.smithi097.stdout:3823 read 0x3dc6b7d thru 0x3dd5ca6 (0xf12a bytes) 2024-09-18T00:18:58.952 INFO:teuthology.orchestra.run.smithi097.stdout:3824 write 0x9e61ae6 thru 0x9e69364 (0x787f bytes) 2024-09-18T00:18:58.957 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:58.958 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:58.958 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:58.958 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:58.958 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:58.965 INFO:teuthology.orchestra.run.smithi097.stdout:3827 read 0x5ee3d46 thru 0x5eefa31 (0xbcec bytes) 2024-09-18T00:18:58.966 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:58.966 INFO:teuthology.orchestra.run.smithi097.stdout:3831 read 0xdffae0 thru 0xe0371a (0x3c3b bytes) 2024-09-18T00:18:58.973 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:18:58.978 INFO:teuthology.orchestra.run.smithi097.stdout:3834 trunc from 0xc801434 to 0x798635f 2024-09-18T00:18:58.992 INFO:teuthology.orchestra.run.smithi097.stdout:3835 read 0x1a11641 thru 0x1a1dab4 (0xc474 bytes) 2024-09-18T00:18:58.992 INFO:teuthology.orchestra.run.smithi097.stdout:3836 read 0x35a999f thru 0x35ab2fe (0x1960 bytes) 2024-09-18T00:18:58.993 INFO:teuthology.orchestra.run.smithi097.stdout:3838 write 0x5ba6e4 thru 0x5bf023 (0x4940 bytes) 2024-09-18T00:18:58.997 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:58.997 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:59.013 INFO:teuthology.orchestra.run.smithi097.stdout:3841 write 0xa1d52ad thru 0xa1dc70d (0x7461 bytes) 2024-09-18T00:18:59.016 INFO:teuthology.orchestra.run.smithi097.stdout:3843 write 0x5505567 thru 0x550e911 (0x93ab bytes) 2024-09-18T00:18:59.020 INFO:teuthology.orchestra.run.smithi097.stdout:3844 read 0x97bccde thru 0x97c30c7 (0x63ea bytes) 2024-09-18T00:18:59.022 INFO:teuthology.orchestra.run.smithi097.stdout:3846 punch from 0x22951ff to 0x2297195, (0x1f96 bytes) 2024-09-18T00:18:59.022 INFO:teuthology.orchestra.run.smithi097.stdout:3847 write 0x8747d72 thru 0x874949d (0x172c bytes) 2024-09-18T00:18:59.024 INFO:teuthology.orchestra.run.smithi097.stdout:3848 read 0x7474b2b thru 0x74778e3 (0x2db9 bytes) 2024-09-18T00:18:59.025 INFO:teuthology.orchestra.run.smithi097.stdout:3849 read 0x3a35866 thru 0x3a41cdd (0xc478 bytes) 2024-09-18T00:18:59.040 INFO:teuthology.orchestra.run.smithi097.stdout:3850 trunc from 0xa1dc70e to 0x3c39438 2024-09-18T00:18:59.055 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:59.055 INFO:teuthology.orchestra.run.smithi097.stdout:3852 writesame 0x243ab5f thru 0x2447e0c data_size 0x65(0xd2ae bytes) 2024-09-18T00:18:59.062 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:59.062+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:59.069 INFO:teuthology.orchestra.run.smithi097.stdout:3853 trunc from 0x3c39438 to 0x811d8f8 2024-09-18T00:18:59.081 INFO:teuthology.orchestra.run.smithi097.stdout:3854 write 0x49fbbff thru 0x49fe5b6 (0x29b8 bytes) 2024-09-18T00:18:59.085 INFO:teuthology.orchestra.run.smithi097.stdout:3855 read 0x183c960 thru 0x1844709 (0x7daa bytes) 2024-09-18T00:18:59.085 INFO:teuthology.orchestra.run.smithi097.stdout:3857 punch from 0x6178791 to 0x6179805, (0x1074 bytes) 2024-09-18T00:18:59.085 INFO:teuthology.orchestra.run.smithi097.stdout:3858 write 0x279624d thru 0x279c55d (0x6311 bytes) 2024-09-18T00:18:59.116 INFO:teuthology.orchestra.run.smithi097.stdout:3859 write 0xe612c93 thru 0xe613060 (0x3ce bytes) 2024-09-18T00:18:59.119 INFO:teuthology.orchestra.run.smithi097.stdout:3861 read 0x7fba4d1 thru 0x7fc1dde (0x790e bytes) 2024-09-18T00:18:59.120 INFO:teuthology.orchestra.run.smithi097.stdout:3862 writesame 0x898b293 thru 0x899100a data_size 0xc(0x5d78 bytes) 2024-09-18T00:18:59.125 INFO:teuthology.orchestra.run.smithi097.stdout:3863 write 0xbb3dc59 thru 0xbb42802 (0x4baa bytes) 2024-09-18T00:18:59.129 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:59.129 INFO:teuthology.orchestra.run.smithi097.stdout:3866 writesame 0x48ebea4 thru 0x48edafc data_size 0x3(0x1c59 bytes) 2024-09-18T00:18:59.132 INFO:teuthology.orchestra.run.smithi097.stdout:3867 write 0xd174255 thru 0xd177015 (0x2dc1 bytes) 2024-09-18T00:18:59.136 INFO:teuthology.orchestra.run.smithi097.stdout:3868 punch from 0x1e10a51 to 0x1e15bcb, (0x517a bytes) 2024-09-18T00:18:59.136 INFO:teuthology.orchestra.run.smithi097.stdout:3870 writesame 0x6cbf194 thru 0x6cc82b0 data_size 0x57(0x911d bytes) 2024-09-18T00:18:59.139 INFO:teuthology.orchestra.run.smithi097.stdout:3871 read 0xa053a22 thru 0xa05dfa4 (0xa583 bytes) 2024-09-18T00:18:59.140 INFO:teuthology.orchestra.run.smithi097.stdout:3872 read 0x7e399bf thru 0x7e47d82 (0xe3c4 bytes) 2024-09-18T00:18:59.141 INFO:teuthology.orchestra.run.smithi097.stdout:3873 writesame 0x3090b58 thru 0x3090cf5 data_size 0x12(0x19e bytes) 2024-09-18T00:18:59.144 INFO:teuthology.orchestra.run.smithi097.stdout:3874 read 0x671d4c4 thru 0x671e023 (0xb60 bytes) 2024-09-18T00:18:59.145 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:59.145 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:18:59.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:18:59.146 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:18:59.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.146 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2229 3.10 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:18:59.146 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:18:59.146 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.879105711999998, 2024-09-18T00:18:59.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048730086999999998, 2024-09-18T00:18:59.146 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.146 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.146 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:59.146 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.146 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.146 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.146 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2229 2024-09-18T00:18:59.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.147 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:18:59.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:18:59.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263362+0000", 2024-09-18T00:18:59.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3869999999999999e-06 2024-09-18T00:18:59.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263381+0000", 2024-09-18T00:18:59.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9153000000000001e-05 2024-09-18T00:18:59.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263418+0000", 2024-09-18T00:18:59.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6346000000000001e-05 2024-09-18T00:18:59.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263435+0000", 2024-09-18T00:18:59.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7309999999999999e-05 2024-09-18T00:18:59.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263589+0000", 2024-09-18T00:18:59.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015451199999999999 2024-09-18T00:18:59.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263902+0000", 2024-09-18T00:18:59.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031291799999999997 2024-09-18T00:18:59.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:59.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.265995+0000", 2024-09-18T00:18:59.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020931809999999999 2024-09-18T00:18:59.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:59.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311788+0000", 2024-09-18T00:18:59.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045792078 2024-09-18T00:18:59.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:59.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311903+0000", 2024-09-18T00:18:59.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011510699999999999 2024-09-18T00:18:59.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:59.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312009+0000", 2024-09-18T00:18:59.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010630599999999999 2024-09-18T00:18:59.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312089+0000", 2024-09-18T00:18:59.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9789000000000005e-05 2024-09-18T00:18:59.151 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.151 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.151 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.151 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2229 3.10 e36/23 ondisk, result = 0)", 2024-09-18T00:18:59.151 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:18:59.152 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.859960053000002, 2024-09-18T00:18:59.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029600955000000002, 2024-09-18T00:18:59.152 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.152 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.152 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:59.152 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:18:59.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:18:59.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282507+0000", 2024-09-18T00:18:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9160000000000001e-06 2024-09-18T00:18:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282522+0000", 2024-09-18T00:18:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5e-05 2024-09-18T00:18:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282558+0000", 2024-09-18T00:18:59.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5694000000000001e-05 2024-09-18T00:18:59.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282580+0000", 2024-09-18T00:18:59.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2323e-05 2024-09-18T00:18:59.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311873+0000", 2024-09-18T00:18:59.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029293013999999999 2024-09-18T00:18:59.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311892+0000", 2024-09-18T00:18:59.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8556e-05 2024-09-18T00:18:59.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312105+0000", 2024-09-18T00:18:59.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021345199999999999 2024-09-18T00:18:59.155 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.155 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.155 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.155 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5714 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:18:59.155 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:18:59.155 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.832257550000001, 2024-09-18T00:18:59.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027448727999999999, 2024-09-18T00:18:59.155 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.156 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.156 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:59.156 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.156 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.156 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.156 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5714 2024-09-18T00:18:59.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.156 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:18:59.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:18:59.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310211+0000", 2024-09-18T00:18:59.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5639999999999998e-06 2024-09-18T00:18:59.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312678+0000", 2024-09-18T00:18:59.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002466761 2024-09-18T00:18:59.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312746+0000", 2024-09-18T00:18:59.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8046000000000005e-05 2024-09-18T00:18:59.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312774+0000", 2024-09-18T00:18:59.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7813e-05 2024-09-18T00:18:59.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312927+0000", 2024-09-18T00:18:59.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015328499999999999 2024-09-18T00:18:59.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313214+0000", 2024-09-18T00:18:59.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028683099999999998 2024-09-18T00:18:59.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:59.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313708+0000", 2024-09-18T00:18:59.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049379100000000004 2024-09-18T00:18:59.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:59.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336895+0000", 2024-09-18T00:18:59.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023186801 2024-09-18T00:18:59.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:59.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337454+0000", 2024-09-18T00:18:59.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055979399999999996 2024-09-18T00:18:59.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:59.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337571+0000", 2024-09-18T00:18:59.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116821 2024-09-18T00:18:59.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337656+0000", 2024-09-18T00:18:59.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4221000000000006e-05 2024-09-18T00:18:59.160 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.160 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.160 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.160 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6801 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:18:59.161 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:18:59.161 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.105421102000001, 2024-09-18T00:18:59.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051953602000000002, 2024-09-18T00:18:59.161 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.161 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.161 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:59.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.161 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6801 2024-09-18T00:18:59.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.161 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:18:59.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:18:59.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037046+0000", 2024-09-18T00:18:59.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.19e-06 2024-09-18T00:18:59.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037062+0000", 2024-09-18T00:18:59.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586999999999998e-05 2024-09-18T00:18:59.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037096+0000", 2024-09-18T00:18:59.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4112000000000001e-05 2024-09-18T00:18:59.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037120+0000", 2024-09-18T00:18:59.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.355e-05 2024-09-18T00:18:59.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037246+0000", 2024-09-18T00:18:59.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012652200000000001 2024-09-18T00:18:59.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037385+0000", 2024-09-18T00:18:59.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013924599999999999 2024-09-18T00:18:59.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:18:59.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.040881+0000", 2024-09-18T00:18:59.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034954830000000002 2024-09-18T00:18:59.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:59.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088794+0000", 2024-09-18T00:18:59.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047912897000000003 2024-09-18T00:18:59.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:59.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088828+0000", 2024-09-18T00:18:59.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3880000000000001e-05 2024-09-18T00:18:59.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:59.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088916+0000", 2024-09-18T00:18:59.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.865e-05 2024-09-18T00:18:59.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088997+0000", 2024-09-18T00:18:59.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0484999999999999e-05 2024-09-18T00:18:59.165 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.166 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.166 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.166 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:6801 3.10 e42/23 ondisk, result = 0)", 2024-09-18T00:18:59.166 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:18:59.166 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.091613692000003, 2024-09-18T00:18:59.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037960394000000001, 2024-09-18T00:18:59.166 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.166 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.166 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:59.166 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:18:59.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:18:59.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050852+0000", 2024-09-18T00:18:59.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:18:59.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050860+0000", 2024-09-18T00:18:59.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7729999999999999e-06 2024-09-18T00:18:59.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050891+0000", 2024-09-18T00:18:59.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0617000000000002e-05 2024-09-18T00:18:59.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050910+0000", 2024-09-18T00:18:59.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.944e-05 2024-09-18T00:18:59.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088766+0000", 2024-09-18T00:18:59.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037855633999999999 2024-09-18T00:18:59.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088784+0000", 2024-09-18T00:18:59.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7767000000000001e-05 2024-09-18T00:18:59.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088811+0000", 2024-09-18T00:18:59.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7382999999999999e-05 2024-09-18T00:18:59.169 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.169 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.169 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.169 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12594 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:59.170 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:18:59.170 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.750837153999999, 2024-09-18T00:18:59.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026492600000000002, 2024-09-18T00:18:59.170 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.170 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.170 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:59.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.170 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12594 2024-09-18T00:18:59.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.170 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:18:59.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:18:59.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391629+0000", 2024-09-18T00:18:59.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.86e-06 2024-09-18T00:18:59.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391637+0000", 2024-09-18T00:18:59.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3999999999999992e-06 2024-09-18T00:18:59.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391664+0000", 2024-09-18T00:18:59.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6536000000000001e-05 2024-09-18T00:18:59.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391681+0000", 2024-09-18T00:18:59.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6563e-05 2024-09-18T00:18:59.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417791+0000", 2024-09-18T00:18:59.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026110936000000001 2024-09-18T00:18:59.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417933+0000", 2024-09-18T00:18:59.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014112099999999999 2024-09-18T00:18:59.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418120+0000", 2024-09-18T00:18:59.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018718399999999999 2024-09-18T00:18:59.173 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.173 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.173 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.174 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12595 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:59.174 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:18:59.174 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.750760956000001, 2024-09-18T00:18:59.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026819428999999999, 2024-09-18T00:18:59.174 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.174 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.174 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:59.174 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.174 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.174 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.174 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12595 2024-09-18T00:18:59.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.175 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:18:59.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:18:59.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391705+0000", 2024-09-18T00:18:59.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3999999999999999e-06 2024-09-18T00:18:59.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391711+0000", 2024-09-18T00:18:59.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7459999999999997e-06 2024-09-18T00:18:59.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391733+0000", 2024-09-18T00:18:59.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2923000000000001e-05 2024-09-18T00:18:59.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391748+0000", 2024-09-18T00:18:59.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4790000000000001e-05 2024-09-18T00:18:59.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418180+0000", 2024-09-18T00:18:59.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026431888000000001 2024-09-18T00:18:59.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418328+0000", 2024-09-18T00:18:59.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000147725 2024-09-18T00:18:59.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418523+0000", 2024-09-18T00:18:59.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000194957 2024-09-18T00:18:59.177 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.178 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.178 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.178 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12596 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:59.178 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:18:59.178 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.750697987999999, 2024-09-18T00:18:59.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027220296000000001, 2024-09-18T00:18:59.178 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.178 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.178 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:59.178 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.178 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.178 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.178 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12596 2024-09-18T00:18:59.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.179 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:18:59.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:18:59.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391768+0000", 2024-09-18T00:18:59.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.283e-06 2024-09-18T00:18:59.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391773+0000", 2024-09-18T00:18:59.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3970000000000002e-06 2024-09-18T00:18:59.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391795+0000", 2024-09-18T00:18:59.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1835e-05 2024-09-18T00:18:59.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391809+0000", 2024-09-18T00:18:59.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4107000000000001e-05 2024-09-18T00:18:59.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418582+0000", 2024-09-18T00:18:59.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026772918999999999 2024-09-18T00:18:59.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418780+0000", 2024-09-18T00:18:59.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000198042 2024-09-18T00:18:59.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418987+0000", 2024-09-18T00:18:59.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000206713 2024-09-18T00:18:59.182 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.182 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.182 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.182 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12597 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:59.182 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:18:59.182 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.750637470000001, 2024-09-18T00:18:59.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027612714, 2024-09-18T00:18:59.182 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.182 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.182 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:59.182 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.182 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.182 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.183 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12597 2024-09-18T00:18:59.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.183 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:18:59.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:18:59.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391828+0000", 2024-09-18T00:18:59.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.203e-06 2024-09-18T00:18:59.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391834+0000", 2024-09-18T00:18:59.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6529999999999999e-06 2024-09-18T00:18:59.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391856+0000", 2024-09-18T00:18:59.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2195999999999999e-05 2024-09-18T00:18:59.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391870+0000", 2024-09-18T00:18:59.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4063000000000001e-05 2024-09-18T00:18:59.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419049+0000", 2024-09-18T00:18:59.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027178751000000001 2024-09-18T00:18:59.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419239+0000", 2024-09-18T00:18:59.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019039400000000001 2024-09-18T00:18:59.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419440+0000", 2024-09-18T00:18:59.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000200454 2024-09-18T00:18:59.186 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.186 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.186 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.186 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12598 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:59.186 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:18:59.186 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.750574418999999, 2024-09-18T00:18:59.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028269056000000001, 2024-09-18T00:18:59.186 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.186 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.187 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:59.187 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.187 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.187 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.187 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12598 2024-09-18T00:18:59.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.187 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:18:59.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:18:59.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391891+0000", 2024-09-18T00:18:59.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.257e-06 2024-09-18T00:18:59.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391897+0000", 2024-09-18T00:18:59.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9560000000000002e-06 2024-09-18T00:18:59.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391920+0000", 2024-09-18T00:18:59.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3025999999999999e-05 2024-09-18T00:18:59.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391935+0000", 2024-09-18T00:18:59.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4853e-05 2024-09-18T00:18:59.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419527+0000", 2024-09-18T00:18:59.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027592378000000001 2024-09-18T00:18:59.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419681+0000", 2024-09-18T00:18:59.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015316 2024-09-18T00:18:59.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420159+0000", 2024-09-18T00:18:59.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000478426 2024-09-18T00:18:59.190 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.190 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.190 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.190 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12599 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:59.190 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:18:59.190 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.750510563999999, 2024-09-18T00:18:59.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028649857000000001, 2024-09-18T00:18:59.190 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.191 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.191 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:59.191 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.191 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.191 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.191 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12599 2024-09-18T00:18:59.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.191 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:18:59.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:18:59.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391955+0000", 2024-09-18T00:18:59.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-06 2024-09-18T00:18:59.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391961+0000", 2024-09-18T00:18:59.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7100000000000004e-06 2024-09-18T00:18:59.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391982+0000", 2024-09-18T00:18:59.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1492e-05 2024-09-18T00:18:59.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391996+0000", 2024-09-18T00:18:59.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4194e-05 2024-09-18T00:18:59.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420237+0000", 2024-09-18T00:18:59.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028240818000000001 2024-09-18T00:18:59.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420430+0000", 2024-09-18T00:18:59.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019228299999999999 2024-09-18T00:18:59.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420604+0000", 2024-09-18T00:18:59.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000174117 2024-09-18T00:18:59.194 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.194 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.194 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.194 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12600 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 270336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:59.194 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:18:59.195 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.750449570000001, 2024-09-18T00:18:59.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028964586, 2024-09-18T00:18:59.195 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.195 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.195 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:59.195 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.195 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.195 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.195 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12600 2024-09-18T00:18:59.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.195 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:18:59.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:18:59.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392016+0000", 2024-09-18T00:18:59.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.31e-06 2024-09-18T00:18:59.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392022+0000", 2024-09-18T00:18:59.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5600000000000001e-06 2024-09-18T00:18:59.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392045+0000", 2024-09-18T00:18:59.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3595999999999999e-05 2024-09-18T00:18:59.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392060+0000", 2024-09-18T00:18:59.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4727e-05 2024-09-18T00:18:59.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420669+0000", 2024-09-18T00:18:59.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028608855999999998 2024-09-18T00:18:59.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420813+0000", 2024-09-18T00:18:59.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014429500000000001 2024-09-18T00:18:59.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420979+0000", 2024-09-18T00:18:59.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016624200000000001 2024-09-18T00:18:59.198 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.198 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.198 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.199 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12601 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:59.199 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:18:59.199 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.750385665, 2024-09-18T00:18:59.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029292832000000001, 2024-09-18T00:18:59.199 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.199 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.199 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:59.199 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.199 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.199 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.199 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12601 2024-09-18T00:18:59.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.199 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:18:59.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:18:59.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392080+0000", 2024-09-18T00:18:59.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:18:59.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392086+0000", 2024-09-18T00:18:59.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7570000000000001e-06 2024-09-18T00:18:59.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392105+0000", 2024-09-18T00:18:59.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9415999999999998e-05 2024-09-18T00:18:59.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392132+0000", 2024-09-18T00:18:59.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7019e-05 2024-09-18T00:18:59.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421049+0000", 2024-09-18T00:18:59.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028916659000000001 2024-09-18T00:18:59.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421204+0000", 2024-09-18T00:18:59.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155224 2024-09-18T00:18:59.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421372+0000", 2024-09-18T00:18:59.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016754699999999999 2024-09-18T00:18:59.202 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.202 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.202 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.203 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12602 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:59.203 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:18:59.203 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.750313034000001, 2024-09-18T00:18:59.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029612833000000002, 2024-09-18T00:18:59.203 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.203 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.203 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:59.203 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.203 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.203 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.203 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12602 2024-09-18T00:18:59.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.203 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:18:59.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:18:59.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392152+0000", 2024-09-18T00:18:59.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0470000000000001e-06 2024-09-18T00:18:59.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392158+0000", 2024-09-18T00:18:59.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:18:59.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392181+0000", 2024-09-18T00:18:59.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2629e-05 2024-09-18T00:18:59.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392222+0000", 2024-09-18T00:18:59.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0772000000000002e-05 2024-09-18T00:18:59.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421441+0000", 2024-09-18T00:18:59.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029219327 2024-09-18T00:18:59.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421608+0000", 2024-09-18T00:18:59.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000166813 2024-09-18T00:18:59.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421764+0000", 2024-09-18T00:18:59.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015628500000000001 2024-09-18T00:18:59.206 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.207 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.207 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.207 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12603 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:59.207 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:18:59.207 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.75022264, 2024-09-18T00:18:59.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029902524, 2024-09-18T00:18:59.207 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.207 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.207 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:59.207 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.207 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.207 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.207 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12603 2024-09-18T00:18:59.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.208 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:18:59.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:18:59.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392243+0000", 2024-09-18T00:18:59.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:18:59.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392260+0000", 2024-09-18T00:18:59.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6860000000000001e-05 2024-09-18T00:18:59.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392282+0000", 2024-09-18T00:18:59.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2713000000000001e-05 2024-09-18T00:18:59.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392297+0000", 2024-09-18T00:18:59.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4375e-05 2024-09-18T00:18:59.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421830+0000", 2024-09-18T00:18:59.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029532668000000002 2024-09-18T00:18:59.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421985+0000", 2024-09-18T00:18:59.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155613 2024-09-18T00:18:59.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422144+0000", 2024-09-18T00:18:59.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000159102 2024-09-18T00:18:59.211 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.211 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.211 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.211 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12604 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:59.211 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:18:59.211 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.750147589000001, 2024-09-18T00:18:59.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030228695, 2024-09-18T00:18:59.211 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.211 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.212 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:59.212 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.212 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.212 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.212 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12604 2024-09-18T00:18:59.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.212 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:18:59.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:18:59.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392318+0000", 2024-09-18T00:18:59.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:18:59.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392337+0000", 2024-09-18T00:18:59.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:18:59.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392364+0000", 2024-09-18T00:18:59.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7288999999999999e-05 2024-09-18T00:18:59.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392379+0000", 2024-09-18T00:18:59.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:18:59.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422209+0000", 2024-09-18T00:18:59.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029829050999999999 2024-09-18T00:18:59.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422388+0000", 2024-09-18T00:18:59.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017988699999999999 2024-09-18T00:18:59.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422545+0000", 2024-09-18T00:18:59.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015701199999999999 2024-09-18T00:18:59.215 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.215 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.215 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.215 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12605 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:59.215 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:18:59.215 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.750063028, 2024-09-18T00:18:59.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030488674, 2024-09-18T00:18:59.215 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.216 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.216 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:59.216 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.216 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.216 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.216 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12605 2024-09-18T00:18:59.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.216 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:18:59.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:18:59.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392403+0000", 2024-09-18T00:18:59.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.883e-06 2024-09-18T00:18:59.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392421+0000", 2024-09-18T00:18:59.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7526000000000001e-05 2024-09-18T00:18:59.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392450+0000", 2024-09-18T00:18:59.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8972e-05 2024-09-18T00:18:59.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392488+0000", 2024-09-18T00:18:59.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8522000000000001e-05 2024-09-18T00:18:59.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422614+0000", 2024-09-18T00:18:59.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030125945000000001 2024-09-18T00:18:59.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422752+0000", 2024-09-18T00:18:59.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000138178 2024-09-18T00:18:59.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422890+0000", 2024-09-18T00:18:59.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013764800000000001 2024-09-18T00:18:59.219 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.219 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.219 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.219 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12606 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:18:59.219 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:18:59.220 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.749954368000001, 2024-09-18T00:18:59.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033727073000000003, 2024-09-18T00:18:59.220 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.220 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.220 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:59.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.220 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12606 2024-09-18T00:18:59.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.220 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:18:59.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:18:59.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392511+0000", 2024-09-18T00:18:59.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3090000000000001e-06 2024-09-18T00:18:59.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392517+0000", 2024-09-18T00:18:59.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.993e-06 2024-09-18T00:18:59.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392543+0000", 2024-09-18T00:18:59.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6152999999999998e-05 2024-09-18T00:18:59.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392558+0000", 2024-09-18T00:18:59.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4610000000000001e-05 2024-09-18T00:18:59.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.425959+0000", 2024-09-18T00:18:59.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033400484000000001 2024-09-18T00:18:59.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426106+0000", 2024-09-18T00:18:59.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014779800000000001 2024-09-18T00:18:59.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426237+0000", 2024-09-18T00:18:59.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013072600000000001 2024-09-18T00:18:59.223 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.223 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.223 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.224 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17562 3.a 3:5578626c:::rbd_data.112a6efb7564.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T00:18:59.224 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:18:59.224 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.1844892070000004, 2024-09-18T00:18:59.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052733453999999999, 2024-09-18T00:18:59.224 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.224 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.224 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:18:59.224 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:18:59.224 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:18:59.224 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:18:59.224 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17562 2024-09-18T00:18:59.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.225 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:18:59.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:18:59.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957977+0000", 2024-09-18T00:18:59.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.119e-06 2024-09-18T00:18:59.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957988+0000", 2024-09-18T00:18:59.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0557000000000001e-05 2024-09-18T00:18:59.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958017+0000", 2024-09-18T00:18:59.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9156e-05 2024-09-18T00:18:59.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958036+0000", 2024-09-18T00:18:59.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8919e-05 2024-09-18T00:18:59.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958144+0000", 2024-09-18T00:18:59.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107905 2024-09-18T00:18:59.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958284+0000", 2024-09-18T00:18:59.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140216 2024-09-18T00:18:59.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:18:59.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.960572+0000", 2024-09-18T00:18:59.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022881860000000002 2024-09-18T00:18:59.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:18:59.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010537+0000", 2024-09-18T00:18:59.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049964664999999998 2024-09-18T00:18:59.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:18:59.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010590+0000", 2024-09-18T00:18:59.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3094999999999999e-05 2024-09-18T00:18:59.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:18:59.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010642+0000", 2024-09-18T00:18:59.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2073000000000001e-05 2024-09-18T00:18:59.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010709+0000", 2024-09-18T00:18:59.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6563000000000006e-05 2024-09-18T00:18:59.229 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.229 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.229 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.229 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17562 3.a e60/23 ondisk, result = 0)", 2024-09-18T00:18:59.229 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:18:59.229 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.1742858359999993, 2024-09-18T00:18:59.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042539307999999998, 2024-09-18T00:18:59.229 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:18:59.229 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:18:59.230 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:18:59.230 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:18:59.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:18:59.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:18:59.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:18:59.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:18:59.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:18:59.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:18:59.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968180+0000", 2024-09-18T00:18:59.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-18T00:18:59.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:18:59.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968188+0000", 2024-09-18T00:18:59.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7419999999999997e-06 2024-09-18T00:18:59.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:18:59.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968217+0000", 2024-09-18T00:18:59.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8926999999999999e-05 2024-09-18T00:18:59.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:18:59.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968231+0000", 2024-09-18T00:18:59.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4168999999999999e-05 2024-09-18T00:18:59.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:18:59.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010575+0000", 2024-09-18T00:18:59.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042343923999999998 2024-09-18T00:18:59.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:18:59.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010584+0000", 2024-09-18T00:18:59.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-18T00:18:59.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:18:59.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:18:59.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:18:59.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010718+0000", 2024-09-18T00:18:59.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013344499999999999 2024-09-18T00:18:59.232 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.233 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.233 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.233 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:18:59.233 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:18:59.233 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:59.235 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:18:59.236 INFO:teuthology.orchestra.run.smithi097.stdout:3876 read 0xd003d73 thru 0xd0092fb (0x5589 bytes) 2024-09-18T00:18:59.236 INFO:teuthology.orchestra.run.smithi097.stdout:3877 write 0xe053243 thru 0xe054e83 (0x1c41 bytes) 2024-09-18T00:18:59.236 INFO:teuthology.orchestra.run.smithi097.stdout:3878 write 0xcc36a5 thru 0xcc3cd5 (0x631 bytes) 2024-09-18T00:18:59.236 INFO:teuthology.orchestra.run.smithi097.stdout:3879 trunc from 0xe613061 to 0xdbdf8f9 2024-09-18T00:18:59.236 INFO:teuthology.orchestra.run.smithi097.stdout:3880 write 0x6f08e20 thru 0x6f11199 (0x837a bytes) 2024-09-18T00:18:59.236 INFO:teuthology.orchestra.run.smithi097.stdout:3881 write 0xa79e025 thru 0xa7a9906 (0xb8e2 bytes) 2024-09-18T00:18:59.236 INFO:teuthology.orchestra.run.smithi097.stdout:3882 writesame 0xe677225 thru 0xe67d397 data_size 0x13(0x6173 bytes) 2024-09-18T00:18:59.236 INFO:teuthology.orchestra.run.smithi097.stdout:3883 punch from 0x1fb0989 to 0x1fbb0be, (0xa735 bytes) 2024-09-18T00:18:59.236 INFO:teuthology.orchestra.run.smithi097.stdout:3884 read 0x8a3a49f thru 0x8a3dbf0 (0x3752 bytes) 2024-09-18T00:18:59.237 INFO:teuthology.orchestra.run.smithi097.stdout:3885 punch from 0x56fea23 to 0x57041b9, (0x5796 bytes) 2024-09-18T00:18:59.237 INFO:teuthology.orchestra.run.smithi097.stdout:3887 read 0x1cdf4ed thru 0x1ce2860 (0x3374 bytes) 2024-09-18T00:18:59.237 INFO:teuthology.orchestra.run.smithi097.stdout:3889 punch from 0x31a46b to 0x31f81b, (0x53b0 bytes) 2024-09-18T00:18:59.237 INFO:teuthology.orchestra.run.smithi097.stdout:3890 read 0xbf16133 thru 0xbf1af4c (0x4e1a bytes) 2024-09-18T00:18:59.237 INFO:teuthology.orchestra.run.smithi097.stdout:3891 write 0x7a61f50 thru 0x7a69192 (0x7243 bytes) 2024-09-18T00:18:59.237 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:59.237 INFO:teuthology.orchestra.run.smithi097.stdout:3893 trunc from 0xe67d398 to 0xe0e15ca 2024-09-18T00:18:59.237 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:59.237 INFO:teuthology.orchestra.run.smithi097.stdout:3895 write 0x8b98650 thru 0x8b9aae8 (0x2499 bytes) 2024-09-18T00:18:59.237 INFO:teuthology.orchestra.run.smithi097.stdout:3897 read 0x4e77cf7 thru 0x4e7e57c (0x6886 bytes) 2024-09-18T00:18:59.237 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:59.237 INFO:teuthology.orchestra.run.smithi097.stdout:3899 read 0x26f7b2a thru 0x26fb29d (0x3774 bytes) 2024-09-18T00:18:59.238 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:59.230+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:59.238 INFO:teuthology.orchestra.run.smithi097.stdout:3901 trunc from 0xe0e15ca to 0xeb6f49f 2024-09-18T00:18:59.247 INFO:teuthology.orchestra.run.smithi097.stdout:3902 read 0xc9c8cac thru 0xc9d063e (0x7993 bytes) 2024-09-18T00:18:59.248 INFO:teuthology.orchestra.run.smithi097.stdout:3903 read 0x7f419f7 thru 0x7f4a3fe (0x8a08 bytes) 2024-09-18T00:18:59.249 INFO:teuthology.orchestra.run.smithi097.stdout:3904 write 0xba610f7 thru 0xba64cdd (0x3be7 bytes) 2024-09-18T00:18:59.253 INFO:teuthology.orchestra.run.smithi097.stdout:3911 write 0x847515b thru 0x847e756 (0x95fc bytes) 2024-09-18T00:18:59.261 INFO:teuthology.orchestra.run.smithi097.stdout:3912 trunc from 0xeb6f49f to 0xe62494e 2024-09-18T00:18:59.279 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:59.279 INFO:teuthology.orchestra.run.smithi097.stdout:3914 write 0xd74894b thru 0xd7526ea (0x9da0 bytes) 2024-09-18T00:18:59.286 INFO:teuthology.orchestra.run.smithi097.stdout:3915 writesame 0xb921859 thru 0xb92f526 data_size 0x3(0xdcce bytes) 2024-09-18T00:18:59.300 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:18:59.345 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:59.342+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:59.363 INFO:teuthology.orchestra.run.smithi097.stdout:3917 punch from 0xc70eac8 to 0xc716840, (0x7d78 bytes) 2024-09-18T00:18:59.383 INFO:teuthology.orchestra.run.smithi097.stdout:3918 trunc from 0xe62494e to 0x3230c06 2024-09-18T00:18:59.393 INFO:teuthology.orchestra.run.smithi097.stdout:3920 punch from 0x26fd62f to 0x27087e6, (0xb1b7 bytes) 2024-09-18T00:18:59.395 INFO:teuthology.orchestra.run.smithi097.stdout:3921 trunc from 0x3230c06 to 0x1ceef5b 2024-09-18T00:18:59.405 INFO:teuthology.orchestra.run.smithi097.stdout:3922 write 0x94d904 thru 0x956538 (0x8c35 bytes) 2024-09-18T00:18:59.409 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:59.409 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:59.409 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:18:59.409 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:59.423 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:18:59.429 INFO:teuthology.orchestra.run.smithi097.stdout:3925 write 0x8f6f3ab thru 0x8f75255 (0x5eab bytes) 2024-09-18T00:18:59.430 INFO:teuthology.orchestra.run.smithi097.stdout:3926 write 0x2d024ca thru 0x2d0382c (0x1363 bytes) 2024-09-18T00:18:59.432 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:59.435 INFO:teuthology.orchestra.run.smithi097.stdout:3928 trunc from 0x8f75256 to 0x71c4517 2024-09-18T00:18:59.443 INFO:teuthology.orchestra.run.smithi097.stdout:3930 read 0x3de0817 thru 0x3de9c76 (0x9460 bytes) 2024-09-18T00:18:59.443 INFO:teuthology.orchestra.run.smithi097.stdout:3931 read 0x30f3d27 thru 0x30ff064 (0xb33e bytes) 2024-09-18T00:18:59.444 INFO:teuthology.orchestra.run.smithi097.stdout:3932 read 0x20b2d73 thru 0x20b4869 (0x1af7 bytes) 2024-09-18T00:18:59.456 INFO:teuthology.orchestra.run.smithi097.stdout:3934 writesame 0x9ed9bb5 thru 0x9ee8cd8 data_size 0x7a(0xf124 bytes) 2024-09-18T00:18:59.464 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:18:59.464 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:18:59.464 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:18:59.467 INFO:teuthology.orchestra.run.smithi097.stdout:3935 write 0xc3e678d thru 0xc3ef83b (0x90af bytes) 2024-09-18T00:18:59.468 INFO:teuthology.orchestra.run.smithi097.stdout:3936 read 0x944e49e thru 0x94558aa (0x740d bytes) 2024-09-18T00:18:59.470 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:18:59.470+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:18:59.481 INFO:teuthology.orchestra.run.smithi097.stdout:3937 trunc from 0xc3ef83c to 0x3777e0f 2024-09-18T00:18:59.483 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:18:59.483 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:18:59.488 INFO:teuthology.orchestra.run.smithi097.stdout:3938 write 0x1904a7c thru 0x19140ee (0xf673 bytes) 2024-09-18T00:18:59.496 INFO:teuthology.orchestra.run.smithi097.stdout:3940 trunc from 0x3777e0f to 0x61b1a2 2024-09-18T00:18:59.502 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:18:59.503 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:18:59.504 INFO:teuthology.orchestra.run.smithi097.stdout:3941 read 0x40075a thru 0x40f219 (0xeac0 bytes) 2024-09-18T00:18:59.505 INFO:teuthology.orchestra.run.smithi097.stdout:3943 read 0x5304e7 thru 0x5314eb (0x1005 bytes) 2024-09-18T00:18:59.505 INFO:teuthology.orchestra.run.smithi097.stdout:3945 punch from 0x1c3ae6 to 0x1d08a3, (0xcdbd bytes) 2024-09-18T00:18:59.505 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:59.523 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:18:59.523 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:18:59.523 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-18T00:18:59.524 INFO:teuthology.orchestra.run.smithi097.stdout:3949 write 0x86fe186 thru 0x870023c (0x20b7 bytes) 2024-09-18T00:18:59.526 INFO:teuthology.orchestra.run.smithi097.stdout:3950 punch from 0x72d3bda to 0x72dff51, (0xc377 bytes) 2024-09-18T00:18:59.526 INFO:teuthology.orchestra.run.smithi097.stdout:3951 writesame 0x3419294 thru 0x341c178 data_size 0x23(0x2ee5 bytes) 2024-09-18T00:18:59.528 INFO:teuthology.orchestra.run.smithi097.stdout:3952 read 0x393354f thru 0x3938fc8 (0x5a7a bytes) 2024-09-18T00:18:59.528 INFO:teuthology.orchestra.run.smithi097.stdout:3953 punch from 0x55ce558 to 0x55d9d38, (0xb7e0 bytes) 2024-09-18T00:18:59.528 INFO:teuthology.orchestra.run.smithi097.stdout:3954 write 0x330f6e thru 0x339fec (0x907f bytes) 2024-09-18T00:18:59.531 INFO:teuthology.orchestra.run.smithi097.stdout:3955 trunc from 0x870023d to 0x9348684 2024-09-18T00:18:59.537 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:59.534+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:59.537 INFO:teuthology.orchestra.run.smithi097.stdout:3956 read 0x32462a0 thru 0x324a591 (0x42f2 bytes) 2024-09-18T00:18:59.545 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:18:59.545 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:18:59.546 INFO:teuthology.orchestra.run.smithi097.stdout:3957 trunc from 0x9348684 to 0x35a34a1 2024-09-18T00:18:59.548 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:18:59.548 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:18:59.548 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:18:59.548 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:18:59.548 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:18:59.555 INFO:teuthology.orchestra.run.smithi097.stdout:3959 read 0x2ed0d6f thru 0x2ed78f2 (0x6b84 bytes) 2024-09-18T00:18:59.555 INFO:teuthology.orchestra.run.smithi097.stdout:3960 read 0x8658f7 thru 0x86814f (0x2859 bytes) 2024-09-18T00:18:59.562 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:18:59.563 INFO:teuthology.orchestra.run.smithi097.stdout:3961 writesame 0x42774c8 thru 0x427f12b data_size 0x26(0x7c64 bytes) 2024-09-18T00:18:59.564 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:59.564 INFO:teuthology.orchestra.run.smithi097.stdout:3963 read 0x280b730 thru 0x280ef3a (0x380b bytes) 2024-09-18T00:18:59.565 INFO:teuthology.orchestra.run.smithi097.stdout:3964 punch from 0x2468790 to 0x2471054, (0x88c4 bytes) 2024-09-18T00:18:59.566 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:18:59.566 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:18:59.572 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:18:59.572 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:18:59.583 INFO:teuthology.orchestra.run.smithi097.stdout:3965 writesame 0xc4d943f thru 0xc4e7cf7 data_size 0x7(0xe8b9 bytes) 2024-09-18T00:18:59.584 INFO:teuthology.orchestra.run.smithi097.stdout:3967 write 0x4b9468f thru 0x4b96e67 (0x27d9 bytes) 2024-09-18T00:18:59.585 INFO:teuthology.orchestra.run.smithi097.stdout:3968 punch from 0x9fba669 to 0x9fbf51b, (0x4eb2 bytes) 2024-09-18T00:18:59.586 INFO:teuthology.orchestra.run.smithi097.stdout:3969 write 0x3cb48a thru 0x3d7847 (0xc3be bytes) 2024-09-18T00:18:59.588 INFO:teuthology.orchestra.run.smithi097.stdout:3970 write 0xa91f49d thru 0xa925bfe (0x6762 bytes) 2024-09-18T00:18:59.590 INFO:teuthology.orchestra.run.smithi097.stdout:3971 punch from 0xa112898 to 0xa117f4c, (0x56b4 bytes) 2024-09-18T00:18:59.603 INFO:teuthology.orchestra.run.smithi097.stdout:3973 trunc from 0xc4e7cf8 to 0x39ed1e1 2024-09-18T00:18:59.631 INFO:teuthology.orchestra.run.smithi097.stdout:3974 write 0xd15ca76 thru 0xd15eafe (0x2089 bytes) 2024-09-18T00:18:59.647 INFO:teuthology.orchestra.run.smithi097.stdout:3976 trunc from 0xd15eaff to 0x2d10976 2024-09-18T00:18:59.650 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:18:59.650+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:18:59.674 INFO:teuthology.orchestra.run.smithi097.stdout:3977 write 0x5a26e09 thru 0x5a2b589 (0x4781 bytes) 2024-09-18T00:18:59.676 INFO:teuthology.orchestra.run.smithi097.stdout:3978 write 0x190ab09 thru 0x191723a (0xc732 bytes) 2024-09-18T00:18:59.686 INFO:teuthology.orchestra.run.smithi097.stdout:3980 trunc from 0x5a2b58a to 0x1748337 2024-09-18T00:18:59.697 INFO:teuthology.orchestra.run.smithi097.stdout:3981 read 0xb3502b thru 0xb364ff (0x14d5 bytes) 2024-09-18T00:18:59.711 INFO:teuthology.orchestra.run.smithi097.stdout:3982 writesame 0x4b9d983 thru 0x4ba0b7b data_size 0x1(0x31f9 bytes) 2024-09-18T00:18:59.713 INFO:teuthology.orchestra.run.smithi097.stdout:3983 read 0x18f5772 thru 0x1903f6e (0xe7fd bytes) 2024-09-18T00:18:59.714 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:59.715 INFO:teuthology.orchestra.run.smithi097.stdout:3985 write 0x379e226 thru 0x37a0b6c (0x2947 bytes) 2024-09-18T00:18:59.729 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:59.729 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:59.738 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:59.734+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:18:59.743 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:18:59.744 INFO:teuthology.orchestra.run.smithi097.stdout:3987 writesame 0xe9a4521 thru 0xe9a6528 data_size 0xa(0x2008 bytes) 2024-09-18T00:18:59.746 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:18:59.746 INFO:teuthology.orchestra.run.smithi097.stdout:3989 read 0x2551a86 thru 0x2560c4c (0xf1c7 bytes) 2024-09-18T00:18:59.747 INFO:teuthology.orchestra.run.smithi097.stdout:3991 clone 16 order 19 su 262144 sc 14 2024-09-18T00:18:59.839 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:18:59.834+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:18:59.874 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:18:59.874 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:18:59.889 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:18:59.985 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:18:59.982+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:00.040 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:00.034+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:00.076 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:00.076 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:00.093 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:00.141 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:00.138+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:00.175 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-18T00:19:00.183 INFO:teuthology.orchestra.run.smithi097.stdout:3993 writesame 0x6cb7c3d thru 0x6cc43c1 data_size 0x1(0xc785 bytes) 2024-09-18T00:19:00.242 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:00.238+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:00.270 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:00.270 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:00.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:19:00.286 INFO:teuthology.orchestra.run.smithi097.stdout:3994 read 0x237a5a8 thru 0x237f892 (0x52eb bytes) 2024-09-18T00:19:00.286 INFO:teuthology.orchestra.run.smithi097.stdout:3996 read 0xa55c4e0 thru 0xa56bbe0 (0xf701 bytes) 2024-09-18T00:19:00.287 INFO:teuthology.orchestra.run.smithi097.stdout:3997 punch from 0x812fd59 to 0x81373a9, (0x7650 bytes) 2024-09-18T00:19:00.287 INFO:teuthology.orchestra.run.smithi097.stdout:3998 read 0x4219eb0 thru 0x4227f44 (0xe095 bytes) 2024-09-18T00:19:00.288 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:00.288 INFO:teuthology.orchestra.run.smithi097.stdout:4000 writesame 0x11ab6b thru 0x11f33f data_size 0x25(0x47d5 bytes) 2024-09-18T00:19:00.305 INFO:teuthology.orchestra.run.smithi097.stdout:4001 write 0x800d2e4 thru 0x8010b58 (0x3875 bytes) 2024-09-18T00:19:00.312 INFO:teuthology.orchestra.run.smithi097.stdout:4003 read 0x9127360 thru 0x91299dc (0x267d bytes) 2024-09-18T00:19:00.312 INFO:teuthology.orchestra.run.smithi097.stdout:4005 writesame 0xe3fb98f thru 0xe3fe72e data_size 0xa(0x2da0 bytes) 2024-09-18T00:19:00.320 INFO:teuthology.orchestra.run.smithi097.stdout:4007 read 0xa576ff7 thru 0xa57e5d5 (0x75df bytes) 2024-09-18T00:19:00.329 INFO:teuthology.orchestra.run.smithi097.stdout:4008 trunc from 0xe9a6529 to 0x9ed9a8c 2024-09-18T00:19:00.343 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:00.338+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:00.376 INFO:teuthology.orchestra.run.smithi097.stdout:4010 write 0xad73d56 thru 0xad7e0ba (0xa365 bytes) 2024-09-18T00:19:00.397 INFO:teuthology.orchestra.run.smithi097.stdout:4011 trunc from 0xad7e0bb to 0xd80195 2024-09-18T00:19:00.441 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:00.441 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:00.444 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:00.442+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:00.459 INFO:teuthology.orchestra.run.smithi097.stdout:4012 write 0xc590bfa thru 0xc599f8a (0x9391 bytes) 2024-09-18T00:19:00.460 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:19:00.466 INFO:teuthology.orchestra.run.smithi097.stdout:4013 read 0x5ecda79 thru 0x5ed000f (0x2597 bytes) 2024-09-18T00:19:00.466 INFO:teuthology.orchestra.run.smithi097.stdout:4015 read 0x692a2a4 thru 0x692b5ae (0x130b bytes) 2024-09-18T00:19:00.467 INFO:teuthology.orchestra.run.smithi097.stdout:4016 punch from 0x750794 to 0x75f6bf, (0xef2b bytes) 2024-09-18T00:19:00.467 INFO:teuthology.orchestra.run.smithi097.stdout:4017 write 0x6fc9e2e thru 0x6fd43d9 (0xa5ac bytes) 2024-09-18T00:19:00.476 INFO:teuthology.orchestra.run.smithi097.stdout:4018 read 0x4ed53cf thru 0x4ed72ef (0x1f21 bytes) 2024-09-18T00:19:00.476 INFO:teuthology.orchestra.run.smithi097.stdout:4020 read 0x9aaf47f thru 0x9abdd25 (0xe8a7 bytes) 2024-09-18T00:19:00.476 INFO:teuthology.orchestra.run.smithi097.stdout:4021 read 0x8f23111 thru 0x8f2f0ec (0xbfdc bytes) 2024-09-18T00:19:00.476 INFO:teuthology.orchestra.run.smithi097.stdout:4022 write 0x978ee80 thru 0x979abca (0xbd4b bytes) 2024-09-18T00:19:00.486 INFO:teuthology.orchestra.run.smithi097.stdout:4023 write 0x2c5af6f thru 0x2c6949f (0xe531 bytes) 2024-09-18T00:19:00.502 INFO:teuthology.orchestra.run.smithi097.stdout:4029 read 0x23ecb42 thru 0x23fb2c2 (0xe781 bytes) 2024-09-18T00:19:00.503 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1, 5] live_osds: [0, 2, 3, 4, 6, 7] 2024-09-18T00:19:00.503 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:19:00.504 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T00:19:00.504 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 6 2024-09-18T00:19:00.517 INFO:teuthology.orchestra.run.smithi097.stdout:4031 writesame 0xeb3e06f thru 0xeb4765e data_size 0x8(0x95f0 bytes) 2024-09-18T00:19:00.527 INFO:teuthology.orchestra.run.smithi097.stdout:4033 trunc from 0xeb4765f to 0x840a522 2024-09-18T00:19:00.542 INFO:teuthology.orchestra.run.smithi097.stdout:4034 read 0x4d90da thru 0x4dac77 (0x1b9e bytes) 2024-09-18T00:19:00.544 INFO:teuthology.orchestra.run.smithi097.stdout:4035 punch from 0x4069bbe to 0x4073091, (0x94d3 bytes) 2024-09-18T00:19:00.544 INFO:teuthology.orchestra.run.smithi097.stdout:4036 read 0x107a50c thru 0x10802f1 (0x5de6 bytes) 2024-09-18T00:19:00.544 INFO:teuthology.orchestra.run.smithi097.stdout:4037 writesame 0x7c44eeb thru 0x7c49264 data_size 0x3(0x437a bytes) 2024-09-18T00:19:00.545 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:00.542+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:00.560 INFO:teuthology.orchestra.run.smithi097.stdout:4038 trunc from 0x840a522 to 0xd400868 2024-09-18T00:19:00.572 INFO:teuthology.orchestra.run.smithi097.stdout:4040 flatten 2024-09-18T00:19:00.606 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:00.606 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:00.614 INFO:teuthology.orchestra.run.smithi097.stdout:4041 punch from 0x93ef38 to 0x94aaf3, (0xbbbb bytes) 2024-09-18T00:19:00.614 INFO:teuthology.orchestra.run.smithi097.stdout:4043 read 0xc80cd53 thru 0xc812040 (0x52ee bytes) 2024-09-18T00:19:00.614 INFO:teuthology.orchestra.run.smithi097.stdout:4046 write 0x240a4fc thru 0x2413d94 (0x9899 bytes) 2024-09-18T00:19:00.623 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:19:00.623 INFO:teuthology.orchestra.run.smithi097.stdout:4047 punch from 0x9fe581d to 0x9fea1f5, (0x49d8 bytes) 2024-09-18T00:19:00.630 INFO:teuthology.orchestra.run.smithi097.stdout:4048 trunc from 0xd400868 to 0x8ba9c5b 2024-09-18T00:19:00.643 INFO:teuthology.orchestra.run.smithi097.stdout:4049 punch from 0x4b17219 to 0x4b1dd07, (0x6aee bytes) 2024-09-18T00:19:00.643 INFO:teuthology.orchestra.run.smithi097.stdout:4051 write 0x6fabb77 thru 0x6fb3146 (0x75d0 bytes) 2024-09-18T00:19:00.645 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:00.642+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:00.650 INFO:teuthology.orchestra.run.smithi097.stdout:4052 read 0x50b15bb thru 0x50c0759 (0xf19f bytes) 2024-09-18T00:19:00.650 INFO:teuthology.orchestra.run.smithi097.stdout:4054 writesame 0x66b2d1f thru 0x66bf1e9 data_size 0x7(0xc4cb bytes) 2024-09-18T00:19:00.672 INFO:teuthology.orchestra.run.smithi097.stdout:4055 trunc from 0x8ba9c5b to 0x8895aaa 2024-09-18T00:19:00.683 INFO:teuthology.orchestra.run.smithi097.stdout:4057 read 0x2878e24 thru 0x287b861 (0x2a3e bytes) 2024-09-18T00:19:00.683 INFO:teuthology.orchestra.run.smithi097.stdout:4059 writesame 0x7042577 thru 0x70514d2 data_size 0x2(0xef5c bytes) 2024-09-18T00:19:00.743 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:00.743 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:00.746 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:00.742+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:00.757 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:19:00.798 INFO:teuthology.orchestra.run.smithi097.stdout:4060 read 0x5f2db0e thru 0x5f374cc (0x99bf bytes) 2024-09-18T00:19:00.798 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:00.798 INFO:teuthology.orchestra.run.smithi097.stdout:4062 punch from 0x1274ebe to 0x1277949, (0x2a8b bytes) 2024-09-18T00:19:00.812 INFO:teuthology.orchestra.run.smithi097.stdout:4063 writesame 0xae2b3f4 thru 0xae3a200 data_size 0x1(0xee0d bytes) 2024-09-18T00:19:00.812 INFO:teuthology.orchestra.run.smithi097.stdout:4065 read 0x4defc5a thru 0x4df72d5 (0x767c bytes) 2024-09-18T00:19:00.812 INFO:teuthology.orchestra.run.smithi097.stdout:4066 write 0x526f071 thru 0x5277d22 (0x8cb2 bytes) 2024-09-18T00:19:00.816 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:19:00.817 INFO:teuthology.orchestra.run.smithi097.stdout:4067 read 0xa2f53d9 thru 0xa2fd613 (0x823b bytes) 2024-09-18T00:19:00.817 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:00.834 INFO:teuthology.orchestra.run.smithi097.stdout:4069 writesame 0xe6ea99a thru 0xe6f9767 data_size 0x2(0xedce bytes) 2024-09-18T00:19:00.834 INFO:teuthology.orchestra.run.smithi097.stdout:4070 writesame 0x945b404 thru 0x945dc7c data_size 0x1(0x2879 bytes) 2024-09-18T00:19:00.839 INFO:teuthology.orchestra.run.smithi097.stdout:4072 write 0xc8ddead thru 0xc8e56c1 (0x7815 bytes) 2024-09-18T00:19:00.847 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:00.842+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:00.853 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-18T00:19:00.850+0000 7f63a6107640 -1 mon.a@0(leader).osd e66 definitely_dead 0 2024-09-18T00:19:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:4073 trunc from 0xe6f9768 to 0x37168e5 2024-09-18T00:19:00.902 INFO:teuthology.orchestra.run.smithi097.stdout:4074 write 0x90bdad0 thru 0x90be4e0 (0xa11 bytes) 2024-09-18T00:19:00.905 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:00.905 INFO:teuthology.orchestra.run.smithi097.stdout:4076 read 0x34d2f9d thru 0x34dff7c (0xcfe0 bytes) 2024-09-18T00:19:00.905 INFO:teuthology.orchestra.run.smithi097.stdout:4077 write 0x7cf3663 thru 0x7cfc605 (0x8fa3 bytes) 2024-09-18T00:19:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:4078 write 0x51eea08 thru 0x51f171b (0x2d14 bytes) 2024-09-18T00:19:00.937 INFO:teuthology.orchestra.run.smithi097.stdout:4079 write 0xd2571c4 thru 0xd260b1a (0x9957 bytes) 2024-09-18T00:19:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:4081 write 0x1a7ed81 thru 0x1a893b9 (0xa639 bytes) 2024-09-18T00:19:00.948 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:00.942+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:4083 punch from 0x31119a3 to 0x3119c17, (0x8274 bytes) 2024-09-18T00:19:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:4084 write 0x23669e9 thru 0x2375cb8 (0xf2d0 bytes) 2024-09-18T00:19:00.957 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:19:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:4087 read 0x24cb433 thru 0x24da8c2 (0xf490 bytes) 2024-09-18T00:19:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:4089 write 0x301b3df thru 0x301c1f0 (0xe12 bytes) 2024-09-18T00:19:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:4090 punch from 0x61589c8 to 0x6165bc1, (0xd1f9 bytes) 2024-09-18T00:19:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:4094 write 0x3dda5d7 thru 0x3ddd70e (0x3138 bytes) 2024-09-18T00:19:00.968 INFO:teuthology.orchestra.run.smithi097.stdout:4096 read 0x8950ca1 thru 0x89538c1 (0x2c21 bytes) 2024-09-18T00:19:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:4099 punch from 0x6769eb5 to 0x6775c40, (0xbd8b bytes) 2024-09-18T00:19:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:4100 punch from 0x7c80fe5 to 0x7c8e513, (0xd52e bytes) 2024-09-18T00:19:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:4102 clone 17 order 21 su 1048576 sc 8 2024-09-18T00:19:01.049 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:01.046+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:01.072 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:01.072 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:01.088 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:19:01.150 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:01.150+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:01.252 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:01.246+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:01.254 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:01.254 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:01.268 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:19:01.353 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:01.350+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:01.402 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:01.402 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:01.416 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:19:01.458 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:01.458+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:01.554 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:01.550+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:01.573 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:19:01.577 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:01.578 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:01.592 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:19:01.682 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:01.682+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:01.725 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:01.726 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:01.726 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:01.743 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:01.744 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:01.756 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:01.750+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:01.759 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:01.760 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:01.761 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:01.761 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:01.777 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:19:01.779 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:01.779 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:01.782 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-18T00:19:01.796 INFO:teuthology.orchestra.run.smithi097.stderr:marked out osd.6. 2024-09-18T00:19:01.797 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is unset 2024-09-18T00:19:01.807 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:01.807 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:01.829 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:01.829 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:01.832 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:01.833 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:01.862 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:01.862+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:01.942 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:01.942 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:01.942 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:01.942 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:01.956 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:19:01.999 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:01.998+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:02.059 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:02.054+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:02.118 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:02.119 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:02.119 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:02.119 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:02.119 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:02.132 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:19:02.178 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:02.178+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:02.253 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:02.253 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:02.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:02.253 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:02.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.253 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2229 3.10 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:02.253 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:02.254 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 41.986013294000003, 2024-09-18T00:19:02.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048730086999999998, 2024-09-18T00:19:02.254 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.254 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.254 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:02.254 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.254 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.254 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.254 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2229 2024-09-18T00:19:02.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.254 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:02.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:02.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263362+0000", 2024-09-18T00:19:02.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3869999999999999e-06 2024-09-18T00:19:02.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263381+0000", 2024-09-18T00:19:02.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9153000000000001e-05 2024-09-18T00:19:02.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263418+0000", 2024-09-18T00:19:02.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6346000000000001e-05 2024-09-18T00:19:02.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263435+0000", 2024-09-18T00:19:02.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7309999999999999e-05 2024-09-18T00:19:02.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263589+0000", 2024-09-18T00:19:02.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015451199999999999 2024-09-18T00:19:02.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263902+0000", 2024-09-18T00:19:02.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031291799999999997 2024-09-18T00:19:02.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:02.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.265995+0000", 2024-09-18T00:19:02.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020931809999999999 2024-09-18T00:19:02.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:02.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311788+0000", 2024-09-18T00:19:02.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045792078 2024-09-18T00:19:02.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:02.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311903+0000", 2024-09-18T00:19:02.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011510699999999999 2024-09-18T00:19:02.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:02.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312009+0000", 2024-09-18T00:19:02.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010630599999999999 2024-09-18T00:19:02.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312089+0000", 2024-09-18T00:19:02.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9789000000000005e-05 2024-09-18T00:19:02.259 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.259 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.259 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.259 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2229 3.10 e36/23 ondisk, result = 0)", 2024-09-18T00:19:02.259 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:02.260 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 41.966867635, 2024-09-18T00:19:02.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029600955000000002, 2024-09-18T00:19:02.260 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.260 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.260 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.260 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:02.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:02.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282507+0000", 2024-09-18T00:19:02.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9160000000000001e-06 2024-09-18T00:19:02.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282522+0000", 2024-09-18T00:19:02.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5e-05 2024-09-18T00:19:02.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282558+0000", 2024-09-18T00:19:02.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5694000000000001e-05 2024-09-18T00:19:02.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282580+0000", 2024-09-18T00:19:02.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2323e-05 2024-09-18T00:19:02.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311873+0000", 2024-09-18T00:19:02.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029293013999999999 2024-09-18T00:19:02.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311892+0000", 2024-09-18T00:19:02.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8556e-05 2024-09-18T00:19:02.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312105+0000", 2024-09-18T00:19:02.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021345199999999999 2024-09-18T00:19:02.263 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.263 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.263 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.263 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5714 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:02.264 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:02.264 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.939165131999999, 2024-09-18T00:19:02.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027448727999999999, 2024-09-18T00:19:02.264 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.264 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.264 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:02.264 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.264 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.264 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.264 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5714 2024-09-18T00:19:02.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.264 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:02.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:02.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310211+0000", 2024-09-18T00:19:02.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5639999999999998e-06 2024-09-18T00:19:02.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312678+0000", 2024-09-18T00:19:02.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002466761 2024-09-18T00:19:02.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312746+0000", 2024-09-18T00:19:02.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8046000000000005e-05 2024-09-18T00:19:02.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312774+0000", 2024-09-18T00:19:02.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7813e-05 2024-09-18T00:19:02.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312927+0000", 2024-09-18T00:19:02.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015328499999999999 2024-09-18T00:19:02.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313214+0000", 2024-09-18T00:19:02.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028683099999999998 2024-09-18T00:19:02.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:02.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313708+0000", 2024-09-18T00:19:02.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049379100000000004 2024-09-18T00:19:02.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:02.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336895+0000", 2024-09-18T00:19:02.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023186801 2024-09-18T00:19:02.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:02.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337454+0000", 2024-09-18T00:19:02.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055979399999999996 2024-09-18T00:19:02.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:02.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337571+0000", 2024-09-18T00:19:02.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116821 2024-09-18T00:19:02.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337656+0000", 2024-09-18T00:19:02.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4221000000000006e-05 2024-09-18T00:19:02.269 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.269 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.269 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.269 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6801 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:02.269 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:02.269 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.212328683999999, 2024-09-18T00:19:02.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051953602000000002, 2024-09-18T00:19:02.270 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.270 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.270 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:02.270 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.270 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.270 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.270 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6801 2024-09-18T00:19:02.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.270 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:02.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:02.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037046+0000", 2024-09-18T00:19:02.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.19e-06 2024-09-18T00:19:02.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037062+0000", 2024-09-18T00:19:02.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586999999999998e-05 2024-09-18T00:19:02.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037096+0000", 2024-09-18T00:19:02.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4112000000000001e-05 2024-09-18T00:19:02.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037120+0000", 2024-09-18T00:19:02.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.355e-05 2024-09-18T00:19:02.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037246+0000", 2024-09-18T00:19:02.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012652200000000001 2024-09-18T00:19:02.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037385+0000", 2024-09-18T00:19:02.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013924599999999999 2024-09-18T00:19:02.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:02.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.040881+0000", 2024-09-18T00:19:02.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034954830000000002 2024-09-18T00:19:02.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:02.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088794+0000", 2024-09-18T00:19:02.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047912897000000003 2024-09-18T00:19:02.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:02.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088828+0000", 2024-09-18T00:19:02.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3880000000000001e-05 2024-09-18T00:19:02.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:02.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088916+0000", 2024-09-18T00:19:02.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.865e-05 2024-09-18T00:19:02.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088997+0000", 2024-09-18T00:19:02.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0484999999999999e-05 2024-09-18T00:19:02.274 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.274 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.275 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.275 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:6801 3.10 e42/23 ondisk, result = 0)", 2024-09-18T00:19:02.275 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:02.275 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.198521274000001, 2024-09-18T00:19:02.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037960394000000001, 2024-09-18T00:19:02.275 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.275 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.276 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.276 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:02.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:02.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050852+0000", 2024-09-18T00:19:02.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:19:02.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050860+0000", 2024-09-18T00:19:02.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7729999999999999e-06 2024-09-18T00:19:02.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050891+0000", 2024-09-18T00:19:02.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0617000000000002e-05 2024-09-18T00:19:02.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050910+0000", 2024-09-18T00:19:02.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.944e-05 2024-09-18T00:19:02.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088766+0000", 2024-09-18T00:19:02.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037855633999999999 2024-09-18T00:19:02.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088784+0000", 2024-09-18T00:19:02.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7767000000000001e-05 2024-09-18T00:19:02.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088811+0000", 2024-09-18T00:19:02.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7382999999999999e-05 2024-09-18T00:19:02.279 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.279 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.279 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.280 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12594 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:02.280 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:02.280 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 25.857744736000001, 2024-09-18T00:19:02.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026492600000000002, 2024-09-18T00:19:02.280 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.280 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.280 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.280 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.280 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.280 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.281 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12594 2024-09-18T00:19:02.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.281 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:02.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:02.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391629+0000", 2024-09-18T00:19:02.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.86e-06 2024-09-18T00:19:02.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391637+0000", 2024-09-18T00:19:02.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3999999999999992e-06 2024-09-18T00:19:02.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391664+0000", 2024-09-18T00:19:02.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6536000000000001e-05 2024-09-18T00:19:02.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391681+0000", 2024-09-18T00:19:02.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6563e-05 2024-09-18T00:19:02.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417791+0000", 2024-09-18T00:19:02.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026110936000000001 2024-09-18T00:19:02.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417933+0000", 2024-09-18T00:19:02.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014112099999999999 2024-09-18T00:19:02.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418120+0000", 2024-09-18T00:19:02.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018718399999999999 2024-09-18T00:19:02.284 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.284 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.284 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.284 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12595 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:02.285 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:02.285 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 25.857668537999999, 2024-09-18T00:19:02.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026819428999999999, 2024-09-18T00:19:02.285 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.285 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.285 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.285 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.285 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.285 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.285 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12595 2024-09-18T00:19:02.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.285 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391705+0000", 2024-09-18T00:19:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3999999999999999e-06 2024-09-18T00:19:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391711+0000", 2024-09-18T00:19:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7459999999999997e-06 2024-09-18T00:19:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391733+0000", 2024-09-18T00:19:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2923000000000001e-05 2024-09-18T00:19:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391748+0000", 2024-09-18T00:19:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4790000000000001e-05 2024-09-18T00:19:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418180+0000", 2024-09-18T00:19:02.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026431888000000001 2024-09-18T00:19:02.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418328+0000", 2024-09-18T00:19:02.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000147725 2024-09-18T00:19:02.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418523+0000", 2024-09-18T00:19:02.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000194957 2024-09-18T00:19:02.288 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.288 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.289 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.289 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12596 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:02.289 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:02.289 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 25.85760557, 2024-09-18T00:19:02.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027220296000000001, 2024-09-18T00:19:02.289 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.289 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.289 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.289 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.289 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.289 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.289 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12596 2024-09-18T00:19:02.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.290 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:02.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:02.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391768+0000", 2024-09-18T00:19:02.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.283e-06 2024-09-18T00:19:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391773+0000", 2024-09-18T00:19:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3970000000000002e-06 2024-09-18T00:19:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391795+0000", 2024-09-18T00:19:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1835e-05 2024-09-18T00:19:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391809+0000", 2024-09-18T00:19:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4107000000000001e-05 2024-09-18T00:19:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418582+0000", 2024-09-18T00:19:02.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026772918999999999 2024-09-18T00:19:02.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418780+0000", 2024-09-18T00:19:02.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000198042 2024-09-18T00:19:02.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418987+0000", 2024-09-18T00:19:02.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000206713 2024-09-18T00:19:02.293 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.293 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.293 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.293 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12597 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:02.293 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:02.293 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 25.857545051999999, 2024-09-18T00:19:02.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027612714, 2024-09-18T00:19:02.293 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.293 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.293 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.293 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.294 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.294 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.294 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12597 2024-09-18T00:19:02.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.294 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:02.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:02.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391828+0000", 2024-09-18T00:19:02.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.203e-06 2024-09-18T00:19:02.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391834+0000", 2024-09-18T00:19:02.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6529999999999999e-06 2024-09-18T00:19:02.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391856+0000", 2024-09-18T00:19:02.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2195999999999999e-05 2024-09-18T00:19:02.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391870+0000", 2024-09-18T00:19:02.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4063000000000001e-05 2024-09-18T00:19:02.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419049+0000", 2024-09-18T00:19:02.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027178751000000001 2024-09-18T00:19:02.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419239+0000", 2024-09-18T00:19:02.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019039400000000001 2024-09-18T00:19:02.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419440+0000", 2024-09-18T00:19:02.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000200454 2024-09-18T00:19:02.297 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.297 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.297 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.298 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12598 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:02.298 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:02.298 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 25.857482001000001, 2024-09-18T00:19:02.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028269056000000001, 2024-09-18T00:19:02.298 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.298 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.298 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.298 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.298 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.298 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.298 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12598 2024-09-18T00:19:02.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.298 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:02.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:02.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391891+0000", 2024-09-18T00:19:02.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.257e-06 2024-09-18T00:19:02.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391897+0000", 2024-09-18T00:19:02.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9560000000000002e-06 2024-09-18T00:19:02.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391920+0000", 2024-09-18T00:19:02.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3025999999999999e-05 2024-09-18T00:19:02.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391935+0000", 2024-09-18T00:19:02.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4853e-05 2024-09-18T00:19:02.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419527+0000", 2024-09-18T00:19:02.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027592378000000001 2024-09-18T00:19:02.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419681+0000", 2024-09-18T00:19:02.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015316 2024-09-18T00:19:02.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420159+0000", 2024-09-18T00:19:02.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000478426 2024-09-18T00:19:02.301 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.302 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.302 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.302 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12599 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:02.302 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:02.302 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 25.857418146000001, 2024-09-18T00:19:02.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028649857000000001, 2024-09-18T00:19:02.302 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.302 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.302 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.302 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.302 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.302 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.302 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12599 2024-09-18T00:19:02.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.303 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:02.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:02.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391955+0000", 2024-09-18T00:19:02.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-06 2024-09-18T00:19:02.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391961+0000", 2024-09-18T00:19:02.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7100000000000004e-06 2024-09-18T00:19:02.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391982+0000", 2024-09-18T00:19:02.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1492e-05 2024-09-18T00:19:02.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391996+0000", 2024-09-18T00:19:02.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4194e-05 2024-09-18T00:19:02.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420237+0000", 2024-09-18T00:19:02.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028240818000000001 2024-09-18T00:19:02.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420430+0000", 2024-09-18T00:19:02.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019228299999999999 2024-09-18T00:19:02.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420604+0000", 2024-09-18T00:19:02.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000174117 2024-09-18T00:19:02.306 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.306 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.306 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.306 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12600 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 270336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:02.306 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:02.306 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 25.857357151999999, 2024-09-18T00:19:02.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028964586, 2024-09-18T00:19:02.307 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.307 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.307 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.307 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.307 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.307 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.307 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12600 2024-09-18T00:19:02.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.307 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:02.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:02.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392016+0000", 2024-09-18T00:19:02.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.31e-06 2024-09-18T00:19:02.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392022+0000", 2024-09-18T00:19:02.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5600000000000001e-06 2024-09-18T00:19:02.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392045+0000", 2024-09-18T00:19:02.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3595999999999999e-05 2024-09-18T00:19:02.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392060+0000", 2024-09-18T00:19:02.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4727e-05 2024-09-18T00:19:02.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420669+0000", 2024-09-18T00:19:02.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028608855999999998 2024-09-18T00:19:02.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420813+0000", 2024-09-18T00:19:02.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014429500000000001 2024-09-18T00:19:02.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420979+0000", 2024-09-18T00:19:02.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016624200000000001 2024-09-18T00:19:02.311 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.311 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.311 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.311 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12601 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:02.311 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:02.311 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 25.857293247000001, 2024-09-18T00:19:02.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029292832000000001, 2024-09-18T00:19:02.311 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.311 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.311 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.311 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.312 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.312 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.312 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12601 2024-09-18T00:19:02.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.312 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:02.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:02.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392080+0000", 2024-09-18T00:19:02.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:02.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392086+0000", 2024-09-18T00:19:02.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7570000000000001e-06 2024-09-18T00:19:02.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392105+0000", 2024-09-18T00:19:02.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9415999999999998e-05 2024-09-18T00:19:02.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392132+0000", 2024-09-18T00:19:02.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7019e-05 2024-09-18T00:19:02.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421049+0000", 2024-09-18T00:19:02.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028916659000000001 2024-09-18T00:19:02.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421204+0000", 2024-09-18T00:19:02.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155224 2024-09-18T00:19:02.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421372+0000", 2024-09-18T00:19:02.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016754699999999999 2024-09-18T00:19:02.315 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.315 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.315 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.315 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12602 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:02.315 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:02.316 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 25.857220615999999, 2024-09-18T00:19:02.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029612833000000002, 2024-09-18T00:19:02.316 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.316 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.316 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.316 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.316 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.316 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.316 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12602 2024-09-18T00:19:02.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.316 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:02.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:02.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392152+0000", 2024-09-18T00:19:02.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0470000000000001e-06 2024-09-18T00:19:02.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392158+0000", 2024-09-18T00:19:02.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:19:02.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392181+0000", 2024-09-18T00:19:02.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2629e-05 2024-09-18T00:19:02.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392222+0000", 2024-09-18T00:19:02.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0772000000000002e-05 2024-09-18T00:19:02.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421441+0000", 2024-09-18T00:19:02.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029219327 2024-09-18T00:19:02.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421608+0000", 2024-09-18T00:19:02.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000166813 2024-09-18T00:19:02.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421764+0000", 2024-09-18T00:19:02.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015628500000000001 2024-09-18T00:19:02.320 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.320 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.320 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.321 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12603 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:02.321 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:02.321 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 25.857130221999999, 2024-09-18T00:19:02.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029902524, 2024-09-18T00:19:02.321 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.321 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.321 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.321 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.321 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.321 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.321 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12603 2024-09-18T00:19:02.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.322 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:02.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:02.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392243+0000", 2024-09-18T00:19:02.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:02.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392260+0000", 2024-09-18T00:19:02.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6860000000000001e-05 2024-09-18T00:19:02.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392282+0000", 2024-09-18T00:19:02.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2713000000000001e-05 2024-09-18T00:19:02.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392297+0000", 2024-09-18T00:19:02.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4375e-05 2024-09-18T00:19:02.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421830+0000", 2024-09-18T00:19:02.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029532668000000002 2024-09-18T00:19:02.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421985+0000", 2024-09-18T00:19:02.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155613 2024-09-18T00:19:02.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422144+0000", 2024-09-18T00:19:02.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000159102 2024-09-18T00:19:02.325 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.325 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.325 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.325 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12604 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:02.325 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:02.325 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 25.857055170999999, 2024-09-18T00:19:02.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030228695, 2024-09-18T00:19:02.325 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.325 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.325 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.326 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.326 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.326 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.326 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12604 2024-09-18T00:19:02.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.326 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:02.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:02.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392318+0000", 2024-09-18T00:19:02.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:02.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392337+0000", 2024-09-18T00:19:02.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:19:02.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392364+0000", 2024-09-18T00:19:02.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7288999999999999e-05 2024-09-18T00:19:02.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392379+0000", 2024-09-18T00:19:02.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:19:02.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422209+0000", 2024-09-18T00:19:02.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029829050999999999 2024-09-18T00:19:02.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422388+0000", 2024-09-18T00:19:02.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017988699999999999 2024-09-18T00:19:02.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422545+0000", 2024-09-18T00:19:02.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015701199999999999 2024-09-18T00:19:02.330 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.330 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.330 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.330 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12605 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:02.330 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:02.330 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 25.856970610000001, 2024-09-18T00:19:02.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030488674, 2024-09-18T00:19:02.330 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.330 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.330 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.331 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.331 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.331 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.331 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12605 2024-09-18T00:19:02.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.331 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:02.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:02.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392403+0000", 2024-09-18T00:19:02.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.883e-06 2024-09-18T00:19:02.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392421+0000", 2024-09-18T00:19:02.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7526000000000001e-05 2024-09-18T00:19:02.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392450+0000", 2024-09-18T00:19:02.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8972e-05 2024-09-18T00:19:02.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392488+0000", 2024-09-18T00:19:02.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8522000000000001e-05 2024-09-18T00:19:02.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422614+0000", 2024-09-18T00:19:02.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030125945000000001 2024-09-18T00:19:02.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422752+0000", 2024-09-18T00:19:02.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000138178 2024-09-18T00:19:02.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422890+0000", 2024-09-18T00:19:02.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013764800000000001 2024-09-18T00:19:02.334 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.334 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.334 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.334 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12606 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:02.334 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:02.334 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 25.856861949999999, 2024-09-18T00:19:02.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033727073000000003, 2024-09-18T00:19:02.335 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.335 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.335 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.335 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.335 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.335 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.335 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12606 2024-09-18T00:19:02.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.335 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:02.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:02.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392511+0000", 2024-09-18T00:19:02.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3090000000000001e-06 2024-09-18T00:19:02.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392517+0000", 2024-09-18T00:19:02.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.993e-06 2024-09-18T00:19:02.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.337 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.337 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392543+0000", 2024-09-18T00:19:02.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6152999999999998e-05 2024-09-18T00:19:02.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.337 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.337 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392558+0000", 2024-09-18T00:19:02.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4610000000000001e-05 2024-09-18T00:19:02.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.337 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.337 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.425959+0000", 2024-09-18T00:19:02.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033400484000000001 2024-09-18T00:19:02.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426106+0000", 2024-09-18T00:19:02.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014779800000000001 2024-09-18T00:19:02.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426237+0000", 2024-09-18T00:19:02.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013072600000000001 2024-09-18T00:19:02.338 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.338 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.338 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.339 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17562 3.a 3:5578626c:::rbd_data.112a6efb7564.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T00:19:02.339 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:02.339 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.291396789, 2024-09-18T00:19:02.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052733453999999999, 2024-09-18T00:19:02.339 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.339 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.339 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:02.339 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.339 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.339 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.339 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17562 2024-09-18T00:19:02.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.339 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:02.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:02.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957977+0000", 2024-09-18T00:19:02.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.119e-06 2024-09-18T00:19:02.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957988+0000", 2024-09-18T00:19:02.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0557000000000001e-05 2024-09-18T00:19:02.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958017+0000", 2024-09-18T00:19:02.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9156e-05 2024-09-18T00:19:02.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958036+0000", 2024-09-18T00:19:02.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8919e-05 2024-09-18T00:19:02.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958144+0000", 2024-09-18T00:19:02.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107905 2024-09-18T00:19:02.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958284+0000", 2024-09-18T00:19:02.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140216 2024-09-18T00:19:02.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:02.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.960572+0000", 2024-09-18T00:19:02.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022881860000000002 2024-09-18T00:19:02.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:02.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010537+0000", 2024-09-18T00:19:02.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049964664999999998 2024-09-18T00:19:02.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:02.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010590+0000", 2024-09-18T00:19:02.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3094999999999999e-05 2024-09-18T00:19:02.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:02.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010642+0000", 2024-09-18T00:19:02.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2073000000000001e-05 2024-09-18T00:19:02.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010709+0000", 2024-09-18T00:19:02.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6563000000000006e-05 2024-09-18T00:19:02.345 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.345 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.345 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.345 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17562 3.a e60/23 ondisk, result = 0)", 2024-09-18T00:19:02.345 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:02.345 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.281193418000001, 2024-09-18T00:19:02.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042539307999999998, 2024-09-18T00:19:02.345 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.345 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.346 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.346 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:02.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:02.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968180+0000", 2024-09-18T00:19:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-18T00:19:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968188+0000", 2024-09-18T00:19:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7419999999999997e-06 2024-09-18T00:19:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968217+0000", 2024-09-18T00:19:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8926999999999999e-05 2024-09-18T00:19:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968231+0000", 2024-09-18T00:19:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4168999999999999e-05 2024-09-18T00:19:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010575+0000", 2024-09-18T00:19:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042343923999999998 2024-09-18T00:19:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010584+0000", 2024-09-18T00:19:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-18T00:19:02.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010718+0000", 2024-09-18T00:19:02.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013344499999999999 2024-09-18T00:19:02.349 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.349 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.349 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.349 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.349 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.349 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:02.351 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:19:02.351 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:02.350+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:02.451 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:02.450+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:02.521 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:02.521 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:02.521 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:02.521 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:02.535 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:19:02.578 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:02.578+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:02.653 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:02.650+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:02.704 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:02.704 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:02.704 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:02.704 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:02.704 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:02.718 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:19:02.762 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:02.762+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:02.839 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:02.839 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:02.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:02.840 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:02.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.840 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2228 3.15 3:aa2fc059:::rbd_data.112aa7ba4578.000000000000014a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:02.840 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:02.840 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.574714260999997, 2024-09-18T00:19:02.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.068966377999999995, 2024-09-18T00:19:02.840 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.840 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.840 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:02.840 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.840 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.840 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.840 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2228 2024-09-18T00:19:02.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.841 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:02.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:02.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263099+0000", 2024-09-18T00:19:02.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2359999999999999e-06 2024-09-18T00:19:02.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263115+0000", 2024-09-18T00:19:02.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5996999999999999e-05 2024-09-18T00:19:02.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263156+0000", 2024-09-18T00:19:02.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:19:02.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263183+0000", 2024-09-18T00:19:02.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7233000000000002e-05 2024-09-18T00:19:02.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263304+0000", 2024-09-18T00:19:02.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121209 2024-09-18T00:19:02.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263528+0000", 2024-09-18T00:19:02.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022401800000000001 2024-09-18T00:19:02.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:19:02.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.266458+0000", 2024-09-18T00:19:02.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029296980000000001 2024-09-18T00:19:02.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:02.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331628+0000", 2024-09-18T00:19:02.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.065170045999999995 2024-09-18T00:19:02.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:02.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331819+0000", 2024-09-18T00:19:02.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000190819 2024-09-18T00:19:02.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:02.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331971+0000", 2024-09-18T00:19:02.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151972 2024-09-18T00:19:02.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332063+0000", 2024-09-18T00:19:02.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2083999999999995e-05 2024-09-18T00:19:02.846 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.846 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.846 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.846 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2228 3.15 e36/32 ondisk, result = 0)", 2024-09-18T00:19:02.846 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:19:02.846 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.561661149000003, 2024-09-18T00:19:02.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055930148999999998, 2024-09-18T00:19:02.846 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.846 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.847 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.847 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:19:02.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:19:02.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276152+0000", 2024-09-18T00:19:02.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0200000000000001e-06 2024-09-18T00:19:02.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276162+0000", 2024-09-18T00:19:02.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0767e-05 2024-09-18T00:19:02.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276202+0000", 2024-09-18T00:19:02.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000003e-05 2024-09-18T00:19:02.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276226+0000", 2024-09-18T00:19:02.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.351e-05 2024-09-18T00:19:02.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331782+0000", 2024-09-18T00:19:02.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055556296999999998 2024-09-18T00:19:02.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331801+0000", 2024-09-18T00:19:02.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9357000000000001e-05 2024-09-18T00:19:02.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332080+0000", 2024-09-18T00:19:02.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027849800000000002 2024-09-18T00:19:02.850 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.850 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.850 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.850 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2854 3.f 3:f02b0bca:::rbd_data.112aa7ba4578.0000000000000135:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:19:02.850 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:02.850 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 40.967741771, 2024-09-18T00:19:02.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.082582802999999996, 2024-09-18T00:19:02.851 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.851 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.851 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:02.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.851 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2854 2024-09-18T00:19:02.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.851 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:02.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:02.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870071+0000", 2024-09-18T00:19:02.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.227e-06 2024-09-18T00:19:02.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870083+0000", 2024-09-18T00:19:02.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1909999999999999e-05 2024-09-18T00:19:02.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870114+0000", 2024-09-18T00:19:02.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0576e-05 2024-09-18T00:19:02.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870129+0000", 2024-09-18T00:19:02.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.485e-05 2024-09-18T00:19:02.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870256+0000", 2024-09-18T00:19:02.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012691300000000001 2024-09-18T00:19:02.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870493+0000", 2024-09-18T00:19:02.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023737399999999999 2024-09-18T00:19:02.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:02.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.874933+0000", 2024-09-18T00:19:02.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044398299999999996 2024-09-18T00:19:02.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:02.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952452+0000", 2024-09-18T00:19:02.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.077518918000000006 2024-09-18T00:19:02.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:02.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952486+0000", 2024-09-18T00:19:02.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4035999999999997e-05 2024-09-18T00:19:02.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:02.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952575+0000", 2024-09-18T00:19:02.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.954e-05 2024-09-18T00:19:02.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952652+0000", 2024-09-18T00:19:02.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6629000000000001e-05 2024-09-18T00:19:02.856 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.856 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.857 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.857 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2854 3.f e38/32 ondisk, result = 0)", 2024-09-18T00:19:02.857 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:02.857 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 40.950210955999999, 2024-09-18T00:19:02.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064868625999999999, 2024-09-18T00:19:02.857 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.857 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.857 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.857 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:02.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:02.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887602+0000", 2024-09-18T00:19:02.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.677e-06 2024-09-18T00:19:02.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887609+0000", 2024-09-18T00:19:02.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6729999999999992e-06 2024-09-18T00:19:02.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887637+0000", 2024-09-18T00:19:02.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8206000000000001e-05 2024-09-18T00:19:02.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887651+0000", 2024-09-18T00:19:02.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3387000000000001e-05 2024-09-18T00:19:02.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952422+0000", 2024-09-18T00:19:02.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064771366999999996 2024-09-18T00:19:02.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952441+0000", 2024-09-18T00:19:02.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8737e-05 2024-09-18T00:19:02.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952469+0000", 2024-09-18T00:19:02.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7579000000000001e-05 2024-09-18T00:19:02.862 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.862 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.862 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.862 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5633 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:02.862 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297194+0000", 2024-09-18T00:19:02.862 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.540617126999997, 2024-09-18T00:19:02.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046979053, 2024-09-18T00:19:02.862 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.863 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.863 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.863 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.863 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.863 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.863 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5633 2024-09-18T00:19:02.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.863 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297194+0000", 2024-09-18T00:19:02.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297194+0000", 2024-09-18T00:19:02.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297195+0000", 2024-09-18T00:19:02.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2440000000000001e-06 2024-09-18T00:19:02.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297201+0000", 2024-09-18T00:19:02.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5489999999999997e-06 2024-09-18T00:19:02.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297218+0000", 2024-09-18T00:19:02.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7652999999999999e-05 2024-09-18T00:19:02.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297245+0000", 2024-09-18T00:19:02.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7212999999999998e-05 2024-09-18T00:19:02.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.343955+0000", 2024-09-18T00:19:02.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046709564000000002 2024-09-18T00:19:02.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344028+0000", 2024-09-18T00:19:02.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3370999999999999e-05 2024-09-18T00:19:02.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344173+0000", 2024-09-18T00:19:02.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014445899999999999 2024-09-18T00:19:02.868 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.868 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.868 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.869 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5634 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:02.869 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297263+0000", 2024-09-18T00:19:02.869 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.540547850999999, 2024-09-18T00:19:02.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047153893000000002, 2024-09-18T00:19:02.869 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.870 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.870 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.870 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.870 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.870 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.871 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5634 2024-09-18T00:19:02.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.871 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297263+0000", 2024-09-18T00:19:02.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297263+0000", 2024-09-18T00:19:02.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297264+0000", 2024-09-18T00:19:02.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4169999999999999e-06 2024-09-18T00:19:02.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297270+0000", 2024-09-18T00:19:02.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4829999999999999e-06 2024-09-18T00:19:02.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297287+0000", 2024-09-18T00:19:02.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6993000000000001e-05 2024-09-18T00:19:02.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297300+0000", 2024-09-18T00:19:02.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.324e-05 2024-09-18T00:19:02.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344221+0000", 2024-09-18T00:19:02.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046921030000000002 2024-09-18T00:19:02.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344310+0000", 2024-09-18T00:19:02.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8533000000000001e-05 2024-09-18T00:19:02.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344417+0000", 2024-09-18T00:19:02.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107197 2024-09-18T00:19:02.877 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.877 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.877 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.878 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5635 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:02.878 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.297317+0000", 2024-09-18T00:19:02.878 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.540494101, 2024-09-18T00:19:02.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047292222000000002, 2024-09-18T00:19:02.878 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.878 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.878 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.878 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.878 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.879 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.879 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5635 2024-09-18T00:19:02.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.879 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297317+0000", 2024-09-18T00:19:02.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297317+0000", 2024-09-18T00:19:02.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297318+0000", 2024-09-18T00:19:02.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2869999999999999e-06 2024-09-18T00:19:02.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297323+0000", 2024-09-18T00:19:02.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1760000000000001e-06 2024-09-18T00:19:02.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297343+0000", 2024-09-18T00:19:02.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9473000000000001e-05 2024-09-18T00:19:02.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.297356+0000", 2024-09-18T00:19:02.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3767e-05 2024-09-18T00:19:02.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344455+0000", 2024-09-18T00:19:02.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047098220000000003 2024-09-18T00:19:02.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344518+0000", 2024-09-18T00:19:02.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2856999999999998e-05 2024-09-18T00:19:02.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.344609+0000", 2024-09-18T00:19:02.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1441999999999996e-05 2024-09-18T00:19:02.883 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.883 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.883 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.883 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5674 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 10682368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:02.883 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298168+0000", 2024-09-18T00:19:02.883 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.539643298000001, 2024-09-18T00:19:02.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047098783999999998, 2024-09-18T00:19:02.883 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.883 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.883 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.884 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.884 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.884 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.884 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5674 2024-09-18T00:19:02.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.884 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298168+0000", 2024-09-18T00:19:02.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298168+0000", 2024-09-18T00:19:02.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298169+0000", 2024-09-18T00:19:02.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1370000000000001e-06 2024-09-18T00:19:02.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298174+0000", 2024-09-18T00:19:02.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6500000000000001e-06 2024-09-18T00:19:02.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298192+0000", 2024-09-18T00:19:02.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7620000000000001e-05 2024-09-18T00:19:02.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298205+0000", 2024-09-18T00:19:02.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3363000000000001e-05 2024-09-18T00:19:02.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345013+0000", 2024-09-18T00:19:02.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046807418000000003 2024-09-18T00:19:02.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345125+0000", 2024-09-18T00:19:02.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111901 2024-09-18T00:19:02.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345266+0000", 2024-09-18T00:19:02.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000141695 2024-09-18T00:19:02.888 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.888 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.888 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.888 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5675 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:02.889 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298223+0000", 2024-09-18T00:19:02.889 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.539587404999999, 2024-09-18T00:19:02.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047324643999999999, 2024-09-18T00:19:02.889 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.889 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.889 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.889 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.889 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.889 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.890 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5675 2024-09-18T00:19:02.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.890 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298223+0000", 2024-09-18T00:19:02.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298223+0000", 2024-09-18T00:19:02.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298225+0000", 2024-09-18T00:19:02.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.13e-06 2024-09-18T00:19:02.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298230+0000", 2024-09-18T00:19:02.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1599999999999997e-06 2024-09-18T00:19:02.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298247+0000", 2024-09-18T00:19:02.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6813999999999999e-05 2024-09-18T00:19:02.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298260+0000", 2024-09-18T00:19:02.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3036e-05 2024-09-18T00:19:02.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345315+0000", 2024-09-18T00:19:02.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047055420000000001 2024-09-18T00:19:02.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345384+0000", 2024-09-18T00:19:02.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9375999999999994e-05 2024-09-18T00:19:02.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345548+0000", 2024-09-18T00:19:02.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016370800000000001 2024-09-18T00:19:02.895 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.895 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.895 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.896 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5676 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 12255232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:02.896 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:19:02.896 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.539535098999998, 2024-09-18T00:19:02.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047571816000000003, 2024-09-18T00:19:02.896 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.896 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.896 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.896 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.896 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.897 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.897 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5676 2024-09-18T00:19:02.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.897 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:19:02.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:19:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298277+0000", 2024-09-18T00:19:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.454e-06 2024-09-18T00:19:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298282+0000", 2024-09-18T00:19:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6299999999999997e-06 2024-09-18T00:19:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298299+0000", 2024-09-18T00:19:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6959000000000001e-05 2024-09-18T00:19:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298313+0000", 2024-09-18T00:19:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.376e-05 2024-09-18T00:19:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345610+0000", 2024-09-18T00:19:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047297128000000001 2024-09-18T00:19:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345708+0000", 2024-09-18T00:19:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8245999999999994e-05 2024-09-18T00:19:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345848+0000", 2024-09-18T00:19:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000139639 2024-09-18T00:19:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5677 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 12779520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:19:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.539481586000001, 2024-09-18T00:19:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047810679000000002, 2024-09-18T00:19:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5677 2024-09-18T00:19:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:19:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:19:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298330+0000", 2024-09-18T00:19:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.217e-06 2024-09-18T00:19:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298354+0000", 2024-09-18T00:19:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3753000000000001e-05 2024-09-18T00:19:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298388+0000", 2024-09-18T00:19:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4276999999999997e-05 2024-09-18T00:19:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298415+0000", 2024-09-18T00:19:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6403000000000001e-05 2024-09-18T00:19:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345896+0000", 2024-09-18T00:19:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047481007999999998 2024-09-18T00:19:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345992+0000", 2024-09-18T00:19:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6141e-05 2024-09-18T00:19:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346140+0000", 2024-09-18T00:19:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014788 2024-09-18T00:19:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5678 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:19:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.539375407000001, 2024-09-18T00:19:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047985466999999997, 2024-09-18T00:19:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5678 2024-09-18T00:19:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:19:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:19:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298437+0000", 2024-09-18T00:19:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1799999999999999e-06 2024-09-18T00:19:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298443+0000", 2024-09-18T00:19:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1140000000000002e-06 2024-09-18T00:19:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298460+0000", 2024-09-18T00:19:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7615999999999999e-05 2024-09-18T00:19:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298474+0000", 2024-09-18T00:19:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4017e-05 2024-09-18T00:19:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346180+0000", 2024-09-18T00:19:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047705801999999999 2024-09-18T00:19:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346274+0000", 2024-09-18T00:19:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3814999999999995e-05 2024-09-18T00:19:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346421+0000", 2024-09-18T00:19:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000146923 2024-09-18T00:19:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5679 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:19:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.539316362999998, 2024-09-18T00:19:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048190771, 2024-09-18T00:19:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5679 2024-09-18T00:19:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:19:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:19:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298496+0000", 2024-09-18T00:19:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4699999999999999e-06 2024-09-18T00:19:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298501+0000", 2024-09-18T00:19:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:19:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298519+0000", 2024-09-18T00:19:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7329e-05 2024-09-18T00:19:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298532+0000", 2024-09-18T00:19:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3083e-05 2024-09-18T00:19:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346462+0000", 2024-09-18T00:19:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047930449999999999 2024-09-18T00:19:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346556+0000", 2024-09-18T00:19:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3222999999999998e-05 2024-09-18T00:19:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346685+0000", 2024-09-18T00:19:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000129696 2024-09-18T00:19:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5680 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:19:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.539258953999997, 2024-09-18T00:19:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048535847, 2024-09-18T00:19:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5680 2024-09-18T00:19:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:19:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:19:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298553+0000", 2024-09-18T00:19:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0759999999999999e-06 2024-09-18T00:19:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298575+0000", 2024-09-18T00:19:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2019999999999999e-05 2024-09-18T00:19:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298593+0000", 2024-09-18T00:19:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7900000000000001e-05 2024-09-18T00:19:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298607+0000", 2024-09-18T00:19:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3844e-05 2024-09-18T00:19:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346791+0000", 2024-09-18T00:19:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048184449999999997 2024-09-18T00:19:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346921+0000", 2024-09-18T00:19:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012994699999999999 2024-09-18T00:19:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347088+0000", 2024-09-18T00:19:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016661000000000001 2024-09-18T00:19:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5681 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:19:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.539186544000003, 2024-09-18T00:19:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048743337999999997, 2024-09-18T00:19:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5681 2024-09-18T00:19:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:19:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:19:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298626+0000", 2024-09-18T00:19:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4899999999999999e-06 2024-09-18T00:19:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298631+0000", 2024-09-18T00:19:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0300000000000001e-06 2024-09-18T00:19:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298650+0000", 2024-09-18T00:19:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9499000000000001e-05 2024-09-18T00:19:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298664+0000", 2024-09-18T00:19:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3767e-05 2024-09-18T00:19:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347139+0000", 2024-09-18T00:19:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048475348000000001 2024-09-18T00:19:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347228+0000", 2024-09-18T00:19:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8843000000000003e-05 2024-09-18T00:19:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347368+0000", 2024-09-18T00:19:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013936100000000001 2024-09-18T00:19:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5682 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:19:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.539129729000003, 2024-09-18T00:19:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048916864999999997, 2024-09-18T00:19:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5682 2024-09-18T00:19:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:19:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:19:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298682+0000", 2024-09-18T00:19:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:19:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298687+0000", 2024-09-18T00:19:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9239999999999998e-06 2024-09-18T00:19:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298704+0000", 2024-09-18T00:19:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6569999999999999e-05 2024-09-18T00:19:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298731+0000", 2024-09-18T00:19:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6752999999999999e-05 2024-09-18T00:19:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347416+0000", 2024-09-18T00:19:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048685056999999997 2024-09-18T00:19:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347506+0000", 2024-09-18T00:19:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9936000000000001e-05 2024-09-18T00:19:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347598+0000", 2024-09-18T00:19:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2335000000000003e-05 2024-09-18T00:19:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15879 3.16 3:692b47b6:::rbd_object_map.112a13ec2d6c.000000000000000b:head [call] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:19:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.183047048999999, 2024-09-18T00:19:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.011184018, 2024-09-18T00:19:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15879 2024-09-18T00:19:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654768+0000", 2024-09-18T00:19:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9129999999999996e-06 2024-09-18T00:19:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654786+0000", 2024-09-18T00:19:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8695999999999999e-05 2024-09-18T00:19:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654832+0000", 2024-09-18T00:19:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5735000000000002e-05 2024-09-18T00:19:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654860+0000", 2024-09-18T00:19:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7789000000000001e-05 2024-09-18T00:19:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655048+0000", 2024-09-18T00:19:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018781900000000001 2024-09-18T00:19:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:19:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655070+0000", 2024-09-18T00:19:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1916000000000002e-05 2024-09-18T00:19:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663335+0000", 2024-09-18T00:19:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0082651210000000003 2024-09-18T00:19:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663358+0000", 2024-09-18T00:19:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2986e-05 2024-09-18T00:19:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663739+0000", 2024-09-18T00:19:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0003814820000001 2024-09-18T00:19:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663920+0000", 2024-09-18T00:19:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018116900000000001 2024-09-18T00:19:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.665948+0000", 2024-09-18T00:19:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020273919999999998 2024-09-18T00:19:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15881 3.16 3:6cc14920:::rbd_header.112a3285400b:head [watch reconnect cookie 140685950184608 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:19:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.182886054000001, 2024-09-18T00:19:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0138723599999999, 2024-09-18T00:19:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15881 2024-09-18T00:19:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654926+0000", 2024-09-18T00:19:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6470000000000001e-06 2024-09-18T00:19:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654935+0000", 2024-09-18T00:19:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8859999999999993e-06 2024-09-18T00:19:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654960+0000", 2024-09-18T00:19:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4689999999999999e-05 2024-09-18T00:19:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654976+0000", 2024-09-18T00:19:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6319000000000001e-05 2024-09-18T00:19:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655238+0000", 2024-09-18T00:19:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026145999999999998 2024-09-18T00:19:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655255+0000", 2024-09-18T00:19:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7249999999999999e-05 2024-09-18T00:19:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663416+0000", 2024-09-18T00:19:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081607959999999997 2024-09-18T00:19:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663436+0000", 2024-09-18T00:19:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0384e-05 2024-09-18T00:19:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666184+0000", 2024-09-18T00:19:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002748121 2024-09-18T00:19:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666452+0000", 2024-09-18T00:19:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026801700000000001 2024-09-18T00:19:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.668797+0000", 2024-09-18T00:19:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023447899999999998 2024-09-18T00:19:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17089 3.f 3:f7681386:::rbd_data.112a6efb7564.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:19:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.957958958000001, 2024-09-18T00:19:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.12573889399999999, 2024-09-18T00:19:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17089 2024-09-18T00:19:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879854+0000", 2024-09-18T00:19:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:19:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879867+0000", 2024-09-18T00:19:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3103e-05 2024-09-18T00:19:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879898+0000", 2024-09-18T00:19:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0103e-05 2024-09-18T00:19:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879912+0000", 2024-09-18T00:19:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4636e-05 2024-09-18T00:19:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879992+0000", 2024-09-18T00:19:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9814000000000005e-05 2024-09-18T00:19:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.880101+0000", 2024-09-18T00:19:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109172 2024-09-18T00:19:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.887269+0000", 2024-09-18T00:19:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071679300000000003 2024-09-18T00:19:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005269+0000", 2024-09-18T00:19:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.117999621 2024-09-18T00:19:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005317+0000", 2024-09-18T00:19:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8279000000000003e-05 2024-09-18T00:19:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005464+0000", 2024-09-18T00:19:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014661200000000001 2024-09-18T00:19:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005591+0000", 2024-09-18T00:19:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000127107 2024-09-18T00:19:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17089 3.f e59/32 ondisk, result = 0)", 2024-09-18T00:19:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.933852093, 2024-09-18T00:19:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101334403, 2024-09-18T00:19:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903960+0000", 2024-09-18T00:19:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.33e-06 2024-09-18T00:19:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903966+0000", 2024-09-18T00:19:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5929999999999996e-06 2024-09-18T00:19:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903986+0000", 2024-09-18T00:19:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.018e-05 2024-09-18T00:19:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903997+0000", 2024-09-18T00:19:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0988999999999999e-05 2024-09-18T00:19:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005171+0000", 2024-09-18T00:19:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101174267 2024-09-18T00:19:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005200+0000", 2024-09-18T00:19:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8540000000000001e-05 2024-09-18T00:19:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005293+0000", 2024-09-18T00:19:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504000000000005e-05 2024-09-18T00:19:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:02.961 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:02.962 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:19:02.962 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:02.958+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:03.064 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:03.058+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:03.092 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:03.092 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:03.092 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:03.093 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:03.108 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:19:03.164 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:03.162+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:03.265 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:03.262+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:03.270 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:03.270 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:03.270 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:03.284 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:03.366 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:03.362+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:03.405 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2230 3.17 3:ef492888:::rbd_data.112aa7ba4578.000000000000014f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 43.136577199999998, 2024-09-18T00:19:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075065283999999996, 2024-09-18T00:19:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.406 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2230 2024-09-18T00:19:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266014+0000", 2024-09-18T00:19:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:19:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266029+0000", 2024-09-18T00:19:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4616e-05 2024-09-18T00:19:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266053+0000", 2024-09-18T00:19:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4369999999999999e-05 2024-09-18T00:19:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266067+0000", 2024-09-18T00:19:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4243999999999999e-05 2024-09-18T00:19:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266157+0000", 2024-09-18T00:19:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9909999999999998e-05 2024-09-18T00:19:03.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266327+0000", 2024-09-18T00:19:03.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016981200000000001 2024-09-18T00:19:03.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:19:03.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269180+0000", 2024-09-18T00:19:03.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028526559999999999 2024-09-18T00:19:03.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:03.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340955+0000", 2024-09-18T00:19:03.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.071774904 2024-09-18T00:19:03.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:03.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340974+0000", 2024-09-18T00:19:03.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8824000000000001e-05 2024-09-18T00:19:03.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:03.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341028+0000", 2024-09-18T00:19:03.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4456999999999997e-05 2024-09-18T00:19:03.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341076+0000", 2024-09-18T00:19:03.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8284000000000001e-05 2024-09-18T00:19:03.412 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.412 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.412 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.412 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2230 3.17 e36/23 ondisk, result = 0)", 2024-09-18T00:19:03.412 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:03.412 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 43.119233618999999, 2024-09-18T00:19:03.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057609645000000001, 2024-09-18T00:19:03.412 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.412 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.412 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:03.412 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:03.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:03.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.413 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.413 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283356+0000", 2024-09-18T00:19:03.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.44e-06 2024-09-18T00:19:03.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283362+0000", 2024-09-18T00:19:03.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2539999999999998e-06 2024-09-18T00:19:03.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283382+0000", 2024-09-18T00:19:03.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9896000000000001e-05 2024-09-18T00:19:03.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283398+0000", 2024-09-18T00:19:03.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5489999999999999e-05 2024-09-18T00:19:03.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340936+0000", 2024-09-18T00:19:03.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057538649999999997 2024-09-18T00:19:03.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340947+0000", 2024-09-18T00:19:03.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0981e-05 2024-09-18T00:19:03.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340964+0000", 2024-09-18T00:19:03.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6934000000000001e-05 2024-09-18T00:19:03.416 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.416 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.416 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.416 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2419 3.17 3:ed94b133:::rbd_data.112aa7ba4578.00000000000001b5:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:19:03.416 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:03.416 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.665112145999998, 2024-09-18T00:19:03.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.064761812000000002, 2024-09-18T00:19:03.416 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.416 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.416 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:03.417 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:03.417 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:03.417 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:03.417 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2419 2024-09-18T00:19:03.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.417 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:03.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:03.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737479+0000", 2024-09-18T00:19:03.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6089999999999999e-06 2024-09-18T00:19:03.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737487+0000", 2024-09-18T00:19:03.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7900000000000005e-06 2024-09-18T00:19:03.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737507+0000", 2024-09-18T00:19:03.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9936e-05 2024-09-18T00:19:03.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737518+0000", 2024-09-18T00:19:03.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1046999999999999e-05 2024-09-18T00:19:03.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737599+0000", 2024-09-18T00:19:03.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0697000000000003e-05 2024-09-18T00:19:03.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737697+0000", 2024-09-18T00:19:03.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8058000000000002e-05 2024-09-18T00:19:03.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:19:03.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.740445+0000", 2024-09-18T00:19:03.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027481279999999999 2024-09-18T00:19:03.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:03.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802023+0000", 2024-09-18T00:19:03.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.061577321999999997 2024-09-18T00:19:03.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:03.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802060+0000", 2024-09-18T00:19:03.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7586000000000002e-05 2024-09-18T00:19:03.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:03.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802158+0000", 2024-09-18T00:19:03.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7583000000000004e-05 2024-09-18T00:19:03.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802238+0000", 2024-09-18T00:19:03.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0055999999999994e-05 2024-09-18T00:19:03.422 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.422 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.422 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.422 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2419 3.17 e37/23 ondisk, result = 0)", 2024-09-18T00:19:03.422 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:03.422 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.652887200000002, 2024-09-18T00:19:03.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052342948, 2024-09-18T00:19:03.422 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.422 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.422 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:03.423 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:03.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:03.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749702+0000", 2024-09-18T00:19:03.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:19:03.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749710+0000", 2024-09-18T00:19:03.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7279999999999995e-06 2024-09-18T00:19:03.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749736+0000", 2024-09-18T00:19:03.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5797999999999999e-05 2024-09-18T00:19:03.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749751+0000", 2024-09-18T00:19:03.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5238999999999999e-05 2024-09-18T00:19:03.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801745+0000", 2024-09-18T00:19:03.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051994022000000001 2024-09-18T00:19:03.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801770+0000", 2024-09-18T00:19:03.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5103e-05 2024-09-18T00:19:03.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802044+0000", 2024-09-18T00:19:03.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027395799999999998 2024-09-18T00:19:03.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.426 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.426 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6503 3.1b 3:d9387a1f:::rbd_data.112a3a39fc45.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:03.426 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:03.426 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 37.838214229999998, 2024-09-18T00:19:03.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050510547000000003, 2024-09-18T00:19:03.426 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.427 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.427 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:03.427 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:03.427 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:03.427 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:03.427 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6503 2024-09-18T00:19:03.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.427 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:03.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:03.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564376+0000", 2024-09-18T00:19:03.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-06 2024-09-18T00:19:03.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564386+0000", 2024-09-18T00:19:03.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0312999999999999e-05 2024-09-18T00:19:03.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564408+0000", 2024-09-18T00:19:03.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2001000000000002e-05 2024-09-18T00:19:03.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564421+0000", 2024-09-18T00:19:03.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2271e-05 2024-09-18T00:19:03.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564491+0000", 2024-09-18T00:19:03.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0630000000000006e-05 2024-09-18T00:19:03.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564606+0000", 2024-09-18T00:19:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011449 2024-09-18T00:19:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.566743+0000", 2024-09-18T00:19:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021373830000000001 2024-09-18T00:19:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614738+0000", 2024-09-18T00:19:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047995359000000001 2024-09-18T00:19:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:03.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614759+0000", 2024-09-18T00:19:03.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1010999999999999e-05 2024-09-18T00:19:03.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:03.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614826+0000", 2024-09-18T00:19:03.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6654000000000003e-05 2024-09-18T00:19:03.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614884+0000", 2024-09-18T00:19:03.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8332999999999999e-05 2024-09-18T00:19:03.432 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.432 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.432 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.432 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6503 3.1b e42/23 ondisk, result = 0)", 2024-09-18T00:19:03.432 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:19:03.432 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 37.828954762000002, 2024-09-18T00:19:03.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041115407, 2024-09-18T00:19:03.432 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.432 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.432 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:03.432 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:19:03.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:19:03.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573635+0000", 2024-09-18T00:19:03.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5689999999999999e-06 2024-09-18T00:19:03.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573643+0000", 2024-09-18T00:19:03.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9699999999999999e-06 2024-09-18T00:19:03.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573668+0000", 2024-09-18T00:19:03.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5536000000000001e-05 2024-09-18T00:19:03.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573683+0000", 2024-09-18T00:19:03.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4025e-05 2024-09-18T00:19:03.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614717+0000", 2024-09-18T00:19:03.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041034517 2024-09-18T00:19:03.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614730+0000", 2024-09-18T00:19:03.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2945999999999999e-05 2024-09-18T00:19:03.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614749+0000", 2024-09-18T00:19:03.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8844000000000001e-05 2024-09-18T00:19:03.436 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.436 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.436 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.436 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6518 3.1 3:8061bf9f:::rbd_data.112a3a39fc45.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:03.436 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:03.436 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 37.751775956000003, 2024-09-18T00:19:03.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062303123000000002, 2024-09-18T00:19:03.437 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.437 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.437 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:03.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:03.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:03.437 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:03.437 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6518 2024-09-18T00:19:03.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.437 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:03.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:03.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650814+0000", 2024-09-18T00:19:03.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.629e-06 2024-09-18T00:19:03.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650823+0000", 2024-09-18T00:19:03.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1800000000000002e-06 2024-09-18T00:19:03.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650843+0000", 2024-09-18T00:19:03.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9836000000000001e-05 2024-09-18T00:19:03.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650854+0000", 2024-09-18T00:19:03.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1564e-05 2024-09-18T00:19:03.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650917+0000", 2024-09-18T00:19:03.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3076000000000004e-05 2024-09-18T00:19:03.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.651029+0000", 2024-09-18T00:19:03.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111154 2024-09-18T00:19:03.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:03.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.655531+0000", 2024-09-18T00:19:03.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045027629999999999 2024-09-18T00:19:03.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:03.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712924+0000", 2024-09-18T00:19:03.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057393006000000003 2024-09-18T00:19:03.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:03.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712988+0000", 2024-09-18T00:19:03.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000005e-05 2024-09-18T00:19:03.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:03.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713056+0000", 2024-09-18T00:19:03.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7904999999999994e-05 2024-09-18T00:19:03.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713115+0000", 2024-09-18T00:19:03.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9339999999999998e-05 2024-09-18T00:19:03.442 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.442 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.442 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.442 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6518 3.1 e42/23 ondisk, result = 0)", 2024-09-18T00:19:03.442 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:03.442 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 37.739350872999999, 2024-09-18T00:19:03.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049887304, 2024-09-18T00:19:03.442 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.442 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.442 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:03.442 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:03.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:03.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663239+0000", 2024-09-18T00:19:03.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:19:03.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663246+0000", 2024-09-18T00:19:03.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4189999999999996e-06 2024-09-18T00:19:03.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663265+0000", 2024-09-18T00:19:03.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8323000000000001e-05 2024-09-18T00:19:03.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663277+0000", 2024-09-18T00:19:03.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2065999999999999e-05 2024-09-18T00:19:03.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712965+0000", 2024-09-18T00:19:03.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049688317000000003 2024-09-18T00:19:03.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712977+0000", 2024-09-18T00:19:03.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.206e-05 2024-09-18T00:19:03.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713125+0000", 2024-09-18T00:19:03.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014749899999999999 2024-09-18T00:19:03.446 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.446 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.446 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.446 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:8831 3.12 3:49cac8cf:::rbd_data.112a9f4259df.0000000000000130:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T00:19:03.446 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:03.446 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.121490624, 2024-09-18T00:19:03.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104125047, 2024-09-18T00:19:03.446 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.446 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.446 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:03.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:03.446 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:03.447 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:03.447 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 8831 2024-09-18T00:19:03.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.447 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:03.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:03.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281099+0000", 2024-09-18T00:19:03.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.029e-06 2024-09-18T00:19:03.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281103+0000", 2024-09-18T00:19:03.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7759999999999997e-06 2024-09-18T00:19:03.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281114+0000", 2024-09-18T00:19:03.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1090000000000001e-05 2024-09-18T00:19:03.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281120+0000", 2024-09-18T00:19:03.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.604e-06 2024-09-18T00:19:03.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281164+0000", 2024-09-18T00:19:03.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3696999999999999e-05 2024-09-18T00:19:03.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281270+0000", 2024-09-18T00:19:03.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106041 2024-09-18T00:19:03.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:03.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.285596+0000", 2024-09-18T00:19:03.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043266659999999998 2024-09-18T00:19:03.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:03.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385028+0000", 2024-09-18T00:19:03.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.099431808999999996 2024-09-18T00:19:03.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:03.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385094+0000", 2024-09-18T00:19:03.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5740000000000004e-05 2024-09-18T00:19:03.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:03.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385166+0000", 2024-09-18T00:19:03.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1637000000000005e-05 2024-09-18T00:19:03.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.452 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.452 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385223+0000", 2024-09-18T00:19:03.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6957999999999999e-05 2024-09-18T00:19:03.452 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.452 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.452 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.452 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:8831 3.12 e48/32 ondisk, result = 0)", 2024-09-18T00:19:03.452 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:03.452 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.104976514000001, 2024-09-18T00:19:03.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087621558000000002, 2024-09-18T00:19:03.452 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.452 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.452 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:03.453 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:03.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:03.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297613+0000", 2024-09-18T00:19:03.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3400000000000001e-06 2024-09-18T00:19:03.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297620+0000", 2024-09-18T00:19:03.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5869999999999998e-06 2024-09-18T00:19:03.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297639+0000", 2024-09-18T00:19:03.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9888999999999999e-05 2024-09-18T00:19:03.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297661+0000", 2024-09-18T00:19:03.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1772000000000001e-05 2024-09-18T00:19:03.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385073+0000", 2024-09-18T00:19:03.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087411303999999995 2024-09-18T00:19:03.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385086+0000", 2024-09-18T00:19:03.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3040000000000001e-05 2024-09-18T00:19:03.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385233+0000", 2024-09-18T00:19:03.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014762600000000001 2024-09-18T00:19:03.456 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.456 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.456 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.456 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10109 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:19:03.456 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:19:03.456 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.829600954, 2024-09-18T00:19:03.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026513858000000001, 2024-09-18T00:19:03.456 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.456 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.456 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:03.457 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:03.457 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:03.457 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:03.457 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10109 2024-09-18T00:19:03.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.457 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:19:03.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:19:03.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572988+0000", 2024-09-18T00:19:03.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1099999999999996e-07 2024-09-18T00:19:03.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.458 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572990+0000", 2024-09-18T00:19:03.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3329999999999999e-06 2024-09-18T00:19:03.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.458 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572998+0000", 2024-09-18T00:19:03.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2039999999999997e-06 2024-09-18T00:19:03.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573012+0000", 2024-09-18T00:19:03.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3879e-05 2024-09-18T00:19:03.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599312+0000", 2024-09-18T00:19:03.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026300093 2024-09-18T00:19:03.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599380+0000", 2024-09-18T00:19:03.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7401000000000001e-05 2024-09-18T00:19:03.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599501+0000", 2024-09-18T00:19:03.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000121437 2024-09-18T00:19:03.460 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.460 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.460 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.460 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10110 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:19:03.461 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:19:03.461 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.829567494999999, 2024-09-18T00:19:03.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026746066999999998, 2024-09-18T00:19:03.461 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.461 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.461 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:03.461 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:03.461 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:03.461 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:03.461 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10110 2024-09-18T00:19:03.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.461 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:19:03.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:19:03.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:19:03.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2799999999999996e-07 2024-09-18T00:19:03.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573030+0000", 2024-09-18T00:19:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5469999999999997e-06 2024-09-18T00:19:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573039+0000", 2024-09-18T00:19:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6080000000000002e-06 2024-09-18T00:19:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573049+0000", 2024-09-18T00:19:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0759999999999992e-06 2024-09-18T00:19:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599539+0000", 2024-09-18T00:19:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026490952000000002 2024-09-18T00:19:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599621+0000", 2024-09-18T00:19:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1601999999999996e-05 2024-09-18T00:19:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599767+0000", 2024-09-18T00:19:03.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145654 2024-09-18T00:19:03.465 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.465 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.465 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.465 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10111 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:19:03.465 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:19:03.465 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.829529279999999, 2024-09-18T00:19:03.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027006651999999999, 2024-09-18T00:19:03.465 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.465 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.465 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:03.465 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:03.466 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:03.466 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:03.466 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10111 2024-09-18T00:19:03.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.466 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:19:03.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:19:03.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:19:03.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9500000000000002e-07 2024-09-18T00:19:03.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573062+0000", 2024-09-18T00:19:03.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6539999999999999e-06 2024-09-18T00:19:03.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573071+0000", 2024-09-18T00:19:03.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5380000000000002e-06 2024-09-18T00:19:03.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573077+0000", 2024-09-18T00:19:03.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999999e-06 2024-09-18T00:19:03.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599818+0000", 2024-09-18T00:19:03.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02674093 2024-09-18T00:19:03.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599923+0000", 2024-09-18T00:19:03.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010490200000000001 2024-09-18T00:19:03.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600066+0000", 2024-09-18T00:19:03.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014284500000000001 2024-09-18T00:19:03.469 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.469 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.469 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.469 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10112 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:19:03.469 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:19:03.470 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.829503486, 2024-09-18T00:19:03.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027148835, 2024-09-18T00:19:03.470 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.470 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.470 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:03.470 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:03.470 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:03.470 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:03.470 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10112 2024-09-18T00:19:03.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.470 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:19:03.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:19:03.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:19:03.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9500000000000003e-07 2024-09-18T00:19:03.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573088+0000", 2024-09-18T00:19:03.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9569999999999998e-06 2024-09-18T00:19:03.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573097+0000", 2024-09-18T00:19:03.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8599999999999999e-06 2024-09-18T00:19:03.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573104+0000", 2024-09-18T00:19:03.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5710000000000003e-06 2024-09-18T00:19:03.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600091+0000", 2024-09-18T00:19:03.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026987315000000001 2024-09-18T00:19:03.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600158+0000", 2024-09-18T00:19:03.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6727e-05 2024-09-18T00:19:03.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600234+0000", 2024-09-18T00:19:03.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5909999999999997e-05 2024-09-18T00:19:03.473 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.474 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.474 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.474 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10113 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:19:03.474 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:19:03.474 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.829459228000001, 2024-09-18T00:19:03.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027356020000000002, 2024-09-18T00:19:03.474 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.474 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.474 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:03.474 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:03.474 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:03.474 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:03.474 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10113 2024-09-18T00:19:03.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.475 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:19:03.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:19:03.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573130+0000", 2024-09-18T00:19:03.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1999999999999999e-07 2024-09-18T00:19:03.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573133+0000", 2024-09-18T00:19:03.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:19:03.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573141+0000", 2024-09-18T00:19:03.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5110000000000003e-06 2024-09-18T00:19:03.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573155+0000", 2024-09-18T00:19:03.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3730000000000001e-05 2024-09-18T00:19:03.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600275+0000", 2024-09-18T00:19:03.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027119621999999999 2024-09-18T00:19:03.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600349+0000", 2024-09-18T00:19:03.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4624999999999998e-05 2024-09-18T00:19:03.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600485+0000", 2024-09-18T00:19:03.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013570499999999999 2024-09-18T00:19:03.478 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.478 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.478 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.478 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10114 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:19:03.478 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:19:03.478 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.829423864999999, 2024-09-18T00:19:03.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027471173000000002, 2024-09-18T00:19:03.479 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.479 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.479 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:03.479 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:03.479 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:03.479 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:03.479 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10114 2024-09-18T00:19:03.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.479 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:19:03.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:19:03.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573165+0000", 2024-09-18T00:19:03.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.99e-07 2024-09-18T00:19:03.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573168+0000", 2024-09-18T00:19:03.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8339999999999999e-06 2024-09-18T00:19:03.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573176+0000", 2024-09-18T00:19:03.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0199999999999994e-06 2024-09-18T00:19:03.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573182+0000", 2024-09-18T00:19:03.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.455e-06 2024-09-18T00:19:03.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600521+0000", 2024-09-18T00:19:03.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027338989000000001 2024-09-18T00:19:03.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600577+0000", 2024-09-18T00:19:03.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5717000000000002e-05 2024-09-18T00:19:03.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600635+0000", 2024-09-18T00:19:03.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8559e-05 2024-09-18T00:19:03.482 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.482 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10115 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.829370988000001, 2024-09-18T00:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027643300999999999, 2024-09-18T00:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:03.483 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10115 2024-09-18T00:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573218+0000", 2024-09-18T00:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-18T00:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573221+0000", 2024-09-18T00:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3079999999999999e-06 2024-09-18T00:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573233+0000", 2024-09-18T00:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1544e-05 2024-09-18T00:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573240+0000", 2024-09-18T00:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4070000000000004e-06 2024-09-18T00:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600665+0000", 2024-09-18T00:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02742522 2024-09-18T00:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600745+0000", 2024-09-18T00:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0014999999999999e-05 2024-09-18T00:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600860+0000", 2024-09-18T00:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115127 2024-09-18T00:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11459 3.1b 3:db862ee8:::rbd_data.112ab1fa102a.0000000000000152:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.140942176999999, 2024-09-18T00:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.032246431999999998, 2024-09-18T00:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.487 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11459 2024-09-18T00:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261649+0000", 2024-09-18T00:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1200000000000002e-06 2024-09-18T00:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261665+0000", 2024-09-18T00:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5668999999999999e-05 2024-09-18T00:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261700+0000", 2024-09-18T00:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4925000000000002e-05 2024-09-18T00:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261723+0000", 2024-09-18T00:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3189000000000001e-05 2024-09-18T00:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261815+0000", 2024-09-18T00:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1662000000000004e-05 2024-09-18T00:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261904+0000", 2024-09-18T00:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9201000000000002e-05 2024-09-18T00:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.262955+0000", 2024-09-18T00:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00105156 2024-09-18T00:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293760+0000", 2024-09-18T00:19:03.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030804292 2024-09-18T00:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293773+0000", 2024-09-18T00:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3361e-05 2024-09-18T00:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293833+0000", 2024-09-18T00:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9626000000000001e-05 2024-09-18T00:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293892+0000", 2024-09-18T00:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9827000000000002e-05 2024-09-18T00:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22269 3.17 3:e9a1eb35:::rbd_data.112a948d5956.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.4934446880000003, 2024-09-18T00:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.063462693000000001, 2024-09-18T00:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:03.493 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22269 2024-09-18T00:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:03.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909146+0000", 2024-09-18T00:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3700000000000002e-06 2024-09-18T00:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909160+0000", 2024-09-18T00:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3825000000000001e-05 2024-09-18T00:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909178+0000", 2024-09-18T00:19:03.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8567999999999999e-05 2024-09-18T00:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909189+0000", 2024-09-18T00:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0881e-05 2024-09-18T00:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909270+0000", 2024-09-18T00:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0846999999999994e-05 2024-09-18T00:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909383+0000", 2024-09-18T00:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112704 2024-09-18T00:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972310+0000", 2024-09-18T00:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062927081999999995 2024-09-18T00:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972433+0000", 2024-09-18T00:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-18T00:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972606+0000", 2024-09-18T00:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017317799999999999 2024-09-18T00:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22379 3.17 3:e9a1eb35:::rbd_data.112a948d5956.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.9993693910000001, 2024-09-18T00:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.12560279099999999, 2024-09-18T00:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:03.498 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22379 2024-09-18T00:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:03.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403221+0000", 2024-09-18T00:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3269999999999999e-06 2024-09-18T00:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403232+0000", 2024-09-18T00:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1392e-05 2024-09-18T00:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403264+0000", 2024-09-18T00:19:03.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1146999999999998e-05 2024-09-18T00:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403279+0000", 2024-09-18T00:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5109000000000001e-05 2024-09-18T00:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403424+0000", 2024-09-18T00:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014516999999999999 2024-09-18T00:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:03.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403567+0000", 2024-09-18T00:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142824 2024-09-18T00:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528723+0000", 2024-09-18T00:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.12515647299999999 2024-09-18T00:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528764+0000", 2024-09-18T00:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0624000000000003e-05 2024-09-18T00:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:03.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528822+0000", 2024-09-18T00:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7725e-05 2024-09-18T00:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:03.503 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:03.504 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:19:03.505 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:03.498+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:03.605 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:03.602+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:03.675 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:03.676 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:03.676 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:03.676 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:03.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:19:03.733 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:03.730+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:03.807 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:03.802+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:03.820 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:19:03.823 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:03.823 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:03.824 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:03.824 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:03.824 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:03.833 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:19:03.845 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:19:03.937 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:03.934+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:04.009 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:04.006+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:04.017 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:04.017 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:04.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:04.018 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:04.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.018 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:5722 3.4 e41/23 3:20381d86:::rbd_data.112a3a39fc45.000000000000001d:head v 41'107, mlcod=40'105)", 2024-09-18T00:19:04.018 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:04.018 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.680488066999999, 2024-09-18T00:19:04.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01185172, 2024-09-18T00:19:04.018 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.018 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.018 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:04.018 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:04.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:04.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335497+0000", 2024-09-18T00:19:04.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.505e-06 2024-09-18T00:19:04.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335945+0000", 2024-09-18T00:19:04.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000447687 2024-09-18T00:19:04.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335961+0000", 2024-09-18T00:19:04.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6430999999999999e-05 2024-09-18T00:19:04.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335969+0000", 2024-09-18T00:19:04.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:19:04.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336030+0000", 2024-09-18T00:19:04.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1255999999999998e-05 2024-09-18T00:19:04.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336061+0000", 2024-09-18T00:19:04.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0579e-05 2024-09-18T00:19:04.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:04.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347296+0000", 2024-09-18T00:19:04.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011235663 2024-09-18T00:19:04.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347347+0000", 2024-09-18T00:19:04.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0914000000000001e-05 2024-09-18T00:19:04.022 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.022 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.022 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.022 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11457 3.0 3:0214e7b3:::rbd_data.112ab1fa102a.0000000000000150:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:04.022 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:04.022 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.756122129000001, 2024-09-18T00:19:04.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028499165, 2024-09-18T00:19:04.022 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.023 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.023 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:04.023 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.023 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.023 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.023 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11457 2024-09-18T00:19:04.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.023 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:04.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:04.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259865+0000", 2024-09-18T00:19:04.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7230000000000002e-06 2024-09-18T00:19:04.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259883+0000", 2024-09-18T00:19:04.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8067000000000002e-05 2024-09-18T00:19:04.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259924+0000", 2024-09-18T00:19:04.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0550999999999999e-05 2024-09-18T00:19:04.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259943+0000", 2024-09-18T00:19:04.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:19:04.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260054+0000", 2024-09-18T00:19:04.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011075 2024-09-18T00:19:04.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260138+0000", 2024-09-18T00:19:04.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4549000000000007e-05 2024-09-18T00:19:04.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:04.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.287926+0000", 2024-09-18T00:19:04.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027787642000000001 2024-09-18T00:19:04.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:04.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288090+0000", 2024-09-18T00:19:04.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016402499999999999 2024-09-18T00:19:04.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288361+0000", 2024-09-18T00:19:04.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027097800000000001 2024-09-18T00:19:04.027 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.027 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.027 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.028 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18661 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:04.028 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:04.028 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 11.081354264, 2024-09-18T00:19:04.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020008037999999999, 2024-09-18T00:19:04.028 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.028 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.028 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:04.028 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.028 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.028 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.028 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18661 2024-09-18T00:19:04.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.028 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:04.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:04.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934633+0000", 2024-09-18T00:19:04.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8e-06 2024-09-18T00:19:04.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935511+0000", 2024-09-18T00:19:04.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087785700000000001 2024-09-18T00:19:04.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935561+0000", 2024-09-18T00:19:04.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9472000000000003e-05 2024-09-18T00:19:04.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935586+0000", 2024-09-18T00:19:04.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5616e-05 2024-09-18T00:19:04.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935838+0000", 2024-09-18T00:19:04.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025202699999999999 2024-09-18T00:19:04.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.936023+0000", 2024-09-18T00:19:04.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018480300000000001 2024-09-18T00:19:04.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:04.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953545+0000", 2024-09-18T00:19:04.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017522157 2024-09-18T00:19:04.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:04.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953723+0000", 2024-09-18T00:19:04.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017753800000000001 2024-09-18T00:19:04.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954637+0000", 2024-09-18T00:19:04.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091476800000000005 2024-09-18T00:19:04.032 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.032 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.032 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.033 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18662 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:04.033 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:04.033 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 11.080349327, 2024-09-18T00:19:04.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019388843999999999, 2024-09-18T00:19:04.033 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.033 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.033 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:04.033 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.033 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.033 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.033 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18662 2024-09-18T00:19:04.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.033 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:04.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:04.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935636+0000", 2024-09-18T00:19:04.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.663e-06 2024-09-18T00:19:04.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935646+0000", 2024-09-18T00:19:04.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000006e-06 2024-09-18T00:19:04.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935672+0000", 2024-09-18T00:19:04.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5868999999999999e-05 2024-09-18T00:19:04.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935692+0000", 2024-09-18T00:19:04.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0604000000000001e-05 2024-09-18T00:19:04.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950738+0000", 2024-09-18T00:19:04.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015045868 2024-09-18T00:19:04.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950992+0000", 2024-09-18T00:19:04.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025368400000000001 2024-09-18T00:19:04.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:04.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954674+0000", 2024-09-18T00:19:04.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003682154 2024-09-18T00:19:04.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:04.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954806+0000", 2024-09-18T00:19:04.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013252000000000001 2024-09-18T00:19:04.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.955023+0000", 2024-09-18T00:19:04.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021681200000000001 2024-09-18T00:19:04.038 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.038 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.038 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.038 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19382 3.4 3:22f09d3a:::rbd_data.112a7fe29ab4.000000000000015a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:04.038 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:04.038 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.393939276999999, 2024-09-18T00:19:04.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075119302999999998, 2024-09-18T00:19:04.038 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.038 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.039 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:04.039 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.039 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.039 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.039 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19382 2024-09-18T00:19:04.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.039 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:04.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:04.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622045+0000", 2024-09-18T00:19:04.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0020000000000001e-06 2024-09-18T00:19:04.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622053+0000", 2024-09-18T00:19:04.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2679999999999997e-06 2024-09-18T00:19:04.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622062+0000", 2024-09-18T00:19:04.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3749999999999992e-06 2024-09-18T00:19:04.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622068+0000", 2024-09-18T00:19:04.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5729999999999998e-06 2024-09-18T00:19:04.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622118+0000", 2024-09-18T00:19:04.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0696999999999999e-05 2024-09-18T00:19:04.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622222+0000", 2024-09-18T00:19:04.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103286 2024-09-18T00:19:04.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:04.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697061+0000", 2024-09-18T00:19:04.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074839141999999997 2024-09-18T00:19:04.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:04.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697102+0000", 2024-09-18T00:19:04.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1513e-05 2024-09-18T00:19:04.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697164+0000", 2024-09-18T00:19:04.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1446999999999996e-05 2024-09-18T00:19:04.043 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.043 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.043 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.043 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19896 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:04.043 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:19:04.043 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.074555138999999, 2024-09-18T00:19:04.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097478370000000005, 2024-09-18T00:19:04.044 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.044 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.044 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:04.044 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.044 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.044 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.044 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19896 2024-09-18T00:19:04.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.044 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:19:04.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:19:04.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:19:04.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6000000000000002e-07 2024-09-18T00:19:04.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941434+0000", 2024-09-18T00:19:04.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4750000000000004e-06 2024-09-18T00:19:04.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941452+0000", 2024-09-18T00:19:04.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8603000000000001e-05 2024-09-18T00:19:04.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941461+0000", 2024-09-18T00:19:04.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5199999999999997e-06 2024-09-18T00:19:04.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.947391+0000", 2024-09-18T00:19:04.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059302399999999998 2024-09-18T00:19:04.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.947479+0000", 2024-09-18T00:19:04.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8002000000000002e-05 2024-09-18T00:19:04.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.951176+0000", 2024-09-18T00:19:04.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036971370000000001 2024-09-18T00:19:04.047 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.047 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.047 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.048 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19910 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:04.048 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941878+0000", 2024-09-18T00:19:04.048 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.07410541, 2024-09-18T00:19:04.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092861479999999993, 2024-09-18T00:19:04.048 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.048 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.048 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:04.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.048 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19910 2024-09-18T00:19:04.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.048 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941878+0000", 2024-09-18T00:19:04.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941878+0000", 2024-09-18T00:19:04.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941879+0000", 2024-09-18T00:19:04.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6199999999999998e-07 2024-09-18T00:19:04.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941881+0000", 2024-09-18T00:19:04.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6390000000000001e-06 2024-09-18T00:19:04.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941890+0000", 2024-09-18T00:19:04.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4999999999999999e-06 2024-09-18T00:19:04.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941896+0000", 2024-09-18T00:19:04.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1299999999999998e-06 2024-09-18T00:19:04.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950898+0000", 2024-09-18T00:19:04.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0090014499999999994 2024-09-18T00:19:04.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950984+0000", 2024-09-18T00:19:04.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6933000000000003e-05 2024-09-18T00:19:04.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.951164+0000", 2024-09-18T00:19:04.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000179934 2024-09-18T00:19:04.052 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.052 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.052 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.052 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20241 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:04.052 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155593+0000", 2024-09-18T00:19:04.053 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.8603902800000007, 2024-09-18T00:19:04.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0090827070000000006, 2024-09-18T00:19:04.053 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.053 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.053 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:04.053 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.053 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.053 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.053 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20241 2024-09-18T00:19:04.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.053 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155593+0000", 2024-09-18T00:19:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155593+0000", 2024-09-18T00:19:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155594+0000", 2024-09-18T00:19:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9700000000000002e-07 2024-09-18T00:19:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155596+0000", 2024-09-18T00:19:04.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1210000000000001e-06 2024-09-18T00:19:04.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155602+0000", 2024-09-18T00:19:04.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.144e-06 2024-09-18T00:19:04.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155607+0000", 2024-09-18T00:19:04.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7539999999999997e-06 2024-09-18T00:19:04.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164450+0000", 2024-09-18T00:19:04.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0088436079999999993 2024-09-18T00:19:04.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164537+0000", 2024-09-18T00:19:04.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6329000000000006e-05 2024-09-18T00:19:04.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164676+0000", 2024-09-18T00:19:04.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013935400000000001 2024-09-18T00:19:04.056 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.056 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20242 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155613+0000", 2024-09-18T00:19:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.8603706199999994, 2024-09-18T00:19:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00933551, 2024-09-18T00:19:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.057 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20242 2024-09-18T00:19:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155613+0000", 2024-09-18T00:19:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155613+0000", 2024-09-18T00:19:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155613+0000", 2024-09-18T00:19:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1e-07 2024-09-18T00:19:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155615+0000", 2024-09-18T00:19:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1110000000000002e-06 2024-09-18T00:19:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155621+0000", 2024-09-18T00:19:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:19:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155628+0000", 2024-09-18T00:19:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7259999999999996e-06 2024-09-18T00:19:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164725+0000", 2024-09-18T00:19:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0090965009999999999 2024-09-18T00:19:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164832+0000", 2024-09-18T00:19:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106956 2024-09-18T00:19:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164949+0000", 2024-09-18T00:19:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011703 2024-09-18T00:19:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20243 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155635+0000", 2024-09-18T00:19:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.8603485769999999, 2024-09-18T00:19:04.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.009569358, 2024-09-18T00:19:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20243 2024-09-18T00:19:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155635+0000", 2024-09-18T00:19:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155635+0000", 2024-09-18T00:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155635+0000", 2024-09-18T00:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8000000000000001e-07 2024-09-18T00:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155639+0000", 2024-09-18T00:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0809999999999998e-06 2024-09-18T00:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155645+0000", 2024-09-18T00:19:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0469999999999999e-06 2024-09-18T00:19:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155649+0000", 2024-09-18T00:19:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7419999999999997e-06 2024-09-18T00:19:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164990+0000", 2024-09-18T00:19:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093401370000000001 2024-09-18T00:19:04.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165066+0000", 2024-09-18T00:19:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6861999999999997e-05 2024-09-18T00:19:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165204+0000", 2024-09-18T00:19:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138109 2024-09-18T00:19:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20244 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155679+0000", 2024-09-18T00:19:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.8603046210000009, 2024-09-18T00:19:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097580089999999998, 2024-09-18T00:19:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20244 2024-09-18T00:19:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.066 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155679+0000", 2024-09-18T00:19:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155679+0000", 2024-09-18T00:19:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155680+0000", 2024-09-18T00:19:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0900000000000002e-07 2024-09-18T00:19:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155682+0000", 2024-09-18T00:19:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6589999999999999e-06 2024-09-18T00:19:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155690+0000", 2024-09-18T00:19:04.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3139999999999998e-06 2024-09-18T00:19:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155695+0000", 2024-09-18T00:19:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0950000000000003e-06 2024-09-18T00:19:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165243+0000", 2024-09-18T00:19:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095483519999999995 2024-09-18T00:19:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165321+0000", 2024-09-18T00:19:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8219000000000004e-05 2024-09-18T00:19:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165437+0000", 2024-09-18T00:19:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011586099999999999 2024-09-18T00:19:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20245 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:04.070 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.8602809479999998, 2024-09-18T00:19:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099981159999999996, 2024-09-18T00:19:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20245 2024-09-18T00:19:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.65e-07 2024-09-18T00:19:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155711+0000", 2024-09-18T00:19:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:19:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155720+0000", 2024-09-18T00:19:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6580000000000008e-06 2024-09-18T00:19:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155725+0000", 2024-09-18T00:19:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3430000000000003e-06 2024-09-18T00:19:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165479+0000", 2024-09-18T00:19:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097538239999999995 2024-09-18T00:19:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165556+0000", 2024-09-18T00:19:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6724000000000006e-05 2024-09-18T00:19:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165701+0000", 2024-09-18T00:19:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145129 2024-09-18T00:19:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20246 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.8602510849999998, 2024-09-18T00:19:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01018784, 2024-09-18T00:19:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20246 2024-09-18T00:19:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2599999999999998e-07 2024-09-18T00:19:04.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155735+0000", 2024-09-18T00:19:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2699999999999999e-06 2024-09-18T00:19:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155744+0000", 2024-09-18T00:19:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8629999999999997e-06 2024-09-18T00:19:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155749+0000", 2024-09-18T00:19:04.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0359999999999997e-06 2024-09-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165748+0000", 2024-09-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099991449999999992 2024-09-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165819+0000", 2024-09-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0964999999999996e-05 2024-09-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165920+0000", 2024-09-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101135 2024-09-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20247 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.8602277429999994, 2024-09-18T00:19:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010443987, 2024-09-18T00:19:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.079 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20247 2024-09-18T00:19:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1899999999999998e-07 2024-09-18T00:19:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155760+0000", 2024-09-18T00:19:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6459999999999999e-06 2024-09-18T00:19:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155767+0000", 2024-09-18T00:19:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0350000000000004e-06 2024-09-18T00:19:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155774+0000", 2024-09-18T00:19:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:19:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165960+0000", 2024-09-18T00:19:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186051999999999 2024-09-18T00:19:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166048+0000", 2024-09-18T00:19:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8848e-05 2024-09-18T00:19:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166200+0000", 2024-09-18T00:19:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015147000000000001 2024-09-18T00:19:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20248 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.8602025789999992, 2024-09-18T00:19:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010617721, 2024-09-18T00:19:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20248 2024-09-18T00:19:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155782+0000", 2024-09-18T00:19:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.84e-07 2024-09-18T00:19:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155784+0000", 2024-09-18T00:19:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2800000000000002e-06 2024-09-18T00:19:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155790+0000", 2024-09-18T00:19:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5679999999999996e-06 2024-09-18T00:19:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155795+0000", 2024-09-18T00:19:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6929999999999998e-06 2024-09-18T00:19:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166245+0000", 2024-09-18T00:19:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010449847999999999 2024-09-18T00:19:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166307+0000", 2024-09-18T00:19:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2287999999999997e-05 2024-09-18T00:19:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166399+0000", 2024-09-18T00:19:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.166e-05 2024-09-18T00:19:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20249 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.8601822030000008, 2024-09-18T00:19:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010811983000000001, 2024-09-18T00:19:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20249 2024-09-18T00:19:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155802+0000", 2024-09-18T00:19:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:19:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155804+0000", 2024-09-18T00:19:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:19:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155810+0000", 2024-09-18T00:19:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0580000000000003e-06 2024-09-18T00:19:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155817+0000", 2024-09-18T00:19:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3980000000000004e-06 2024-09-18T00:19:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166451+0000", 2024-09-18T00:19:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010634825000000001 2024-09-18T00:19:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166520+0000", 2024-09-18T00:19:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8317000000000003e-05 2024-09-18T00:19:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166613+0000", 2024-09-18T00:19:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3801000000000005e-05 2024-09-18T00:19:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:21913 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.7223370569999998, 2024-09-18T00:19:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090508916999999994, 2024-09-18T00:19:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 21913 2024-09-18T00:19:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293648+0000", 2024-09-18T00:19:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6400000000000001e-07 2024-09-18T00:19:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293653+0000", 2024-09-18T00:19:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3820000000000003e-06 2024-09-18T00:19:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293665+0000", 2024-09-18T00:19:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1778e-05 2024-09-18T00:19:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293673+0000", 2024-09-18T00:19:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4130000000000006e-06 2024-09-18T00:19:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293719+0000", 2024-09-18T00:19:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5534999999999997e-05 2024-09-18T00:19:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293773+0000", 2024-09-18T00:19:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4389999999999999e-05 2024-09-18T00:19:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384053+0000", 2024-09-18T00:19:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090279617000000006 2024-09-18T00:19:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384088+0000", 2024-09-18T00:19:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5466000000000002e-05 2024-09-18T00:19:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384156+0000", 2024-09-18T00:19:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7471999999999994e-05 2024-09-18T00:19:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22274 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.9112208370000001, 2024-09-18T00:19:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.069820742000000005, 2024-09-18T00:19:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22274 2024-09-18T00:19:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104766+0000", 2024-09-18T00:19:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3859999999999999e-06 2024-09-18T00:19:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104783+0000", 2024-09-18T00:19:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6697e-05 2024-09-18T00:19:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104816+0000", 2024-09-18T00:19:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2786000000000001e-05 2024-09-18T00:19:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104833+0000", 2024-09-18T00:19:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7125999999999998e-05 2024-09-18T00:19:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104965+0000", 2024-09-18T00:19:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013215899999999999 2024-09-18T00:19:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.105103+0000", 2024-09-18T00:19:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138193 2024-09-18T00:19:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174496+0000", 2024-09-18T00:19:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.06939244 2024-09-18T00:19:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174530+0000", 2024-09-18T00:19:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4601999999999998e-05 2024-09-18T00:19:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174584+0000", 2024-09-18T00:19:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3353000000000002e-05 2024-09-18T00:19:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22485 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write 16521958~30847] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.0624318270000002, 2024-09-18T00:19:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012574648000000001, 2024-09-18T00:19:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22485 2024-09-18T00:19:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953554+0000", 2024-09-18T00:19:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1150000000000001e-06 2024-09-18T00:19:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962036+0000", 2024-09-18T00:19:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0084817599999999996 2024-09-18T00:19:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962062+0000", 2024-09-18T00:19:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6499000000000002e-05 2024-09-18T00:19:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962074+0000", 2024-09-18T00:19:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2095e-05 2024-09-18T00:19:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962165+0000", 2024-09-18T00:19:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0703000000000002e-05 2024-09-18T00:19:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962276+0000", 2024-09-18T00:19:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111406 2024-09-18T00:19:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965925+0000", 2024-09-18T00:19:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036488919999999999 2024-09-18T00:19:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965992+0000", 2024-09-18T00:19:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6806999999999999e-05 2024-09-18T00:19:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.966127+0000", 2024-09-18T00:19:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013437099999999999 2024-09-18T00:19:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22681 3.4 3:22e76021:::rbd_data.112a948d5956.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.7284325220000003, 2024-09-18T00:19:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076591189000000004, 2024-09-18T00:19:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22681 2024-09-18T00:19:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287554+0000", 2024-09-18T00:19:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.514e-06 2024-09-18T00:19:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287567+0000", 2024-09-18T00:19:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-18T00:19:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287597+0000", 2024-09-18T00:19:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0406e-05 2024-09-18T00:19:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287626+0000", 2024-09-18T00:19:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9306e-05 2024-09-18T00:19:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287801+0000", 2024-09-18T00:19:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017411400000000001 2024-09-18T00:19:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.288053+0000", 2024-09-18T00:19:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025223500000000001 2024-09-18T00:19:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364016+0000", 2024-09-18T00:19:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075963301999999996 2024-09-18T00:19:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364064+0000", 2024-09-18T00:19:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7886000000000002e-05 2024-09-18T00:19:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364142+0000", 2024-09-18T00:19:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8345999999999998e-05 2024-09-18T00:19:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:04.112 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:04.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:19:04.114 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:04.114 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:04.114 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:04.114 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-18T00:19:04.115 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:04.115 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:04.115 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:04.115 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:04.116 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:04.116 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:04.116 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-18T00:19:04.116 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:04.116 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:04.117 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:04.110+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:04.133 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:04.134 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:04.137 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:04.137 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:04.214 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:04.214+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:04.250 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:04.250 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:04.264 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:19:04.315 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:04.314+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:04.381 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:04.381 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:04.396 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:19:04.416 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:04.414+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:04.448 INFO:teuthology.orchestra.run.smithi097.stdout:4105 writesame 0x2f1f1c1 thru 0x2f2031b data_size 0x1(0x115b bytes) 2024-09-18T00:19:04.461 INFO:teuthology.orchestra.run.smithi097.stdout:4107 punch from 0xa6bdc75 to 0xa6ca7d8, (0xcb63 bytes) 2024-09-18T00:19:04.461 INFO:teuthology.orchestra.run.smithi097.stdout:4108 read 0xa55ef60 thru 0xa566f71 (0x8012 bytes) 2024-09-18T00:19:04.461 INFO:teuthology.orchestra.run.smithi097.stdout:4109 write 0x3dbfed1 thru 0x3dc5deb (0x5f1b bytes) 2024-09-18T00:19:04.489 INFO:teuthology.orchestra.run.smithi097.stdout:4110 trunc from 0xd260b1b to 0x1bc4d09 2024-09-18T00:19:04.515 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:04.510+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:04.518 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:04.518 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:04.520 INFO:teuthology.orchestra.run.smithi097.stdout:4111 read 0x13c93a4 thru 0x13cbfdb (0x2c38 bytes) 2024-09-18T00:19:04.539 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:19:04.542 INFO:teuthology.orchestra.run.smithi097.stdout:4112 writesame 0xbc16d9e thru 0xbc1d6a0 data_size 0x57(0x6903 bytes) 2024-09-18T00:19:04.542 INFO:teuthology.orchestra.run.smithi097.stdout:4113 read 0xade826e thru 0xadf3959 (0xb6ec bytes) 2024-09-18T00:19:04.542 INFO:teuthology.orchestra.run.smithi097.stdout:4114 read 0x3f1739f thru 0x3f1b9ff (0x4661 bytes) 2024-09-18T00:19:04.542 INFO:teuthology.orchestra.run.smithi097.stdout:4115 read 0x7040280 thru 0x7047b3f (0x78c0 bytes) 2024-09-18T00:19:04.543 INFO:teuthology.orchestra.run.smithi097.stdout:4118 write 0xa76f3f7 thru 0xa76f7d8 (0x3e2 bytes) 2024-09-18T00:19:04.549 INFO:teuthology.orchestra.run.smithi097.stdout:4119 punch from 0x5dd76bd to 0x5dda585, (0x2ec8 bytes) 2024-09-18T00:19:04.549 INFO:teuthology.orchestra.run.smithi097.stdout:4120 punch from 0x7d75272 to 0x7d7a06a, (0x4df8 bytes) 2024-09-18T00:19:04.549 INFO:teuthology.orchestra.run.smithi097.stdout:4121 write 0x5959d7c thru 0x5967dcb (0xe050 bytes) 2024-09-18T00:19:04.560 INFO:teuthology.orchestra.run.smithi097.stdout:4123 trunc from 0xbc1d6a1 to 0xd5e33b0 2024-09-18T00:19:04.569 INFO:teuthology.orchestra.run.smithi097.stdout:4124 clone 18 order 23 su 262144 sc 9 2024-09-18T00:19:04.616 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:04.614+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:04.669 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:04.669 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:04.683 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:19:04.717 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:04.714+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:04.818 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:04.818+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:04.822 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is set 2024-09-18T00:19:04.850 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:04.850 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:04.865 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:19:04.918 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:04.914+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:04.922 INFO:teuthology.orchestra.run.smithi097.stdout:flattening image image_client.0-clone17 2024-09-18T00:19:05.019 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:05.018+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:05.030 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:05.030 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:05.045 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:19:05.130 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:05.130+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:05.211 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:05.211 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:05.221 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:05.218+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:05.226 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:19:05.321 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:05.322+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:05.391 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:05.392 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:05.406 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:19:05.450 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:05.450+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:05.523 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:05.523 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:05.523 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:05.518+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:05.536 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:19:05.624 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:05.617+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:05.703 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:05.703 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:05.717 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:19:05.725 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:05.721+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:05.826 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:05.821+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:05.841 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:05.841 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:05.856 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:19:05.927 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:05.921+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:05.977 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:05.977 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:05.993 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:19:06.028 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:06.026+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:06.100 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-18T00:19:06.122 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:06.122 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:06.128 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:06.126+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:06.139 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:19:06.140 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:19:06.229 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:06.225+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:06.311 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:06.311 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:06.330 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:06.330+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:06.335 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:06.342 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:06.361 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:06.362 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:06.379 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:06.379 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:06.399 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:06.400 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:06.400 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-18T00:19:06.421 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:06.421 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:06.431 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:06.425+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:06.442 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:06.445 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:06.445 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:06.464 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:06.464 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:06.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:06.465 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:06.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.465 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2230 3.17 3:ef492888:::rbd_data.112aa7ba4578.000000000000014f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:06.465 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:06.465 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.196712097999999, 2024-09-18T00:19:06.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075065283999999996, 2024-09-18T00:19:06.465 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.465 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.465 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:06.465 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:06.465 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:06.465 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:06.465 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2230 2024-09-18T00:19:06.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.466 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:06.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:06.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266014+0000", 2024-09-18T00:19:06.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:19:06.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266029+0000", 2024-09-18T00:19:06.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4616e-05 2024-09-18T00:19:06.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266053+0000", 2024-09-18T00:19:06.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4369999999999999e-05 2024-09-18T00:19:06.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266067+0000", 2024-09-18T00:19:06.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4243999999999999e-05 2024-09-18T00:19:06.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266157+0000", 2024-09-18T00:19:06.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9909999999999998e-05 2024-09-18T00:19:06.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266327+0000", 2024-09-18T00:19:06.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016981200000000001 2024-09-18T00:19:06.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:19:06.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269180+0000", 2024-09-18T00:19:06.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028526559999999999 2024-09-18T00:19:06.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:06.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340955+0000", 2024-09-18T00:19:06.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.071774904 2024-09-18T00:19:06.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:06.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340974+0000", 2024-09-18T00:19:06.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8824000000000001e-05 2024-09-18T00:19:06.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:06.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341028+0000", 2024-09-18T00:19:06.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4456999999999997e-05 2024-09-18T00:19:06.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341076+0000", 2024-09-18T00:19:06.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8284000000000001e-05 2024-09-18T00:19:06.470 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.470 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.471 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.471 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2230 3.17 e36/23 ondisk, result = 0)", 2024-09-18T00:19:06.471 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:06.471 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.179368517, 2024-09-18T00:19:06.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057609645000000001, 2024-09-18T00:19:06.472 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.472 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.472 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:06.472 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:06.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:06.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283356+0000", 2024-09-18T00:19:06.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.44e-06 2024-09-18T00:19:06.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283362+0000", 2024-09-18T00:19:06.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2539999999999998e-06 2024-09-18T00:19:06.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283382+0000", 2024-09-18T00:19:06.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9896000000000001e-05 2024-09-18T00:19:06.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283398+0000", 2024-09-18T00:19:06.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5489999999999999e-05 2024-09-18T00:19:06.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340936+0000", 2024-09-18T00:19:06.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057538649999999997 2024-09-18T00:19:06.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340947+0000", 2024-09-18T00:19:06.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0981e-05 2024-09-18T00:19:06.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.475 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.475 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340964+0000", 2024-09-18T00:19:06.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6934000000000001e-05 2024-09-18T00:19:06.475 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.476 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.476 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.476 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2419 3.17 3:ed94b133:::rbd_data.112aa7ba4578.00000000000001b5:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:19:06.476 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:06.476 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.725247044, 2024-09-18T00:19:06.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.064761812000000002, 2024-09-18T00:19:06.476 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.476 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.476 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:06.476 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:06.476 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:06.477 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:06.477 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2419 2024-09-18T00:19:06.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.477 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:06.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:06.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737479+0000", 2024-09-18T00:19:06.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6089999999999999e-06 2024-09-18T00:19:06.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737487+0000", 2024-09-18T00:19:06.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7900000000000005e-06 2024-09-18T00:19:06.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737507+0000", 2024-09-18T00:19:06.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9936e-05 2024-09-18T00:19:06.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737518+0000", 2024-09-18T00:19:06.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1046999999999999e-05 2024-09-18T00:19:06.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737599+0000", 2024-09-18T00:19:06.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0697000000000003e-05 2024-09-18T00:19:06.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737697+0000", 2024-09-18T00:19:06.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8058000000000002e-05 2024-09-18T00:19:06.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:19:06.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.740445+0000", 2024-09-18T00:19:06.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027481279999999999 2024-09-18T00:19:06.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:06.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802023+0000", 2024-09-18T00:19:06.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.061577321999999997 2024-09-18T00:19:06.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.481 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:06.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802060+0000", 2024-09-18T00:19:06.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7586000000000002e-05 2024-09-18T00:19:06.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:06.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802158+0000", 2024-09-18T00:19:06.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7583000000000004e-05 2024-09-18T00:19:06.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802238+0000", 2024-09-18T00:19:06.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0055999999999994e-05 2024-09-18T00:19:06.482 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.482 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.483 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.483 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2419 3.17 e37/23 ondisk, result = 0)", 2024-09-18T00:19:06.483 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:06.483 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.713022098000003, 2024-09-18T00:19:06.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052342948, 2024-09-18T00:19:06.483 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.483 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.483 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:06.483 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:06.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:06.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749702+0000", 2024-09-18T00:19:06.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:19:06.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749710+0000", 2024-09-18T00:19:06.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7279999999999995e-06 2024-09-18T00:19:06.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749736+0000", 2024-09-18T00:19:06.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5797999999999999e-05 2024-09-18T00:19:06.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749751+0000", 2024-09-18T00:19:06.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5238999999999999e-05 2024-09-18T00:19:06.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801745+0000", 2024-09-18T00:19:06.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051994022000000001 2024-09-18T00:19:06.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801770+0000", 2024-09-18T00:19:06.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5103e-05 2024-09-18T00:19:06.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802044+0000", 2024-09-18T00:19:06.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027395799999999998 2024-09-18T00:19:06.486 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.486 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.486 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.487 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6503 3.1b 3:d9387a1f:::rbd_data.112a3a39fc45.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:06.487 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:06.487 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 40.898349128, 2024-09-18T00:19:06.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050510547000000003, 2024-09-18T00:19:06.487 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.487 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.487 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:06.487 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:06.487 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:06.487 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:06.487 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6503 2024-09-18T00:19:06.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.487 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:06.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.488 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:06.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564376+0000", 2024-09-18T00:19:06.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-06 2024-09-18T00:19:06.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564386+0000", 2024-09-18T00:19:06.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0312999999999999e-05 2024-09-18T00:19:06.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564408+0000", 2024-09-18T00:19:06.489 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2001000000000002e-05 2024-09-18T00:19:06.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564421+0000", 2024-09-18T00:19:06.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2271e-05 2024-09-18T00:19:06.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564491+0000", 2024-09-18T00:19:06.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0630000000000006e-05 2024-09-18T00:19:06.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564606+0000", 2024-09-18T00:19:06.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011449 2024-09-18T00:19:06.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:06.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.566743+0000", 2024-09-18T00:19:06.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021373830000000001 2024-09-18T00:19:06.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:06.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614738+0000", 2024-09-18T00:19:06.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047995359000000001 2024-09-18T00:19:06.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:06.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614759+0000", 2024-09-18T00:19:06.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1010999999999999e-05 2024-09-18T00:19:06.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:06.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614826+0000", 2024-09-18T00:19:06.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6654000000000003e-05 2024-09-18T00:19:06.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614884+0000", 2024-09-18T00:19:06.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8332999999999999e-05 2024-09-18T00:19:06.493 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.493 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.493 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.494 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6503 3.1b e42/23 ondisk, result = 0)", 2024-09-18T00:19:06.494 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:19:06.494 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 40.889089660000003, 2024-09-18T00:19:06.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041115407, 2024-09-18T00:19:06.494 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.494 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.494 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:06.494 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:19:06.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:19:06.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573635+0000", 2024-09-18T00:19:06.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5689999999999999e-06 2024-09-18T00:19:06.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573643+0000", 2024-09-18T00:19:06.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9699999999999999e-06 2024-09-18T00:19:06.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573668+0000", 2024-09-18T00:19:06.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5536000000000001e-05 2024-09-18T00:19:06.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.496 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.496 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573683+0000", 2024-09-18T00:19:06.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4025e-05 2024-09-18T00:19:06.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614717+0000", 2024-09-18T00:19:06.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041034517 2024-09-18T00:19:06.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614730+0000", 2024-09-18T00:19:06.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2945999999999999e-05 2024-09-18T00:19:06.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614749+0000", 2024-09-18T00:19:06.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8844000000000001e-05 2024-09-18T00:19:06.497 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.497 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.498 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.498 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6518 3.1 3:8061bf9f:::rbd_data.112a3a39fc45.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:06.498 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:06.498 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 40.811910853999997, 2024-09-18T00:19:06.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062303123000000002, 2024-09-18T00:19:06.498 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.498 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.498 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:06.498 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:06.498 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:06.498 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:06.498 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6518 2024-09-18T00:19:06.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.499 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:06.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:06.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650814+0000", 2024-09-18T00:19:06.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.629e-06 2024-09-18T00:19:06.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650823+0000", 2024-09-18T00:19:06.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1800000000000002e-06 2024-09-18T00:19:06.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650843+0000", 2024-09-18T00:19:06.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9836000000000001e-05 2024-09-18T00:19:06.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650854+0000", 2024-09-18T00:19:06.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1564e-05 2024-09-18T00:19:06.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650917+0000", 2024-09-18T00:19:06.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3076000000000004e-05 2024-09-18T00:19:06.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.651029+0000", 2024-09-18T00:19:06.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111154 2024-09-18T00:19:06.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:06.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.655531+0000", 2024-09-18T00:19:06.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045027629999999999 2024-09-18T00:19:06.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:06.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712924+0000", 2024-09-18T00:19:06.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057393006000000003 2024-09-18T00:19:06.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:06.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712988+0000", 2024-09-18T00:19:06.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000005e-05 2024-09-18T00:19:06.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:06.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713056+0000", 2024-09-18T00:19:06.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7904999999999994e-05 2024-09-18T00:19:06.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713115+0000", 2024-09-18T00:19:06.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9339999999999998e-05 2024-09-18T00:19:06.503 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.503 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.503 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.504 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6518 3.1 e42/23 ondisk, result = 0)", 2024-09-18T00:19:06.504 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:06.504 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 40.799485771000001, 2024-09-18T00:19:06.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049887304, 2024-09-18T00:19:06.504 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.504 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.504 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:06.504 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:06.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:06.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663239+0000", 2024-09-18T00:19:06.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:19:06.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663246+0000", 2024-09-18T00:19:06.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4189999999999996e-06 2024-09-18T00:19:06.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663265+0000", 2024-09-18T00:19:06.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8323000000000001e-05 2024-09-18T00:19:06.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663277+0000", 2024-09-18T00:19:06.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2065999999999999e-05 2024-09-18T00:19:06.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712965+0000", 2024-09-18T00:19:06.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049688317000000003 2024-09-18T00:19:06.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712977+0000", 2024-09-18T00:19:06.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.206e-05 2024-09-18T00:19:06.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713125+0000", 2024-09-18T00:19:06.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014749899999999999 2024-09-18T00:19:06.507 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.507 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.507 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.508 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:8831 3.12 3:49cac8cf:::rbd_data.112a9f4259df.0000000000000130:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T00:19:06.508 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:06.508 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 34.181625521999997, 2024-09-18T00:19:06.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104125047, 2024-09-18T00:19:06.508 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.508 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.508 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:06.508 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:06.508 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:06.508 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:06.508 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 8831 2024-09-18T00:19:06.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.508 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:06.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:06.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281099+0000", 2024-09-18T00:19:06.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.029e-06 2024-09-18T00:19:06.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281103+0000", 2024-09-18T00:19:06.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7759999999999997e-06 2024-09-18T00:19:06.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281114+0000", 2024-09-18T00:19:06.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1090000000000001e-05 2024-09-18T00:19:06.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281120+0000", 2024-09-18T00:19:06.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.604e-06 2024-09-18T00:19:06.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281164+0000", 2024-09-18T00:19:06.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3696999999999999e-05 2024-09-18T00:19:06.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281270+0000", 2024-09-18T00:19:06.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106041 2024-09-18T00:19:06.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:06.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.285596+0000", 2024-09-18T00:19:06.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043266659999999998 2024-09-18T00:19:06.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:06.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385028+0000", 2024-09-18T00:19:06.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.099431808999999996 2024-09-18T00:19:06.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:06.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385094+0000", 2024-09-18T00:19:06.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5740000000000004e-05 2024-09-18T00:19:06.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:06.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385166+0000", 2024-09-18T00:19:06.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1637000000000005e-05 2024-09-18T00:19:06.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385223+0000", 2024-09-18T00:19:06.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6957999999999999e-05 2024-09-18T00:19:06.514 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.514 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.514 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.514 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:8831 3.12 e48/32 ondisk, result = 0)", 2024-09-18T00:19:06.515 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:06.515 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 34.165111412000002, 2024-09-18T00:19:06.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087621558000000002, 2024-09-18T00:19:06.515 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.515 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.515 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:06.515 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:06.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:06.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297613+0000", 2024-09-18T00:19:06.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3400000000000001e-06 2024-09-18T00:19:06.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297620+0000", 2024-09-18T00:19:06.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5869999999999998e-06 2024-09-18T00:19:06.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297639+0000", 2024-09-18T00:19:06.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9888999999999999e-05 2024-09-18T00:19:06.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297661+0000", 2024-09-18T00:19:06.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1772000000000001e-05 2024-09-18T00:19:06.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385073+0000", 2024-09-18T00:19:06.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087411303999999995 2024-09-18T00:19:06.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385086+0000", 2024-09-18T00:19:06.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3040000000000001e-05 2024-09-18T00:19:06.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385233+0000", 2024-09-18T00:19:06.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014762600000000001 2024-09-18T00:19:06.519 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.519 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.519 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.519 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10109 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:19:06.519 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:19:06.519 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.889735852000001, 2024-09-18T00:19:06.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026513858000000001, 2024-09-18T00:19:06.520 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.520 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.520 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:06.520 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:06.520 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:06.520 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:06.520 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10109 2024-09-18T00:19:06.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.520 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:19:06.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572987+0000", 2024-09-18T00:19:06.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572988+0000", 2024-09-18T00:19:06.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1099999999999996e-07 2024-09-18T00:19:06.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572990+0000", 2024-09-18T00:19:06.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3329999999999999e-06 2024-09-18T00:19:06.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.572998+0000", 2024-09-18T00:19:06.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2039999999999997e-06 2024-09-18T00:19:06.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573012+0000", 2024-09-18T00:19:06.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3879e-05 2024-09-18T00:19:06.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599312+0000", 2024-09-18T00:19:06.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026300093 2024-09-18T00:19:06.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599380+0000", 2024-09-18T00:19:06.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7401000000000001e-05 2024-09-18T00:19:06.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599501+0000", 2024-09-18T00:19:06.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000121437 2024-09-18T00:19:06.523 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.523 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.523 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.524 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10110 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:19:06.524 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:19:06.524 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.889702393, 2024-09-18T00:19:06.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026746066999999998, 2024-09-18T00:19:06.524 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.524 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.524 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:06.524 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:06.524 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:06.524 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:06.524 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10110 2024-09-18T00:19:06.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.524 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:19:06.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:19:06.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573021+0000", 2024-09-18T00:19:06.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2799999999999996e-07 2024-09-18T00:19:06.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573030+0000", 2024-09-18T00:19:06.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5469999999999997e-06 2024-09-18T00:19:06.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573039+0000", 2024-09-18T00:19:06.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6080000000000002e-06 2024-09-18T00:19:06.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573049+0000", 2024-09-18T00:19:06.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0759999999999992e-06 2024-09-18T00:19:06.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599539+0000", 2024-09-18T00:19:06.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026490952000000002 2024-09-18T00:19:06.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599621+0000", 2024-09-18T00:19:06.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1601999999999996e-05 2024-09-18T00:19:06.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599767+0000", 2024-09-18T00:19:06.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145654 2024-09-18T00:19:06.531 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.531 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.531 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.532 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10111 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:19:06.532 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:19:06.532 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.889664177999997, 2024-09-18T00:19:06.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027006651999999999, 2024-09-18T00:19:06.532 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.533 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.533 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:06.533 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:06.533 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:06.533 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:06.533 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10111 2024-09-18T00:19:06.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.533 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:19:06.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:19:06.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573059+0000", 2024-09-18T00:19:06.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9500000000000002e-07 2024-09-18T00:19:06.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573062+0000", 2024-09-18T00:19:06.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6539999999999999e-06 2024-09-18T00:19:06.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573071+0000", 2024-09-18T00:19:06.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5380000000000002e-06 2024-09-18T00:19:06.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573077+0000", 2024-09-18T00:19:06.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1879999999999999e-06 2024-09-18T00:19:06.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599818+0000", 2024-09-18T00:19:06.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02674093 2024-09-18T00:19:06.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.599923+0000", 2024-09-18T00:19:06.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010490200000000001 2024-09-18T00:19:06.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600066+0000", 2024-09-18T00:19:06.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014284500000000001 2024-09-18T00:19:06.540 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.540 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.540 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.540 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10112 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:19:06.541 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:19:06.541 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.889638384000001, 2024-09-18T00:19:06.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027148835, 2024-09-18T00:19:06.541 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.541 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.542 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:06.542 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:06.542 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:06.542 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:06.542 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10112 2024-09-18T00:19:06.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.543 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:19:06.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:19:06.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573085+0000", 2024-09-18T00:19:06.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9500000000000003e-07 2024-09-18T00:19:06.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573088+0000", 2024-09-18T00:19:06.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9569999999999998e-06 2024-09-18T00:19:06.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573097+0000", 2024-09-18T00:19:06.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8599999999999999e-06 2024-09-18T00:19:06.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573104+0000", 2024-09-18T00:19:06.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5710000000000003e-06 2024-09-18T00:19:06.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600091+0000", 2024-09-18T00:19:06.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.026987315000000001 2024-09-18T00:19:06.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600158+0000", 2024-09-18T00:19:06.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6727e-05 2024-09-18T00:19:06.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600234+0000", 2024-09-18T00:19:06.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5909999999999997e-05 2024-09-18T00:19:06.547 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.547 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.547 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.547 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10113 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:19:06.547 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:19:06.547 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.889594125999999, 2024-09-18T00:19:06.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027356020000000002, 2024-09-18T00:19:06.547 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.547 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.547 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:06.547 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:06.547 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:06.547 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:06.548 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10113 2024-09-18T00:19:06.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.548 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:19:06.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573129+0000", 2024-09-18T00:19:06.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573130+0000", 2024-09-18T00:19:06.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1999999999999999e-07 2024-09-18T00:19:06.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573133+0000", 2024-09-18T00:19:06.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:19:06.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573141+0000", 2024-09-18T00:19:06.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5110000000000003e-06 2024-09-18T00:19:06.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573155+0000", 2024-09-18T00:19:06.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3730000000000001e-05 2024-09-18T00:19:06.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600275+0000", 2024-09-18T00:19:06.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027119621999999999 2024-09-18T00:19:06.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600349+0000", 2024-09-18T00:19:06.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4624999999999998e-05 2024-09-18T00:19:06.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600485+0000", 2024-09-18T00:19:06.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013570499999999999 2024-09-18T00:19:06.551 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.551 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.551 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.551 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10114 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:19:06.552 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:19:06.552 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.889558762999997, 2024-09-18T00:19:06.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027471173000000002, 2024-09-18T00:19:06.552 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.552 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.552 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:06.552 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:06.552 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:06.552 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:06.552 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10114 2024-09-18T00:19:06.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.552 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:19:06.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573164+0000", 2024-09-18T00:19:06.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573165+0000", 2024-09-18T00:19:06.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.99e-07 2024-09-18T00:19:06.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573168+0000", 2024-09-18T00:19:06.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.8339999999999999e-06 2024-09-18T00:19:06.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573176+0000", 2024-09-18T00:19:06.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0199999999999994e-06 2024-09-18T00:19:06.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573182+0000", 2024-09-18T00:19:06.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.455e-06 2024-09-18T00:19:06.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600521+0000", 2024-09-18T00:19:06.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027338989000000001 2024-09-18T00:19:06.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600577+0000", 2024-09-18T00:19:06.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5717000000000002e-05 2024-09-18T00:19:06.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.556 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600635+0000", 2024-09-18T00:19:06.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8559e-05 2024-09-18T00:19:06.556 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.556 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.556 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.557 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:10115 3.6 3:67857717:::rbd_data.112a9f4259df.0000000000000004:8 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-18T00:19:06.557 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:19:06.557 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.889505886000002, 2024-09-18T00:19:06.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027643300999999999, 2024-09-18T00:19:06.557 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.557 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.557 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:06.557 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:06.557 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:06.557 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:06.557 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 10115 2024-09-18T00:19:06.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.557 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:19:06.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573217+0000", 2024-09-18T00:19:06.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573218+0000", 2024-09-18T00:19:06.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7999999999999995e-07 2024-09-18T00:19:06.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573221+0000", 2024-09-18T00:19:06.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3079999999999999e-06 2024-09-18T00:19:06.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573233+0000", 2024-09-18T00:19:06.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1544e-05 2024-09-18T00:19:06.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.573240+0000", 2024-09-18T00:19:06.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4070000000000004e-06 2024-09-18T00:19:06.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600665+0000", 2024-09-18T00:19:06.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.02742522 2024-09-18T00:19:06.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600745+0000", 2024-09-18T00:19:06.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0014999999999999e-05 2024-09-18T00:19:06.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:33.600860+0000", 2024-09-18T00:19:06.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000115127 2024-09-18T00:19:06.561 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.561 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.561 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.561 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11459 3.1b 3:db862ee8:::rbd_data.112ab1fa102a.0000000000000152:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:06.561 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:19:06.561 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.201077075000001, 2024-09-18T00:19:06.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.032246431999999998, 2024-09-18T00:19:06.561 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.561 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.561 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:06.561 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:06.562 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:06.562 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:06.562 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11459 2024-09-18T00:19:06.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.562 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:19:06.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.562 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.562 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261646+0000", 2024-09-18T00:19:06.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261649+0000", 2024-09-18T00:19:06.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1200000000000002e-06 2024-09-18T00:19:06.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261665+0000", 2024-09-18T00:19:06.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5668999999999999e-05 2024-09-18T00:19:06.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261700+0000", 2024-09-18T00:19:06.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4925000000000002e-05 2024-09-18T00:19:06.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261723+0000", 2024-09-18T00:19:06.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3189000000000001e-05 2024-09-18T00:19:06.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261815+0000", 2024-09-18T00:19:06.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1662000000000004e-05 2024-09-18T00:19:06.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.261904+0000", 2024-09-18T00:19:06.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9201000000000002e-05 2024-09-18T00:19:06.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:06.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.262955+0000", 2024-09-18T00:19:06.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00105156 2024-09-18T00:19:06.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:06.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293760+0000", 2024-09-18T00:19:06.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030804292 2024-09-18T00:19:06.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:06.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293773+0000", 2024-09-18T00:19:06.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3361e-05 2024-09-18T00:19:06.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:06.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293833+0000", 2024-09-18T00:19:06.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9626000000000001e-05 2024-09-18T00:19:06.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.293892+0000", 2024-09-18T00:19:06.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9827000000000002e-05 2024-09-18T00:19:06.567 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.567 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.567 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.567 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22269 3.17 3:e9a1eb35:::rbd_data.112a948d5956.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:06.567 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:06.567 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.5535795859999997, 2024-09-18T00:19:06.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.063462693000000001, 2024-09-18T00:19:06.567 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.567 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.568 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:06.568 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:06.568 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:06.568 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:06.568 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22269 2024-09-18T00:19:06.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.568 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:06.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:06.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909146+0000", 2024-09-18T00:19:06.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3700000000000002e-06 2024-09-18T00:19:06.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909160+0000", 2024-09-18T00:19:06.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3825000000000001e-05 2024-09-18T00:19:06.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909178+0000", 2024-09-18T00:19:06.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8567999999999999e-05 2024-09-18T00:19:06.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909189+0000", 2024-09-18T00:19:06.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0881e-05 2024-09-18T00:19:06.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909270+0000", 2024-09-18T00:19:06.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0846999999999994e-05 2024-09-18T00:19:06.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909383+0000", 2024-09-18T00:19:06.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112704 2024-09-18T00:19:06.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:06.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972310+0000", 2024-09-18T00:19:06.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062927081999999995 2024-09-18T00:19:06.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:06.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972433+0000", 2024-09-18T00:19:06.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-18T00:19:06.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.572 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972606+0000", 2024-09-18T00:19:06.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017317799999999999 2024-09-18T00:19:06.572 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.572 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.572 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.572 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22379 3.17 3:e9a1eb35:::rbd_data.112a948d5956.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:06.572 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:06.572 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0595042889999995, 2024-09-18T00:19:06.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.12560279099999999, 2024-09-18T00:19:06.572 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:06.573 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:06.573 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:06.573 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:06.573 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:06.573 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:06.573 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22379 2024-09-18T00:19:06.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.573 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:06.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:06.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:06.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:06.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:06.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:06.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:06.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403221+0000", 2024-09-18T00:19:06.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3269999999999999e-06 2024-09-18T00:19:06.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:06.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403232+0000", 2024-09-18T00:19:06.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1392e-05 2024-09-18T00:19:06.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:06.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403264+0000", 2024-09-18T00:19:06.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1146999999999998e-05 2024-09-18T00:19:06.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:06.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403279+0000", 2024-09-18T00:19:06.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5109000000000001e-05 2024-09-18T00:19:06.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:06.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403424+0000", 2024-09-18T00:19:06.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014516999999999999 2024-09-18T00:19:06.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:06.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403567+0000", 2024-09-18T00:19:06.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142824 2024-09-18T00:19:06.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:06.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528723+0000", 2024-09-18T00:19:06.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.12515647299999999 2024-09-18T00:19:06.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:06.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528764+0000", 2024-09-18T00:19:06.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0624000000000003e-05 2024-09-18T00:19:06.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:06.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:06.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:06.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528822+0000", 2024-09-18T00:19:06.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7725e-05 2024-09-18T00:19:06.578 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.578 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.578 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.578 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:06.578 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:06.578 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:06.581 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:19:06.583 INFO:teuthology.orchestra.run.smithi097.stdout:4126 trunc from 0xd5e33b0 to 0x84721c4 2024-09-18T00:19:06.583 INFO:teuthology.orchestra.run.smithi097.stdout:4127 trunc from 0x84721c4 to 0x64df392 2024-09-18T00:19:06.583 INFO:teuthology.orchestra.run.smithi097.stdout:4128 read 0x364c314 thru 0x3657afa (0xb7e7 bytes) 2024-09-18T00:19:06.583 INFO:teuthology.orchestra.run.smithi097.stdout:4130 punch from 0x159fb2d to 0x15a4f0a, (0x53dd bytes) 2024-09-18T00:19:06.584 INFO:teuthology.orchestra.run.smithi097.stdout:4132 read 0x27b6c06 thru 0x27ba89f (0x3c9a bytes) 2024-09-18T00:19:06.584 INFO:teuthology.orchestra.run.smithi097.stdout:4133 write 0x5e3679c thru 0x5e3c8b9 (0x611e bytes) 2024-09-18T00:19:06.584 INFO:teuthology.orchestra.run.smithi097.stdout:4135 read 0x566f90f thru 0x5676e02 (0x74f4 bytes) 2024-09-18T00:19:06.584 INFO:teuthology.orchestra.run.smithi097.stdout:4138 read 0x17b052b thru 0x17c023f (0xfd15 bytes) 2024-09-18T00:19:06.584 INFO:teuthology.orchestra.run.smithi097.stdout:4139 read 0x3fbcffc thru 0x3fc48c7 (0x78cc bytes) 2024-09-18T00:19:06.584 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:06.585 INFO:teuthology.orchestra.run.smithi097.stdout:4142 write 0x44c29dd thru 0x44c7852 (0x4e76 bytes) 2024-09-18T00:19:06.585 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:06.578+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:06.605 INFO:teuthology.orchestra.run.smithi097.stdout:4143 write 0xa87788c thru 0xa8800a4 (0x8819 bytes) 2024-09-18T00:19:06.613 INFO:teuthology.orchestra.run.smithi097.stdout:4144 punch from 0x4e7edac to 0x4e891b3, (0xa407 bytes) 2024-09-18T00:19:06.613 INFO:teuthology.orchestra.run.smithi097.stdout:4145 write 0x82d223 thru 0x83351b (0x62f9 bytes) 2024-09-18T00:19:06.636 INFO:teuthology.orchestra.run.smithi097.stdout:4147 writesame 0xcb3a25c thru 0xcb40f4d data_size 0x5(0x6cf2 bytes) 2024-09-18T00:19:06.636 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:06.636 INFO:teuthology.orchestra.run.smithi097.stdout:4150 punch from 0x763e5a9 to 0x764369b, (0x50f2 bytes) 2024-09-18T00:19:06.636 INFO:teuthology.orchestra.run.smithi097.stdout:4151 punch from 0x342ea00 to 0x34345b0, (0x5bb0 bytes) 2024-09-18T00:19:06.636 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:06.636 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:06.637 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:06.637 INFO:teuthology.orchestra.run.smithi097.stdout:4155 writesame 0x6d314af thru 0x6d3f5e8 data_size 0x2(0xe13a bytes) 2024-09-18T00:19:06.682 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:06.678+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:06.729 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:06.729 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:06.729 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:06.729 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:06.743 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:19:06.744 INFO:teuthology.orchestra.run.smithi097.stdout:4156 write 0x36b52fb thru 0x36b99db (0x46e1 bytes) 2024-09-18T00:19:06.763 INFO:teuthology.orchestra.run.smithi097.stdout:4157 read 0x33a6ce7 thru 0x33b475e (0xda78 bytes) 2024-09-18T00:19:06.763 INFO:teuthology.orchestra.run.smithi097.stdout:4158 writesame 0x72437f thru 0x730d8f data_size 0x3(0xca11 bytes) 2024-09-18T00:19:06.782 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:06.777+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:06.795 INFO:teuthology.orchestra.run.smithi097.stdout:4161 write 0x7ab3aef thru 0x7ab68c0 (0x2dd2 bytes) 2024-09-18T00:19:06.815 INFO:teuthology.orchestra.run.smithi097.stdout:4163 trunc from 0xcb40f4e to 0x126b772 2024-09-18T00:19:06.819 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7] out_osds: [6] dead_osds: [1, 5] live_osds: [0, 2, 3, 4, 6, 7] 2024-09-18T00:19:06.819 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:19:06.820 INFO:tasks.thrashosds.thrasher:Reviving osd 5 2024-09-18T00:19:06.820 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-18T00:19:06.820 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-18T00:19:06.823 INFO:tasks.ceph.osd.5:Started 2024-09-18T00:19:06.823 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:19:06.848 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:19:06.859 INFO:teuthology.orchestra.run.smithi097.stdout:4165 punch from 0xebb637 to 0xec7fa7, (0xc970 bytes) 2024-09-18T00:19:06.859 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:06.859 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:06.859 INFO:teuthology.orchestra.run.smithi097.stdout:4170 punch from 0x8c3257 to 0x8cd9d7, (0xa780 bytes) 2024-09-18T00:19:06.877 INFO:teuthology.orchestra.run.smithi097.stdout:4172 write 0x7df444e thru 0x7dfffd0 (0xbb83 bytes) 2024-09-18T00:19:06.881 INFO:teuthology.orchestra.run.smithi097.stdout:4173 read 0x36d637b thru 0x36e03ea (0xa070 bytes) 2024-09-18T00:19:06.881 INFO:teuthology.orchestra.run.smithi097.stdout:4174 write 0x45cfd34 thru 0x45d3d15 (0x3fe2 bytes) 2024-09-18T00:19:06.882 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:06.882+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:06.884 INFO:teuthology.orchestra.run.smithi097.stdout:4177 punch from 0x14ea931 to 0x14f815a, (0xd829 bytes) 2024-09-18T00:19:06.884 INFO:teuthology.orchestra.run.smithi097.stdout:4178 read 0x19465e6 thru 0x194f070 (0x8a8b bytes) 2024-09-18T00:19:06.885 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:06.885 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:06.886 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:06.886 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:06.886 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:06.891 INFO:teuthology.orchestra.run.smithi097.stdout:4179 write 0x85cf5bc thru 0x85d32b7 (0x3cfc bytes) 2024-09-18T00:19:06.894 INFO:teuthology.orchestra.run.smithi097.stdout:4180 writesame 0x745e15c thru 0x74617c7 data_size 0x24(0x366c bytes) 2024-09-18T00:19:06.900 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:19:06.901 INFO:teuthology.orchestra.run.smithi097.stdout:4181 trunc from 0x85d32b8 to 0x6c52096 2024-09-18T00:19:06.907 INFO:teuthology.orchestra.run.smithi097.stdout:4182 write 0x6ffbbf thru 0x705c18 (0x605a bytes) 2024-09-18T00:19:06.909 INFO:teuthology.orchestra.run.smithi097.stdout:4185 read 0x2f5f91b thru 0x2f630c1 (0x37a7 bytes) 2024-09-18T00:19:06.910 INFO:teuthology.orchestra.run.smithi097.stdout:4186 read 0x5f6632b thru 0x5f74484 (0xe15a bytes) 2024-09-18T00:19:06.910 INFO:teuthology.orchestra.run.smithi097.stdout:4187 writesame 0x12affae thru 0x12b8ecd data_size 0x50(0x8f20 bytes) 2024-09-18T00:19:06.928 INFO:teuthology.orchestra.run.smithi097.stdout:4189 write 0xe66bd12 thru 0xe66c67b (0x96a bytes) 2024-09-18T00:19:06.931 INFO:teuthology.orchestra.run.smithi097.stdout:4190 writesame 0x915868d thru 0x915fd2a data_size 0x2a(0x769e bytes) 2024-09-18T00:19:06.934 INFO:teuthology.orchestra.run.smithi097.stdout:4191 read 0x73e8748 thru 0x73ef25f (0x6b18 bytes) 2024-09-18T00:19:06.935 INFO:teuthology.orchestra.run.smithi097.stdout:4192 writesame 0xa3233b4 thru 0xa32fb65 data_size 0x1(0xc7b2 bytes) 2024-09-18T00:19:06.954 INFO:teuthology.orchestra.run.smithi114.stderr:no valid command found; 10 closest matches: 2024-09-18T00:19:06.955 INFO:teuthology.orchestra.run.smithi114.stderr:0 2024-09-18T00:19:06.955 INFO:teuthology.orchestra.run.smithi114.stderr:1 2024-09-18T00:19:06.955 INFO:teuthology.orchestra.run.smithi114.stderr:2 2024-09-18T00:19:06.955 INFO:teuthology.orchestra.run.smithi114.stderr:abort 2024-09-18T00:19:06.955 INFO:teuthology.orchestra.run.smithi114.stderr:assert 2024-09-18T00:19:06.955 INFO:teuthology.orchestra.run.smithi114.stderr:config diff 2024-09-18T00:19:06.955 INFO:teuthology.orchestra.run.smithi114.stderr:config diff get 2024-09-18T00:19:06.955 INFO:teuthology.orchestra.run.smithi114.stderr:config get 2024-09-18T00:19:06.955 INFO:teuthology.orchestra.run.smithi114.stderr:config help [] 2024-09-18T00:19:06.955 INFO:teuthology.orchestra.run.smithi114.stderr:config set ... 2024-09-18T00:19:06.955 INFO:teuthology.orchestra.run.smithi114.stderr:admin_socket: invalid command 2024-09-18T00:19:06.957 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T00:19:06.957 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-09-18T00:19:06.998 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:06.998+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:07.028 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:07.029 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:07.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:07.029 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:07.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.029 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:5722 3.4 e41/23 3:20381d86:::rbd_data.112a3a39fc45.000000000000001d:head v 41'107, mlcod=40'105)", 2024-09-18T00:19:07.029 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:07.029 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.692233399999999, 2024-09-18T00:19:07.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01185172, 2024-09-18T00:19:07.029 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.029 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.029 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.029 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:07.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:07.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335497+0000", 2024-09-18T00:19:07.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.505e-06 2024-09-18T00:19:07.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335945+0000", 2024-09-18T00:19:07.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000447687 2024-09-18T00:19:07.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335961+0000", 2024-09-18T00:19:07.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6430999999999999e-05 2024-09-18T00:19:07.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335969+0000", 2024-09-18T00:19:07.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:19:07.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336030+0000", 2024-09-18T00:19:07.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1255999999999998e-05 2024-09-18T00:19:07.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336061+0000", 2024-09-18T00:19:07.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0579e-05 2024-09-18T00:19:07.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347296+0000", 2024-09-18T00:19:07.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011235663 2024-09-18T00:19:07.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347347+0000", 2024-09-18T00:19:07.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0914000000000001e-05 2024-09-18T00:19:07.034 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.034 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.034 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.034 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11457 3.0 3:0214e7b3:::rbd_data.112ab1fa102a.0000000000000150:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:07.034 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:07.034 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.767867462000002, 2024-09-18T00:19:07.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028499165, 2024-09-18T00:19:07.034 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.034 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.035 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.035 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.035 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.035 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.035 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11457 2024-09-18T00:19:07.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.035 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:07.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:07.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259865+0000", 2024-09-18T00:19:07.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7230000000000002e-06 2024-09-18T00:19:07.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259883+0000", 2024-09-18T00:19:07.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8067000000000002e-05 2024-09-18T00:19:07.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259924+0000", 2024-09-18T00:19:07.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0550999999999999e-05 2024-09-18T00:19:07.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259943+0000", 2024-09-18T00:19:07.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:19:07.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260054+0000", 2024-09-18T00:19:07.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011075 2024-09-18T00:19:07.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260138+0000", 2024-09-18T00:19:07.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4549000000000007e-05 2024-09-18T00:19:07.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.287926+0000", 2024-09-18T00:19:07.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027787642000000001 2024-09-18T00:19:07.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288090+0000", 2024-09-18T00:19:07.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016402499999999999 2024-09-18T00:19:07.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288361+0000", 2024-09-18T00:19:07.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027097800000000001 2024-09-18T00:19:07.039 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.039 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.039 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.039 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18661 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.039 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:07.039 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.093099597, 2024-09-18T00:19:07.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020008037999999999, 2024-09-18T00:19:07.040 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.040 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.040 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.040 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.040 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.040 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.040 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18661 2024-09-18T00:19:07.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.040 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:07.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:07.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934633+0000", 2024-09-18T00:19:07.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8e-06 2024-09-18T00:19:07.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935511+0000", 2024-09-18T00:19:07.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087785700000000001 2024-09-18T00:19:07.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935561+0000", 2024-09-18T00:19:07.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9472000000000003e-05 2024-09-18T00:19:07.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935586+0000", 2024-09-18T00:19:07.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5616e-05 2024-09-18T00:19:07.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935838+0000", 2024-09-18T00:19:07.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025202699999999999 2024-09-18T00:19:07.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.936023+0000", 2024-09-18T00:19:07.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018480300000000001 2024-09-18T00:19:07.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953545+0000", 2024-09-18T00:19:07.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017522157 2024-09-18T00:19:07.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953723+0000", 2024-09-18T00:19:07.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017753800000000001 2024-09-18T00:19:07.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954637+0000", 2024-09-18T00:19:07.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091476800000000005 2024-09-18T00:19:07.044 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.044 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.044 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.044 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18662 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.045 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:07.045 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.092094660000001, 2024-09-18T00:19:07.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019388843999999999, 2024-09-18T00:19:07.045 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.045 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.045 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.045 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.045 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.045 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.045 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18662 2024-09-18T00:19:07.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.045 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:07.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:07.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935636+0000", 2024-09-18T00:19:07.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.663e-06 2024-09-18T00:19:07.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935646+0000", 2024-09-18T00:19:07.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000006e-06 2024-09-18T00:19:07.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935672+0000", 2024-09-18T00:19:07.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5868999999999999e-05 2024-09-18T00:19:07.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935692+0000", 2024-09-18T00:19:07.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0604000000000001e-05 2024-09-18T00:19:07.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950738+0000", 2024-09-18T00:19:07.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015045868 2024-09-18T00:19:07.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950992+0000", 2024-09-18T00:19:07.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025368400000000001 2024-09-18T00:19:07.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954674+0000", 2024-09-18T00:19:07.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003682154 2024-09-18T00:19:07.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954806+0000", 2024-09-18T00:19:07.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013252000000000001 2024-09-18T00:19:07.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.955023+0000", 2024-09-18T00:19:07.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021681200000000001 2024-09-18T00:19:07.050 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.050 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.050 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.050 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19382 3.4 3:22f09d3a:::rbd_data.112a7fe29ab4.000000000000015a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.050 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:07.050 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.40568461, 2024-09-18T00:19:07.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075119302999999998, 2024-09-18T00:19:07.050 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.050 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.050 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.051 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.051 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.051 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.051 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19382 2024-09-18T00:19:07.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.051 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:07.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:07.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622045+0000", 2024-09-18T00:19:07.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0020000000000001e-06 2024-09-18T00:19:07.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622053+0000", 2024-09-18T00:19:07.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2679999999999997e-06 2024-09-18T00:19:07.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622062+0000", 2024-09-18T00:19:07.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3749999999999992e-06 2024-09-18T00:19:07.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622068+0000", 2024-09-18T00:19:07.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5729999999999998e-06 2024-09-18T00:19:07.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622118+0000", 2024-09-18T00:19:07.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0696999999999999e-05 2024-09-18T00:19:07.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622222+0000", 2024-09-18T00:19:07.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103286 2024-09-18T00:19:07.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697061+0000", 2024-09-18T00:19:07.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074839141999999997 2024-09-18T00:19:07.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697102+0000", 2024-09-18T00:19:07.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1513e-05 2024-09-18T00:19:07.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697164+0000", 2024-09-18T00:19:07.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1446999999999996e-05 2024-09-18T00:19:07.056 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.056 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.056 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.056 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19896 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.056 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:19:07.056 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.086300472, 2024-09-18T00:19:07.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097478370000000005, 2024-09-18T00:19:07.056 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.056 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.056 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:07.056 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.056 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.057 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19896 2024-09-18T00:19:07.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.057 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:19:07.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:19:07.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:19:07.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6000000000000002e-07 2024-09-18T00:19:07.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941434+0000", 2024-09-18T00:19:07.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4750000000000004e-06 2024-09-18T00:19:07.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941452+0000", 2024-09-18T00:19:07.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8603000000000001e-05 2024-09-18T00:19:07.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941461+0000", 2024-09-18T00:19:07.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5199999999999997e-06 2024-09-18T00:19:07.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.947391+0000", 2024-09-18T00:19:07.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059302399999999998 2024-09-18T00:19:07.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.947479+0000", 2024-09-18T00:19:07.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8002000000000002e-05 2024-09-18T00:19:07.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.951176+0000", 2024-09-18T00:19:07.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036971370000000001 2024-09-18T00:19:07.060 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.060 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.060 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.060 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19910 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.060 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941878+0000", 2024-09-18T00:19:07.060 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.085850743, 2024-09-18T00:19:07.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092861479999999993, 2024-09-18T00:19:07.061 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.061 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.061 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:07.061 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.061 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.061 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.061 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19910 2024-09-18T00:19:07.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.061 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941878+0000", 2024-09-18T00:19:07.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941878+0000", 2024-09-18T00:19:07.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941879+0000", 2024-09-18T00:19:07.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6199999999999998e-07 2024-09-18T00:19:07.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.062 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.062 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941881+0000", 2024-09-18T00:19:07.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6390000000000001e-06 2024-09-18T00:19:07.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941890+0000", 2024-09-18T00:19:07.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4999999999999999e-06 2024-09-18T00:19:07.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941896+0000", 2024-09-18T00:19:07.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1299999999999998e-06 2024-09-18T00:19:07.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950898+0000", 2024-09-18T00:19:07.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0090014499999999994 2024-09-18T00:19:07.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950984+0000", 2024-09-18T00:19:07.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6933000000000003e-05 2024-09-18T00:19:07.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.951164+0000", 2024-09-18T00:19:07.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000179934 2024-09-18T00:19:07.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.064 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.064 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.065 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20241 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.065 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155593+0000", 2024-09-18T00:19:07.065 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.872135612999999, 2024-09-18T00:19:07.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0090827070000000006, 2024-09-18T00:19:07.065 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.065 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.065 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:07.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.065 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.065 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20241 2024-09-18T00:19:07.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.065 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155593+0000", 2024-09-18T00:19:07.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155593+0000", 2024-09-18T00:19:07.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155594+0000", 2024-09-18T00:19:07.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9700000000000002e-07 2024-09-18T00:19:07.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155596+0000", 2024-09-18T00:19:07.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1210000000000001e-06 2024-09-18T00:19:07.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155602+0000", 2024-09-18T00:19:07.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.144e-06 2024-09-18T00:19:07.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.067 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155607+0000", 2024-09-18T00:19:07.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7539999999999997e-06 2024-09-18T00:19:07.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.067 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164450+0000", 2024-09-18T00:19:07.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0088436079999999993 2024-09-18T00:19:07.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164537+0000", 2024-09-18T00:19:07.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6329000000000006e-05 2024-09-18T00:19:07.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164676+0000", 2024-09-18T00:19:07.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013935400000000001 2024-09-18T00:19:07.068 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.068 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20242 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155613+0000", 2024-09-18T00:19:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.872115953, 2024-09-18T00:19:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00933551, 2024-09-18T00:19:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20242 2024-09-18T00:19:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155613+0000", 2024-09-18T00:19:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155613+0000", 2024-09-18T00:19:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155613+0000", 2024-09-18T00:19:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1e-07 2024-09-18T00:19:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155615+0000", 2024-09-18T00:19:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1110000000000002e-06 2024-09-18T00:19:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155621+0000", 2024-09-18T00:19:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:19:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155628+0000", 2024-09-18T00:19:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7259999999999996e-06 2024-09-18T00:19:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164725+0000", 2024-09-18T00:19:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0090965009999999999 2024-09-18T00:19:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164832+0000", 2024-09-18T00:19:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106956 2024-09-18T00:19:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164949+0000", 2024-09-18T00:19:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011703 2024-09-18T00:19:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20243 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155635+0000", 2024-09-18T00:19:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.87209391, 2024-09-18T00:19:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.009569358, 2024-09-18T00:19:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20243 2024-09-18T00:19:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155635+0000", 2024-09-18T00:19:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155635+0000", 2024-09-18T00:19:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155635+0000", 2024-09-18T00:19:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8000000000000001e-07 2024-09-18T00:19:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155639+0000", 2024-09-18T00:19:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0809999999999998e-06 2024-09-18T00:19:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155645+0000", 2024-09-18T00:19:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0469999999999999e-06 2024-09-18T00:19:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155649+0000", 2024-09-18T00:19:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7419999999999997e-06 2024-09-18T00:19:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164990+0000", 2024-09-18T00:19:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093401370000000001 2024-09-18T00:19:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165066+0000", 2024-09-18T00:19:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6861999999999997e-05 2024-09-18T00:19:07.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165204+0000", 2024-09-18T00:19:07.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138109 2024-09-18T00:19:07.081 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.081 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.081 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.081 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20244 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.081 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155679+0000", 2024-09-18T00:19:07.081 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.872049954, 2024-09-18T00:19:07.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097580089999999998, 2024-09-18T00:19:07.082 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.082 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.082 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:07.082 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.082 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.082 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.082 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20244 2024-09-18T00:19:07.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.082 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155679+0000", 2024-09-18T00:19:07.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155679+0000", 2024-09-18T00:19:07.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155680+0000", 2024-09-18T00:19:07.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0900000000000002e-07 2024-09-18T00:19:07.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155682+0000", 2024-09-18T00:19:07.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6589999999999999e-06 2024-09-18T00:19:07.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155690+0000", 2024-09-18T00:19:07.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3139999999999998e-06 2024-09-18T00:19:07.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155695+0000", 2024-09-18T00:19:07.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0950000000000003e-06 2024-09-18T00:19:07.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165243+0000", 2024-09-18T00:19:07.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095483519999999995 2024-09-18T00:19:07.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165321+0000", 2024-09-18T00:19:07.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8219000000000004e-05 2024-09-18T00:19:07.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165437+0000", 2024-09-18T00:19:07.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011586099999999999 2024-09-18T00:19:07.085 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.085 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.085 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.086 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20245 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.086 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:07.086 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.872026281, 2024-09-18T00:19:07.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099981159999999996, 2024-09-18T00:19:07.086 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.086 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.086 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:07.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.086 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.086 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20245 2024-09-18T00:19:07.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.086 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:07.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:07.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:07.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.65e-07 2024-09-18T00:19:07.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155711+0000", 2024-09-18T00:19:07.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:19:07.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155720+0000", 2024-09-18T00:19:07.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6580000000000008e-06 2024-09-18T00:19:07.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155725+0000", 2024-09-18T00:19:07.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3430000000000003e-06 2024-09-18T00:19:07.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165479+0000", 2024-09-18T00:19:07.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097538239999999995 2024-09-18T00:19:07.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165556+0000", 2024-09-18T00:19:07.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6724000000000006e-05 2024-09-18T00:19:07.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165701+0000", 2024-09-18T00:19:07.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145129 2024-09-18T00:19:07.090 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.090 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.090 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.090 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20246 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.090 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:07.090 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.871996418, 2024-09-18T00:19:07.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01018784, 2024-09-18T00:19:07.090 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.090 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.090 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:07.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.091 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20246 2024-09-18T00:19:07.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.091 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:07.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:07.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:07.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2599999999999998e-07 2024-09-18T00:19:07.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155735+0000", 2024-09-18T00:19:07.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2699999999999999e-06 2024-09-18T00:19:07.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155744+0000", 2024-09-18T00:19:07.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8629999999999997e-06 2024-09-18T00:19:07.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155749+0000", 2024-09-18T00:19:07.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0359999999999997e-06 2024-09-18T00:19:07.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165748+0000", 2024-09-18T00:19:07.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099991449999999992 2024-09-18T00:19:07.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165819+0000", 2024-09-18T00:19:07.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0964999999999996e-05 2024-09-18T00:19:07.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165920+0000", 2024-09-18T00:19:07.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101135 2024-09-18T00:19:07.094 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.094 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.094 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.094 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20247 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.095 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:07.095 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.871973076, 2024-09-18T00:19:07.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010443987, 2024-09-18T00:19:07.095 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.095 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.095 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:07.095 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.095 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.095 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.095 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20247 2024-09-18T00:19:07.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.095 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:07.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:07.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:07.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1899999999999998e-07 2024-09-18T00:19:07.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155760+0000", 2024-09-18T00:19:07.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6459999999999999e-06 2024-09-18T00:19:07.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155767+0000", 2024-09-18T00:19:07.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0350000000000004e-06 2024-09-18T00:19:07.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155774+0000", 2024-09-18T00:19:07.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:19:07.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165960+0000", 2024-09-18T00:19:07.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186051999999999 2024-09-18T00:19:07.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166048+0000", 2024-09-18T00:19:07.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8848e-05 2024-09-18T00:19:07.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166200+0000", 2024-09-18T00:19:07.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015147000000000001 2024-09-18T00:19:07.098 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.099 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.099 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.099 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20248 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.099 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:07.099 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.871947912, 2024-09-18T00:19:07.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010617721, 2024-09-18T00:19:07.099 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.099 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.099 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:07.099 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.099 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.099 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.100 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20248 2024-09-18T00:19:07.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.100 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:07.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:07.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155782+0000", 2024-09-18T00:19:07.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.84e-07 2024-09-18T00:19:07.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155784+0000", 2024-09-18T00:19:07.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2800000000000002e-06 2024-09-18T00:19:07.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155790+0000", 2024-09-18T00:19:07.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5679999999999996e-06 2024-09-18T00:19:07.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155795+0000", 2024-09-18T00:19:07.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6929999999999998e-06 2024-09-18T00:19:07.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166245+0000", 2024-09-18T00:19:07.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010449847999999999 2024-09-18T00:19:07.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166307+0000", 2024-09-18T00:19:07.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2287999999999997e-05 2024-09-18T00:19:07.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166399+0000", 2024-09-18T00:19:07.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.166e-05 2024-09-18T00:19:07.103 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.103 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.103 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.103 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20249 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.103 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:07.103 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.871927535999999, 2024-09-18T00:19:07.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010811983000000001, 2024-09-18T00:19:07.104 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.104 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.104 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:07.104 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.104 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.104 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.104 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20249 2024-09-18T00:19:07.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.104 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:07.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:07.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155802+0000", 2024-09-18T00:19:07.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:19:07.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155804+0000", 2024-09-18T00:19:07.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:19:07.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155810+0000", 2024-09-18T00:19:07.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0580000000000003e-06 2024-09-18T00:19:07.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155817+0000", 2024-09-18T00:19:07.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3980000000000004e-06 2024-09-18T00:19:07.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166451+0000", 2024-09-18T00:19:07.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010634825000000001 2024-09-18T00:19:07.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166520+0000", 2024-09-18T00:19:07.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8317000000000003e-05 2024-09-18T00:19:07.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166613+0000", 2024-09-18T00:19:07.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3801000000000005e-05 2024-09-18T00:19:07.108 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.108 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.108 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.108 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:21913 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:07.108 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:07.108 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.7340823899999993, 2024-09-18T00:19:07.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090508916999999994, 2024-09-18T00:19:07.109 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.109 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.109 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.109 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.109 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.109 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.109 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 21913 2024-09-18T00:19:07.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.109 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:07.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:07.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293648+0000", 2024-09-18T00:19:07.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6400000000000001e-07 2024-09-18T00:19:07.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293653+0000", 2024-09-18T00:19:07.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3820000000000003e-06 2024-09-18T00:19:07.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293665+0000", 2024-09-18T00:19:07.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1778e-05 2024-09-18T00:19:07.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293673+0000", 2024-09-18T00:19:07.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4130000000000006e-06 2024-09-18T00:19:07.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293719+0000", 2024-09-18T00:19:07.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5534999999999997e-05 2024-09-18T00:19:07.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293773+0000", 2024-09-18T00:19:07.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4389999999999999e-05 2024-09-18T00:19:07.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384053+0000", 2024-09-18T00:19:07.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090279617000000006 2024-09-18T00:19:07.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384088+0000", 2024-09-18T00:19:07.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5466000000000002e-05 2024-09-18T00:19:07.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384156+0000", 2024-09-18T00:19:07.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7471999999999994e-05 2024-09-18T00:19:07.113 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.113 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.113 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.113 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22274 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:07.114 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:07.114 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.9229661700000005, 2024-09-18T00:19:07.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.069820742000000005, 2024-09-18T00:19:07.114 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.114 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.114 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.114 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.114 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.114 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.114 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22274 2024-09-18T00:19:07.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.114 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:07.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:07.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104766+0000", 2024-09-18T00:19:07.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3859999999999999e-06 2024-09-18T00:19:07.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104783+0000", 2024-09-18T00:19:07.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6697e-05 2024-09-18T00:19:07.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104816+0000", 2024-09-18T00:19:07.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2786000000000001e-05 2024-09-18T00:19:07.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104833+0000", 2024-09-18T00:19:07.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7125999999999998e-05 2024-09-18T00:19:07.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104965+0000", 2024-09-18T00:19:07.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013215899999999999 2024-09-18T00:19:07.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.105103+0000", 2024-09-18T00:19:07.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138193 2024-09-18T00:19:07.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174496+0000", 2024-09-18T00:19:07.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.06939244 2024-09-18T00:19:07.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174530+0000", 2024-09-18T00:19:07.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4601999999999998e-05 2024-09-18T00:19:07.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174584+0000", 2024-09-18T00:19:07.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3353000000000002e-05 2024-09-18T00:19:07.118 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.118 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.118 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.119 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22485 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write 16521958~30847] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:07.119 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:07.119 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.0741771599999996, 2024-09-18T00:19:07.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012574648000000001, 2024-09-18T00:19:07.119 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.119 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.119 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.119 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.119 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.119 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.119 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22485 2024-09-18T00:19:07.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.120 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:07.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:07.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953554+0000", 2024-09-18T00:19:07.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1150000000000001e-06 2024-09-18T00:19:07.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962036+0000", 2024-09-18T00:19:07.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0084817599999999996 2024-09-18T00:19:07.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962062+0000", 2024-09-18T00:19:07.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6499000000000002e-05 2024-09-18T00:19:07.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962074+0000", 2024-09-18T00:19:07.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2095e-05 2024-09-18T00:19:07.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962165+0000", 2024-09-18T00:19:07.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0703000000000002e-05 2024-09-18T00:19:07.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962276+0000", 2024-09-18T00:19:07.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111406 2024-09-18T00:19:07.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965925+0000", 2024-09-18T00:19:07.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036488919999999999 2024-09-18T00:19:07.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965992+0000", 2024-09-18T00:19:07.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6806999999999999e-05 2024-09-18T00:19:07.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.966127+0000", 2024-09-18T00:19:07.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013437099999999999 2024-09-18T00:19:07.123 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.124 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.124 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.124 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22681 3.4 3:22e76021:::rbd_data.112a948d5956.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:07.124 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:07.124 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.7401778549999998, 2024-09-18T00:19:07.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076591189000000004, 2024-09-18T00:19:07.124 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.124 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.124 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.124 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.124 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.124 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.125 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22681 2024-09-18T00:19:07.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.125 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:07.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:07.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287554+0000", 2024-09-18T00:19:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.514e-06 2024-09-18T00:19:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287567+0000", 2024-09-18T00:19:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-18T00:19:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287597+0000", 2024-09-18T00:19:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0406e-05 2024-09-18T00:19:07.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287626+0000", 2024-09-18T00:19:07.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9306e-05 2024-09-18T00:19:07.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287801+0000", 2024-09-18T00:19:07.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017411400000000001 2024-09-18T00:19:07.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.288053+0000", 2024-09-18T00:19:07.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025223500000000001 2024-09-18T00:19:07.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364016+0000", 2024-09-18T00:19:07.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075963301999999996 2024-09-18T00:19:07.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364064+0000", 2024-09-18T00:19:07.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7886000000000002e-05 2024-09-18T00:19:07.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.128 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.128 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364142+0000", 2024-09-18T00:19:07.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8345999999999998e-05 2024-09-18T00:19:07.129 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.129 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.129 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.129 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.129 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.129 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:07.130 INFO:teuthology.orchestra.run.smithi097.stdout:4193 write 0x4ab4ac6 thru 0x4ab5b56 (0x1091 bytes) 2024-09-18T00:19:07.130 INFO:teuthology.orchestra.run.smithi097.stdout:4194 writesame 0xa33a90c thru 0xa341e59 data_size 0x82(0x754e bytes) 2024-09-18T00:19:07.130 INFO:teuthology.orchestra.run.smithi097.stdout:4197 writesame 0xbf7e50f thru 0xbf7f1cb data_size 0x1(0xcbd bytes) 2024-09-18T00:19:07.131 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:07.131 INFO:teuthology.orchestra.run.smithi097.stdout:4204 read 0x86a4be7 thru 0x86acc3b (0x8055 bytes) 2024-09-18T00:19:07.131 INFO:teuthology.orchestra.run.smithi097.stdout:4206 read 0x45598f8 thru 0x455ae1f (0x1528 bytes) 2024-09-18T00:19:07.132 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:19:07.133 INFO:teuthology.orchestra.run.smithi097.stdout:4207 writesame 0x6bb708e thru 0x6bc4f0b data_size 0x21(0xde7e bytes) 2024-09-18T00:19:07.134 INFO:teuthology.orchestra.run.smithi097.stdout:4208 read 0x25e13e9 thru 0x25e5332 (0x3f4a bytes) 2024-09-18T00:19:07.134 INFO:teuthology.orchestra.run.smithi097.stdout:4210 write 0x4a651b4 thru 0x4a6deac (0x8cf9 bytes) 2024-09-18T00:19:07.134 INFO:teuthology.orchestra.run.smithi097.stdout:4211 write 0x67d082d thru 0x67de236 (0xda0a bytes) 2024-09-18T00:19:07.134 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:07.134 INFO:teuthology.orchestra.run.smithi097.stdout:4213 trunc from 0xe66c67c to 0xd3dae78 2024-09-18T00:19:07.134 INFO:teuthology.orchestra.run.smithi097.stdout:4214 read 0xc1d3820 thru 0xc1d6997 (0x3178 bytes) 2024-09-18T00:19:07.134 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:07.134 INFO:teuthology.orchestra.run.smithi097.stdout:4218 write 0xdfbe120 thru 0xdfc7ced (0x9bce bytes) 2024-09-18T00:19:07.134 INFO:teuthology.orchestra.run.smithi097.stdout:4219 trunc from 0xdfc7cee to 0x7281162 2024-09-18T00:19:07.134 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:07.125+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:07.163 INFO:teuthology.orchestra.run.smithi097.stdout:4220 trunc from 0x7281162 to 0xc16a20a 2024-09-18T00:19:07.175 INFO:teuthology.orchestra.run.smithi097.stdout:4221 read 0x8056017 thru 0x805d12e (0x7118 bytes) 2024-09-18T00:19:07.176 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:07.176 INFO:teuthology.orchestra.run.smithi097.stdout:4225 punch from 0x82fa446 to 0x82fe7a2, (0x435c bytes) 2024-09-18T00:19:07.176 INFO:teuthology.orchestra.run.smithi097.stdout:4226 read 0x9e193a3 thru 0x9e19e41 (0xa9f bytes) 2024-09-18T00:19:07.176 INFO:teuthology.orchestra.run.smithi097.stdout:4229 write 0x9fd8960 thru 0x9fe39ff (0xb0a0 bytes) 2024-09-18T00:19:07.183 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:07.183 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:07.183 INFO:teuthology.orchestra.run.smithi097.stdout:4233 read 0x9cedd thru 0xa0b8d (0x3cb1 bytes) 2024-09-18T00:19:07.185 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:07.185 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:07.185 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:07.196 INFO:teuthology.orchestra.run.smithi097.stdout:4237 write 0xda7855e thru 0xda848f3 (0xc396 bytes) 2024-09-18T00:19:07.204 INFO:teuthology.orchestra.run.smithi097.stdout:4238 trunc from 0xda848f4 to 0xde2e566 2024-09-18T00:19:07.213 INFO:teuthology.orchestra.run.smithi097.stdout:4239 write 0x1c74784 thru 0x1c7c7cd (0x804a bytes) 2024-09-18T00:19:07.216 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:07.216 INFO:teuthology.orchestra.run.smithi097.stdout:4242 read 0x8b28972 thru 0x8b2a57b (0x1c0a bytes) 2024-09-18T00:19:07.216 INFO:teuthology.orchestra.run.smithi097.stdout:4243 read 0x3ad0b3 thru 0x3afef6 (0x2e44 bytes) 2024-09-18T00:19:07.217 INFO:teuthology.orchestra.run.smithi097.stdout:4244 read 0xb80162a thru 0xb80708a (0x5a61 bytes) 2024-09-18T00:19:07.217 INFO:teuthology.orchestra.run.smithi097.stdout:4245 read 0x64c46e3 thru 0x64c4efb (0x819 bytes) 2024-09-18T00:19:07.217 INFO:teuthology.orchestra.run.smithi097.stdout:4246 punch from 0xbeb7616 to 0xbec0a2d, (0x9417 bytes) 2024-09-18T00:19:07.218 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:07.218 INFO:teuthology.orchestra.run.smithi097.stdout:4248 read 0xb25bb81 thru 0xb26afbd (0xf43d bytes) 2024-09-18T00:19:07.218 INFO:teuthology.orchestra.run.smithi097.stdout:4249 punch from 0xaf613ae to 0xaf6c93c, (0xb58e bytes) 2024-09-18T00:19:07.218 INFO:teuthology.orchestra.run.smithi097.stdout:4251 write 0x7856c6e thru 0x78646b4 (0xda47 bytes) 2024-09-18T00:19:07.222 INFO:teuthology.orchestra.run.smithi097.stdout:4252 writesame 0x19f47f5 thru 0x1a00c3d data_size 0x1(0xc449 bytes) 2024-09-18T00:19:07.231 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:07.230+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:07.287 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:07.287 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:07.287 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:07.287 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:07.300 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:19:07.332 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:07.330+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:07.370 INFO:teuthology.orchestra.run.smithi097.stdout:4254 writesame 0x31dc15f thru 0x31dd085 data_size 0x3(0xf27 bytes) 2024-09-18T00:19:07.392 INFO:teuthology.orchestra.run.smithi097.stdout:4255 trunc from 0xde2e566 to 0x3bac04c 2024-09-18T00:19:07.409 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:07.425 INFO:teuthology.orchestra.run.smithi097.stdout:4257 trunc from 0x3bac04c to 0xe2f2719 2024-09-18T00:19:07.433 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:07.429+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:07.433 INFO:teuthology.orchestra.run.smithi097.stdout:4258 read 0xb13ab98 thru 0xb143789 (0x8bf2 bytes) 2024-09-18T00:19:07.433 INFO:teuthology.orchestra.run.smithi097.stdout:4260 read 0x9a8fb11 thru 0x9a9464e (0x4b3e bytes) 2024-09-18T00:19:07.433 INFO:teuthology.orchestra.run.smithi097.stdout:4261 write 0x4718f9e thru 0x4724be8 (0xbc4b bytes) 2024-09-18T00:19:07.438 INFO:teuthology.orchestra.run.smithi097.stdout:4262 write 0xab4e1f1 thru 0xab4e558 (0x368 bytes) 2024-09-18T00:19:07.443 INFO:teuthology.orchestra.run.smithi097.stdout:4263 write 0x5b8a4c4 thru 0x5b950cb (0xac08 bytes) 2024-09-18T00:19:07.452 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:07.452 INFO:teuthology.orchestra.run.smithi097.stdout:4267 punch from 0x48c5a8 to 0x48f5b5, (0x300d bytes) 2024-09-18T00:19:07.452 INFO:teuthology.orchestra.run.smithi097.stdout:4269 write 0x3b92724 thru 0x3b98eda (0x67b7 bytes) 2024-09-18T00:19:07.455 INFO:teuthology.orchestra.run.smithi097.stdout:4273 writesame 0x4196b0a thru 0x41a4b86 data_size 0x1(0xe07d bytes) 2024-09-18T00:19:07.479 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:07.479 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:07.479 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:07.479 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:07.479 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:07.493 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:19:07.537 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:07.533+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:07.539 INFO:teuthology.orchestra.run.smithi097.stdout:4274 trunc from 0xe2f2719 to 0x720e711 2024-09-18T00:19:07.564 INFO:teuthology.orchestra.run.smithi097.stdout:4275 punch from 0x53e43aa to 0x53e7726, (0x337c bytes) 2024-09-18T00:19:07.564 INFO:teuthology.orchestra.run.smithi097.stdout:4277 punch from 0x6549253 to 0x6558233, (0xefe0 bytes) 2024-09-18T00:19:07.564 INFO:teuthology.orchestra.run.smithi097.stdout:4278 read 0x6b8c287 thru 0x6b96dd1 (0xab4b bytes) 2024-09-18T00:19:07.565 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:07.583 INFO:teuthology.orchestra.run.smithi097.stdout:4280 write 0xb8dd249 thru 0xb8e78aa (0xa662 bytes) 2024-09-18T00:19:07.594 INFO:teuthology.orchestra.run.smithi097.stdout:4282 trunc from 0xb8e78ab to 0x7b6cc6d 2024-09-18T00:19:07.618 INFO:teuthology.orchestra.run.smithi097.stdout:4283 read 0x67371fc thru 0x673a9b9 (0x37be bytes) 2024-09-18T00:19:07.618 INFO:teuthology.orchestra.run.smithi097.stdout:4284 read 0x7859654 thru 0x785ae74 (0x1821 bytes) 2024-09-18T00:19:07.625 INFO:teuthology.orchestra.run.smithi097.stdout:4286 trunc from 0x7b6cc6d to 0x4092ece 2024-09-18T00:19:07.635 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:07.634+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:07.649 INFO:teuthology.orchestra.run.smithi097.stdout:4287 read 0x2c14cf0 thru 0x2c1c607 (0x7918 bytes) 2024-09-18T00:19:07.669 INFO:teuthology.orchestra.run.smithi097.stdout:4289 write 0x8369b3b thru 0x836c3f9 (0x28bf bytes) 2024-09-18T00:19:07.669 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:07.670 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:07.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:07.670 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:07.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.670 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:19:07.670 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:07.670 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.866430954000002, 2024-09-18T00:19:07.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:19:07.670 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.670 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.670 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.670 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.670 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.671 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.671 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:19:07.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.671 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:07.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:07.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:19:07.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:19:07.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:19:07.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:19:07.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:19:07.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:19:07.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:19:07.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:19:07.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:19:07.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:19:07.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:19:07.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:19:07.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:07.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:19:07.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:19:07.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:19:07.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:19:07.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:07.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:19:07.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:19:07.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:19:07.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:19:07.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:19:07.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:19:07.675 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.675 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.675 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.676 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:19:07.676 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:07.676 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.850223081000003, 2024-09-18T00:19:07.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:19:07.676 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.676 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.676 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:07.676 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:07.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:19:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:19:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:19:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:19:07.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:19:07.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:19:07.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:19:07.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:19:07.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:19:07.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:19:07.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:19:07.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:19:07.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:19:07.679 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.679 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.679 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.679 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:07.679 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:07.679 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.823181761000001, 2024-09-18T00:19:07.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:19:07.680 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.680 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.680 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.680 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.680 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.680 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.680 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:19:07.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.680 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:07.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:07.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:19:07.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:19:07.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:19:07.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:19:07.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:19:07.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:19:07.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:19:07.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:19:07.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:19:07.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:19:07.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:19:07.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:19:07.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:07.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:19:07.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:19:07.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:19:07.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:19:07.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:07.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:19:07.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:19:07.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:19:07.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:19:07.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:19:07.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:19:07.685 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.685 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.685 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.685 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:19:07.685 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:07.685 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.801905208999997, 2024-09-18T00:19:07.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:19:07.686 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.686 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.686 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:07.686 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:07.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:07.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:19:07.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:19:07.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:19:07.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:19:07.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.687 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:19:07.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:19:07.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:19:07.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:19:07.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:19:07.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:19:07.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:19:07.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:19:07.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:19:07.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:19:07.689 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.689 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.689 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.690 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11460 3.19 3:9e578b6c:::rbd_data.112ab1fa102a.0000000000000153:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:07.690 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:07.690 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.399533757, 2024-09-18T00:19:07.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041989011999999999, 2024-09-18T00:19:07.690 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.690 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.690 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.690 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.690 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.690 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.690 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11460 2024-09-18T00:19:07.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.690 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:07.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:07.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267067+0000", 2024-09-18T00:19:07.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:19:07.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267088+0000", 2024-09-18T00:19:07.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0908999999999999e-05 2024-09-18T00:19:07.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267133+0000", 2024-09-18T00:19:07.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5232000000000003e-05 2024-09-18T00:19:07.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.692 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267157+0000", 2024-09-18T00:19:07.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3833e-05 2024-09-18T00:19:07.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267300+0000", 2024-09-18T00:19:07.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142282 2024-09-18T00:19:07.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267447+0000", 2024-09-18T00:19:07.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147851 2024-09-18T00:19:07.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308889+0000", 2024-09-18T00:19:07.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041441779999999998 2024-09-18T00:19:07.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308956+0000", 2024-09-18T00:19:07.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7136000000000002e-05 2024-09-18T00:19:07.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.309053+0000", 2024-09-18T00:19:07.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6552999999999995e-05 2024-09-18T00:19:07.694 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.694 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.695 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.695 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11464 3.b 3:d331f8ac:::rbd_data.112ab1fa102a.0000000000000157:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:07.695 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:19:07.695 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.394543271000003, 2024-09-18T00:19:07.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034709218, 2024-09-18T00:19:07.695 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.695 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.695 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.695 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.695 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.695 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.695 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11464 2024-09-18T00:19:07.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.696 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:19:07.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272054+0000", 2024-09-18T00:19:07.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272058+0000", 2024-09-18T00:19:07.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4070000000000002e-06 2024-09-18T00:19:07.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272074+0000", 2024-09-18T00:19:07.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5872999999999999e-05 2024-09-18T00:19:07.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.697 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.697 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272113+0000", 2024-09-18T00:19:07.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9224999999999998e-05 2024-09-18T00:19:07.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272135+0000", 2024-09-18T00:19:07.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2093e-05 2024-09-18T00:19:07.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272238+0000", 2024-09-18T00:19:07.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000102939 2024-09-18T00:19:07.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.272327+0000", 2024-09-18T00:19:07.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8746999999999996e-05 2024-09-18T00:19:07.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:07.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.273486+0000", 2024-09-18T00:19:07.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011588779999999999 2024-09-18T00:19:07.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:07.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306439+0000", 2024-09-18T00:19:07.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.032953142999999997 2024-09-18T00:19:07.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306491+0000", 2024-09-18T00:19:07.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2475000000000002e-05 2024-09-18T00:19:07.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306648+0000", 2024-09-18T00:19:07.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015725099999999999 2024-09-18T00:19:07.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.306764+0000", 2024-09-18T00:19:07.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011518699999999999 2024-09-18T00:19:07.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.700 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.700 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:16947 3.18 3:18b58c7b:::rbd_data.112a6efb7564.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:19:07.701 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:07.701 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.049756056, 2024-09-18T00:19:07.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.101692484, 2024-09-18T00:19:07.701 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.701 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.701 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.701 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 16947 2024-09-18T00:19:07.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.701 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:07.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:07.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616843+0000", 2024-09-18T00:19:07.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0780000000000001e-06 2024-09-18T00:19:07.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617785+0000", 2024-09-18T00:19:07.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094204100000000002 2024-09-18T00:19:07.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617801+0000", 2024-09-18T00:19:07.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6436999999999999e-05 2024-09-18T00:19:07.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617809+0000", 2024-09-18T00:19:07.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9049999999999997e-06 2024-09-18T00:19:07.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617904+0000", 2024-09-18T00:19:07.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5204999999999994e-05 2024-09-18T00:19:07.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.618071+0000", 2024-09-18T00:19:07.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016640000000000001 2024-09-18T00:19:07.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:07.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.624534+0000", 2024-09-18T00:19:07.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064634499999999999 2024-09-18T00:19:07.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718299+0000", 2024-09-18T00:19:07.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093764739 2024-09-18T00:19:07.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:07.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718375+0000", 2024-09-18T00:19:07.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6532999999999994e-05 2024-09-18T00:19:07.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718466+0000", 2024-09-18T00:19:07.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0581000000000006e-05 2024-09-18T00:19:07.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718534+0000", 2024-09-18T00:19:07.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8114999999999994e-05 2024-09-18T00:19:07.706 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.706 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.706 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.707 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:16947 3.18 e59/23 ondisk, result = 0)", 2024-09-18T00:19:07.707 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:07.707 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.024539361999999, 2024-09-18T00:19:07.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076488069000000006, 2024-09-18T00:19:07.707 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.707 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.707 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:07.707 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:07.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:07.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642060+0000", 2024-09-18T00:19:07.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6640000000000001e-06 2024-09-18T00:19:07.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642071+0000", 2024-09-18T00:19:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1204e-05 2024-09-18T00:19:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642091+0000", 2024-09-18T00:19:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0313999999999999e-05 2024-09-18T00:19:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642103+0000", 2024-09-18T00:19:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1214e-05 2024-09-18T00:19:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718350+0000", 2024-09-18T00:19:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076247681999999997 2024-09-18T00:19:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718366+0000", 2024-09-18T00:19:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5420000000000001e-05 2024-09-18T00:19:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718546+0000", 2024-09-18T00:19:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000180571 2024-09-18T00:19:07.710 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18686 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.717861992, 2024-09-18T00:19:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041132573999999998, 2024-09-18T00:19:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18686 2024-09-18T00:19:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:07.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948737+0000", 2024-09-18T00:19:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3769999999999999e-06 2024-09-18T00:19:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949968+0000", 2024-09-18T00:19:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001230493 2024-09-18T00:19:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949993+0000", 2024-09-18T00:19:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5282e-05 2024-09-18T00:19:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950011+0000", 2024-09-18T00:19:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8199999999999999e-05 2024-09-18T00:19:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950180+0000", 2024-09-18T00:19:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016867600000000001 2024-09-18T00:19:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950337+0000", 2024-09-18T00:19:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156902 2024-09-18T00:19:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950818+0000", 2024-09-18T00:19:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048192599999999998 2024-09-18T00:19:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978491+0000", 2024-09-18T00:19:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027672568000000002 2024-09-18T00:19:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989759+0000", 2024-09-18T00:19:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011268144000000001 2024-09-18T00:19:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989824+0000", 2024-09-18T00:19:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4556999999999999e-05 2024-09-18T00:19:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989868+0000", 2024-09-18T00:19:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4449000000000001e-05 2024-09-18T00:19:07.716 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18687 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.716534227, 2024-09-18T00:19:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040034156000000001, 2024-09-18T00:19:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.717 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18687 2024-09-18T00:19:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950065+0000", 2024-09-18T00:19:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5820000000000001e-06 2024-09-18T00:19:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950072+0000", 2024-09-18T00:19:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1029999999999996e-06 2024-09-18T00:19:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950092+0000", 2024-09-18T00:19:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0288e-05 2024-09-18T00:19:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950107+0000", 2024-09-18T00:19:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4290999999999999e-05 2024-09-18T00:19:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975820+0000", 2024-09-18T00:19:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025713573999999999 2024-09-18T00:19:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975980+0000", 2024-09-18T00:19:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016002799999999999 2024-09-18T00:19:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.976299+0000", 2024-09-18T00:19:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031853300000000002 2024-09-18T00:19:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989936+0000", 2024-09-18T00:19:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013637046999999999 2024-09-18T00:19:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990025+0000", 2024-09-18T00:19:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9040000000000001e-05 2024-09-18T00:19:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990064+0000", 2024-09-18T00:19:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9394000000000003e-05 2024-09-18T00:19:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990098+0000", 2024-09-18T00:19:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3275999999999998e-05 2024-09-18T00:19:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.722 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18688 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.716432875000001, 2024-09-18T00:19:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040041259000000003, 2024-09-18T00:19:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18688 2024-09-18T00:19:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950166+0000", 2024-09-18T00:19:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:19:07.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951295+0000", 2024-09-18T00:19:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011293480000000001 2024-09-18T00:19:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951328+0000", 2024-09-18T00:19:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2524999999999998e-05 2024-09-18T00:19:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951346+0000", 2024-09-18T00:19:07.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8170999999999999e-05 2024-09-18T00:19:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978546+0000", 2024-09-18T00:19:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027200418000000001 2024-09-18T00:19:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978666+0000", 2024-09-18T00:19:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119913 2024-09-18T00:19:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.979098+0000", 2024-09-18T00:19:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00043169399999999999 2024-09-18T00:19:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990000+0000", 2024-09-18T00:19:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010901719000000001 2024-09-18T00:19:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990109+0000", 2024-09-18T00:19:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010964299999999999 2024-09-18T00:19:07.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990152+0000", 2024-09-18T00:19:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2877000000000003e-05 2024-09-18T00:19:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990206+0000", 2024-09-18T00:19:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3717000000000001e-05 2024-09-18T00:19:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18689 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.715221177, 2024-09-18T00:19:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051022338, 2024-09-18T00:19:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18689 2024-09-18T00:19:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.729 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951378+0000", 2024-09-18T00:19:07.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:19:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951385+0000", 2024-09-18T00:19:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-18T00:19:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951402+0000", 2024-09-18T00:19:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6538999999999999e-05 2024-09-18T00:19:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951565+0000", 2024-09-18T00:19:07.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016307999999999999 2024-09-18T00:19:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988232+0000", 2024-09-18T00:19:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036667656999999999 2024-09-18T00:19:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988318+0000", 2024-09-18T00:19:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5348999999999999e-05 2024-09-18T00:19:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:07.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988574+0000", 2024-09-18T00:19:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025670800000000002 2024-09-18T00:19:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002254+0000", 2024-09-18T00:19:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013679274 2024-09-18T00:19:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002326+0000", 2024-09-18T00:19:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2720999999999997e-05 2024-09-18T00:19:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002364+0000", 2024-09-18T00:19:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7314000000000002e-05 2024-09-18T00:19:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002399+0000", 2024-09-18T00:19:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5162e-05 2024-09-18T00:19:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18690 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.734 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.715004158999999, 2024-09-18T00:19:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050872134999999999, 2024-09-18T00:19:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18690 2024-09-18T00:19:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:07.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951595+0000", 2024-09-18T00:19:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4479999999999999e-06 2024-09-18T00:19:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952167+0000", 2024-09-18T00:19:07.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00057188400000000004 2024-09-18T00:19:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952192+0000", 2024-09-18T00:19:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4936999999999999e-05 2024-09-18T00:19:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952206+0000", 2024-09-18T00:19:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4565e-05 2024-09-18T00:19:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990303+0000", 2024-09-18T00:19:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038096954000000002 2024-09-18T00:19:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990403+0000", 2024-09-18T00:19:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9486000000000003e-05 2024-09-18T00:19:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990773+0000", 2024-09-18T00:19:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037037799999999998 2024-09-18T00:19:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002316+0000", 2024-09-18T00:19:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011542581 2024-09-18T00:19:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002407+0000", 2024-09-18T00:19:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0968e-05 2024-09-18T00:19:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002442+0000", 2024-09-18T00:19:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4962000000000002e-05 2024-09-18T00:19:07.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002466+0000", 2024-09-18T00:19:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3972e-05 2024-09-18T00:19:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18691 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.714371282, 2024-09-18T00:19:07.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051495530999999997, 2024-09-18T00:19:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18691 2024-09-18T00:19:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:07.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952228+0000", 2024-09-18T00:19:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:19:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952234+0000", 2024-09-18T00:19:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0310000000000004e-06 2024-09-18T00:19:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952248+0000", 2024-09-18T00:19:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4577e-05 2024-09-18T00:19:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952260+0000", 2024-09-18T00:19:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1659e-05 2024-09-18T00:19:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.000989+0000", 2024-09-18T00:19:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048729445000000003 2024-09-18T00:19:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001076+0000", 2024-09-18T00:19:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6323999999999995e-05 2024-09-18T00:19:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001295+0000", 2024-09-18T00:19:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021944299999999999 2024-09-18T00:19:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003502+0000", 2024-09-18T00:19:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022065209999999999 2024-09-18T00:19:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003581+0000", 2024-09-18T00:19:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9404000000000001e-05 2024-09-18T00:19:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003652+0000", 2024-09-18T00:19:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0506999999999994e-05 2024-09-18T00:19:07.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003722+0000", 2024-09-18T00:19:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0300000000000001e-05 2024-09-18T00:19:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18692 3.7 3:e69124d0:::rbd_data.112a6efb7564.0000000000000001:f [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.714323221000001, 2024-09-18T00:19:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050418114999999999, 2024-09-18T00:19:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.746 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18692 2024-09-18T00:19:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:07.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952275+0000", 2024-09-18T00:19:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4e-07 2024-09-18T00:19:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952281+0000", 2024-09-18T00:19:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2000000000000002e-06 2024-09-18T00:19:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952295+0000", 2024-09-18T00:19:07.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4731e-05 2024-09-18T00:19:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952307+0000", 2024-09-18T00:19:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1877e-05 2024-09-18T00:19:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002506+0000", 2024-09-18T00:19:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050199230999999997 2024-09-18T00:19:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002574+0000", 2024-09-18T00:19:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8076000000000003e-05 2024-09-18T00:19:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002693+0000", 2024-09-18T00:19:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011806 2024-09-18T00:19:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19383 3.5 3:a1859bb0:::rbd_data.112a7fe29ab4.000000000000015b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.044311624000001, 2024-09-18T00:19:07.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056760192000000001, 2024-09-18T00:19:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19383 2024-09-18T00:19:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622287+0000", 2024-09-18T00:19:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-18T00:19:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622292+0000", 2024-09-18T00:19:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6110000000000004e-06 2024-09-18T00:19:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622304+0000", 2024-09-18T00:19:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2106000000000001e-05 2024-09-18T00:19:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622309+0000", 2024-09-18T00:19:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.818e-06 2024-09-18T00:19:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622360+0000", 2024-09-18T00:19:07.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0430999999999999e-05 2024-09-18T00:19:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622438+0000", 2024-09-18T00:19:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8362999999999995e-05 2024-09-18T00:19:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678811+0000", 2024-09-18T00:19:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056372894 2024-09-18T00:19:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678906+0000", 2024-09-18T00:19:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5078e-05 2024-09-18T00:19:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.679046+0000", 2024-09-18T00:19:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139987 2024-09-18T00:19:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22019 3.c 3:37b77e94:::rbd_data.112a948d5956.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:07.755 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:19:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.167336795000001, 2024-09-18T00:19:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.039176065000000003, 2024-09-18T00:19:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22019 2024-09-18T00:19:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:19:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:19:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499262+0000", 2024-09-18T00:19:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:19:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499268+0000", 2024-09-18T00:19:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8780000000000003e-06 2024-09-18T00:19:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499283+0000", 2024-09-18T00:19:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5184000000000001e-05 2024-09-18T00:19:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499291+0000", 2024-09-18T00:19:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5959999999999998e-06 2024-09-18T00:19:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499364+0000", 2024-09-18T00:19:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2855000000000006e-05 2024-09-18T00:19:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499427+0000", 2024-09-18T00:19:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3645999999999993e-05 2024-09-18T00:19:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538252+0000", 2024-09-18T00:19:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038825207 2024-09-18T00:19:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538328+0000", 2024-09-18T00:19:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5938000000000004e-05 2024-09-18T00:19:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538437+0000", 2024-09-18T00:19:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108527 2024-09-18T00:19:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22468 3.5 3:a250f1ef:::rbd_data.112a948d5956.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.895230218, 2024-09-18T00:19:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.055599787999999997, 2024-09-18T00:19:07.760 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22468 2024-09-18T00:19:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771370+0000", 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9629999999999998e-06 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771385+0000", 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5053e-05 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771429+0000", 2024-09-18T00:19:07.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3192999999999998e-05 2024-09-18T00:19:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771453+0000", 2024-09-18T00:19:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3995999999999999e-05 2024-09-18T00:19:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771623+0000", 2024-09-18T00:19:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000170846 2024-09-18T00:19:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771838+0000", 2024-09-18T00:19:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021416600000000001 2024-09-18T00:19:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826866+0000", 2024-09-18T00:19:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.05502866 2024-09-18T00:19:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826906+0000", 2024-09-18T00:19:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9805999999999998e-05 2024-09-18T00:19:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826967+0000", 2024-09-18T00:19:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1105000000000005e-05 2024-09-18T00:19:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22469 3.2 3:474595b6:::rbd_data.112a948d5956.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.771512+0000", 2024-09-18T00:19:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.8950860780000003, 2024-09-18T00:19:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034768343, 2024-09-18T00:19:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.765 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22469 2024-09-18T00:19:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771512+0000", 2024-09-18T00:19:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771512+0000", 2024-09-18T00:19:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771513+0000", 2024-09-18T00:19:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8840000000000001e-06 2024-09-18T00:19:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771523+0000", 2024-09-18T00:19:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7429999999999997e-06 2024-09-18T00:19:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771548+0000", 2024-09-18T00:19:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4426e-05 2024-09-18T00:19:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771564+0000", 2024-09-18T00:19:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6820000000000002e-05 2024-09-18T00:19:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771695+0000", 2024-09-18T00:19:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000130321 2024-09-18T00:19:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771897+0000", 2024-09-18T00:19:07.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020252200000000001 2024-09-18T00:19:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.806120+0000", 2024-09-18T00:19:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034222475000000002 2024-09-18T00:19:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.806178+0000", 2024-09-18T00:19:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8696999999999997e-05 2024-09-18T00:19:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.806280+0000", 2024-09-18T00:19:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101455 2024-09-18T00:19:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22470 3.2 3:474595b6:::rbd_data.112a948d5956.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.8390392339999995, 2024-09-18T00:19:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088321282000000001, 2024-09-18T00:19:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:07.770 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22470 2024-09-18T00:19:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827560+0000", 2024-09-18T00:19:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.762e-06 2024-09-18T00:19:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827569+0000", 2024-09-18T00:19:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8740000000000001e-06 2024-09-18T00:19:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827585+0000", 2024-09-18T00:19:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5962000000000001e-05 2024-09-18T00:19:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827595+0000", 2024-09-18T00:19:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5910000000000002e-06 2024-09-18T00:19:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827811+0000", 2024-09-18T00:19:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000216858 2024-09-18T00:19:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:19:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827818+0000", 2024-09-18T00:19:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.99e-06 2024-09-18T00:19:07.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829799+0000", 2024-09-18T00:19:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019803849999999999 2024-09-18T00:19:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829884+0000", 2024-09-18T00:19:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4852999999999995e-05 2024-09-18T00:19:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915616+0000", 2024-09-18T00:19:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085732808999999993 2024-09-18T00:19:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915711+0000", 2024-09-18T00:19:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4578000000000001e-05 2024-09-18T00:19:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915880+0000", 2024-09-18T00:19:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016861999999999999 2024-09-18T00:19:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:07.775 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:07.775 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:07.776 INFO:teuthology.orchestra.run.smithi097.stdout:4291 read 0x375f24e thru 0x3763c10 (0x49c3 bytes) 2024-09-18T00:19:07.776 INFO:teuthology.orchestra.run.smithi097.stdout:4292 read 0x1e6d0b5 thru 0x1e7c799 (0xf6e5 bytes) 2024-09-18T00:19:07.777 INFO:teuthology.orchestra.run.smithi097.stdout:4294 write 0x67558fc thru 0x6760a3d (0xb142 bytes) 2024-09-18T00:19:07.777 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:19:07.778 INFO:teuthology.orchestra.run.smithi097.stdout:4295 read 0x65662a1 thru 0x656d4ca (0x722a bytes) 2024-09-18T00:19:07.778 INFO:teuthology.orchestra.run.smithi097.stdout:4296 read 0xad1386 thru 0xadadd0 (0x9a4b bytes) 2024-09-18T00:19:07.778 INFO:teuthology.orchestra.run.smithi097.stdout:4300 write 0x3b4968f thru 0x3b51e10 (0x8782 bytes) 2024-09-18T00:19:07.778 INFO:teuthology.orchestra.run.smithi097.stdout:4301 read 0x764126c thru 0x7641f16 (0xcab bytes) 2024-09-18T00:19:07.779 INFO:teuthology.orchestra.run.smithi097.stdout:4302 punch from 0x2330959 to 0x2339af1, (0x9198 bytes) 2024-09-18T00:19:07.779 INFO:teuthology.orchestra.run.smithi097.stdout:4303 read 0x25f24b5 thru 0x25f3317 (0xe63 bytes) 2024-09-18T00:19:07.779 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:07.779 INFO:teuthology.orchestra.run.smithi097.stdout:4305 clone 19 order 18 su 131072 sc 5 2024-09-18T00:19:07.779 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:07.774+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:07.846 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is set 2024-09-18T00:19:07.877 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:07.874+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:07.918 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:07.918 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:07.918 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:07.918 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:07.932 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:07.930+0000 7f73da6c37c0 -1 Falling back to public interface 2024-09-18T00:19:07.933 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:19:07.949 INFO:teuthology.orchestra.run.smithi097.stdout:truncating image image_client.0-clone18 from 0x836c3fa (overlap 0x126b772) to 0x0 2024-09-18T00:19:07.978 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:07.978+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:08.055 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-18T00:19:08.082 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:08.078+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:08.095 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:08.096 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:08.109 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:19:08.207 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:08.206+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:08.283 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:08.282+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:08.287 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:08.287 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2418 3.1a 3:59a8b40d:::rbd_data.112aa7ba4578.00000000000001b0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:19:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 47.549665978, 2024-09-18T00:19:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10488562999999999, 2024-09-18T00:19:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2418 2024-09-18T00:19:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735592+0000", 2024-09-18T00:19:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5229999999999998e-06 2024-09-18T00:19:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735604+0000", 2024-09-18T00:19:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2143e-05 2024-09-18T00:19:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735629+0000", 2024-09-18T00:19:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.497e-05 2024-09-18T00:19:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735648+0000", 2024-09-18T00:19:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8896999999999999e-05 2024-09-18T00:19:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735775+0000", 2024-09-18T00:19:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126935 2024-09-18T00:19:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.736118+0000", 2024-09-18T00:19:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034330800000000001 2024-09-18T00:19:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-18T00:19:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.741633+0000", 2024-09-18T00:19:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055144139999999996 2024-09-18T00:19:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840134+0000", 2024-09-18T00:19:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.098501618999999999 2024-09-18T00:19:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840184+0000", 2024-09-18T00:19:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9953000000000001e-05 2024-09-18T00:19:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840336+0000", 2024-09-18T00:19:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151476 2024-09-18T00:19:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840475+0000", 2024-09-18T00:19:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013939200000000001 2024-09-18T00:19:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2418 3.1a e37/23 ondisk, result = 0)", 2024-09-18T00:19:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:19:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 47.531076134999999, 2024-09-18T00:19:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.085979978999999998, 2024-09-18T00:19:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:19:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:19:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754181+0000", 2024-09-18T00:19:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8530000000000001e-06 2024-09-18T00:19:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754194+0000", 2024-09-18T00:19:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.292e-05 2024-09-18T00:19:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754230+0000", 2024-09-18T00:19:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5602999999999998e-05 2024-09-18T00:19:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754249+0000", 2024-09-18T00:19:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8850000000000001e-05 2024-09-18T00:19:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840070+0000", 2024-09-18T00:19:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.085821123999999999 2024-09-18T00:19:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840098+0000", 2024-09-18T00:19:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8546000000000001e-05 2024-09-18T00:19:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840159+0000", 2024-09-18T00:19:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1082999999999998e-05 2024-09-18T00:19:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12545 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:19:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.890013372999999, 2024-09-18T00:19:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045427642999999997, 2024-09-18T00:19:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12545 2024-09-18T00:19:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:19:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:19:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395243+0000", 2024-09-18T00:19:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.206e-06 2024-09-18T00:19:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395249+0000", 2024-09-18T00:19:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:19:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395270+0000", 2024-09-18T00:19:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1223e-05 2024-09-18T00:19:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395285+0000", 2024-09-18T00:19:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4725999999999999e-05 2024-09-18T00:19:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440296+0000", 2024-09-18T00:19:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045011402999999998 2024-09-18T00:19:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440453+0000", 2024-09-18T00:19:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015742299999999999 2024-09-18T00:19:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440670+0000", 2024-09-18T00:19:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000216302 2024-09-18T00:19:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12546 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:19:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.889951164999999, 2024-09-18T00:19:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045915171999999997, 2024-09-18T00:19:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12546 2024-09-18T00:19:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:19:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:19:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395306+0000", 2024-09-18T00:19:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.206e-06 2024-09-18T00:19:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395330+0000", 2024-09-18T00:19:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4686e-05 2024-09-18T00:19:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395356+0000", 2024-09-18T00:19:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5418999999999998e-05 2024-09-18T00:19:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395387+0000", 2024-09-18T00:19:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0942000000000003e-05 2024-09-18T00:19:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440743+0000", 2024-09-18T00:19:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045356592000000001 2024-09-18T00:19:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440880+0000", 2024-09-18T00:19:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013641700000000001 2024-09-18T00:19:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441219+0000", 2024-09-18T00:19:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033991000000000002 2024-09-18T00:19:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12547 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 212992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:19:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.889841831999998, 2024-09-18T00:19:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046163691999999999, 2024-09-18T00:19:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12547 2024-09-18T00:19:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:19:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:19:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395415+0000", 2024-09-18T00:19:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.553e-06 2024-09-18T00:19:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395426+0000", 2024-09-18T00:19:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0682999999999999e-05 2024-09-18T00:19:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395449+0000", 2024-09-18T00:19:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3082999999999999e-05 2024-09-18T00:19:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395469+0000", 2024-09-18T00:19:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0173000000000002e-05 2024-09-18T00:19:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441289+0000", 2024-09-18T00:19:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045820195000000001 2024-09-18T00:19:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441426+0000", 2024-09-18T00:19:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013712699999999999 2024-09-18T00:19:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441577+0000", 2024-09-18T00:19:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015087799999999999 2024-09-18T00:19:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12548 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:19:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.889768070999999, 2024-09-18T00:19:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046293932000000003, 2024-09-18T00:19:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12548 2024-09-18T00:19:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:19:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:19:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395489+0000", 2024-09-18T00:19:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.22e-06 2024-09-18T00:19:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395512+0000", 2024-09-18T00:19:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3312e-05 2024-09-18T00:19:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395621+0000", 2024-09-18T00:19:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010859 2024-09-18T00:19:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395689+0000", 2024-09-18T00:19:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8232000000000007e-05 2024-09-18T00:19:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441610+0000", 2024-09-18T00:19:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045920751000000003 2024-09-18T00:19:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441695+0000", 2024-09-18T00:19:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5379999999999999e-05 2024-09-18T00:19:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441781+0000", 2024-09-18T00:19:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6446999999999994e-05 2024-09-18T00:19:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12549 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 294912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:19:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.889507419000001, 2024-09-18T00:19:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046410264999999999, 2024-09-18T00:19:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12549 2024-09-18T00:19:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:19:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:19:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395750+0000", 2024-09-18T00:19:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:19:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395758+0000", 2024-09-18T00:19:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5259999999999993e-06 2024-09-18T00:19:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395793+0000", 2024-09-18T00:19:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4459000000000003e-05 2024-09-18T00:19:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395809+0000", 2024-09-18T00:19:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6433e-05 2024-09-18T00:19:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441851+0000", 2024-09-18T00:19:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046041493000000003 2024-09-18T00:19:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442002+0000", 2024-09-18T00:19:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151433 2024-09-18T00:19:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442158+0000", 2024-09-18T00:19:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000156358 2024-09-18T00:19:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12550 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:19:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.889395929999999, 2024-09-18T00:19:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046498398000000003, 2024-09-18T00:19:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12550 2024-09-18T00:19:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:19:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:19:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395861+0000", 2024-09-18T00:19:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.234e-06 2024-09-18T00:19:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395872+0000", 2024-09-18T00:19:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1409e-05 2024-09-18T00:19:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395892+0000", 2024-09-18T00:19:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9680000000000001e-05 2024-09-18T00:19:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395911+0000", 2024-09-18T00:19:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9283000000000002e-05 2024-09-18T00:19:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442210+0000", 2024-09-18T00:19:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046298372999999997 2024-09-18T00:19:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442279+0000", 2024-09-18T00:19:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9055999999999997e-05 2024-09-18T00:19:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442358+0000", 2024-09-18T00:19:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9363000000000006e-05 2024-09-18T00:19:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12551 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:19:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.888723477999999, 2024-09-18T00:19:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046066902999999999, 2024-09-18T00:19:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12551 2024-09-18T00:19:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:19:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:19:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396534+0000", 2024-09-18T00:19:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6869999999999999e-06 2024-09-18T00:19:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396543+0000", 2024-09-18T00:19:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3430000000000002e-06 2024-09-18T00:19:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396563+0000", 2024-09-18T00:19:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0063000000000001e-05 2024-09-18T00:19:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396579+0000", 2024-09-18T00:19:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5610000000000001e-05 2024-09-18T00:19:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442434+0000", 2024-09-18T00:19:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045855094999999998 2024-09-18T00:19:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442515+0000", 2024-09-18T00:19:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0988000000000005e-05 2024-09-18T00:19:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442599+0000", 2024-09-18T00:19:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4116999999999994e-05 2024-09-18T00:19:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12552 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:19:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.888648675999999, 2024-09-18T00:19:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046210804000000001, 2024-09-18T00:19:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12552 2024-09-18T00:19:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:19:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:19:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396608+0000", 2024-09-18T00:19:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4729999999999999e-06 2024-09-18T00:19:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396614+0000", 2024-09-18T00:19:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1e-06 2024-09-18T00:19:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396633+0000", 2024-09-18T00:19:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9009e-05 2024-09-18T00:19:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396650+0000", 2024-09-18T00:19:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7045999999999999e-05 2024-09-18T00:19:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442662+0000", 2024-09-18T00:19:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046011170999999997 2024-09-18T00:19:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442737+0000", 2024-09-18T00:19:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5759000000000004e-05 2024-09-18T00:19:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442818+0000", 2024-09-18T00:19:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0246000000000004e-05 2024-09-18T00:19:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12553 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:19:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.888585972000001, 2024-09-18T00:19:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046339454000000002, 2024-09-18T00:19:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12553 2024-09-18T00:19:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:19:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:19:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396671+0000", 2024-09-18T00:19:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.314e-06 2024-09-18T00:19:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396676+0000", 2024-09-18T00:19:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6659999999999996e-06 2024-09-18T00:19:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396695+0000", 2024-09-18T00:19:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8383e-05 2024-09-18T00:19:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396709+0000", 2024-09-18T00:19:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4309e-05 2024-09-18T00:19:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442871+0000", 2024-09-18T00:19:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046162295999999998 2024-09-18T00:19:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442939+0000", 2024-09-18T00:19:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7607999999999994e-05 2024-09-18T00:19:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443009+0000", 2024-09-18T00:19:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9877999999999997e-05 2024-09-18T00:19:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12554 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:19:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.888528050000001, 2024-09-18T00:19:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046504050999999998, 2024-09-18T00:19:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12554 2024-09-18T00:19:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:19:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:19:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396729+0000", 2024-09-18T00:19:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2559999999999999e-06 2024-09-18T00:19:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396736+0000", 2024-09-18T00:19:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6140000000000003e-06 2024-09-18T00:19:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396755+0000", 2024-09-18T00:19:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8221999999999999e-05 2024-09-18T00:19:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396769+0000", 2024-09-18T00:19:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4372999999999999e-05 2024-09-18T00:19:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443078+0000", 2024-09-18T00:19:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046308850999999998 2024-09-18T00:19:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443152+0000", 2024-09-18T00:19:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4083000000000002e-05 2024-09-18T00:19:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443231+0000", 2024-09-18T00:19:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9652000000000002e-05 2024-09-18T00:19:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12632 3.3 3:c21fac02:::rbd_data.112a7b1589a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1490944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:19:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.881969765000001, 2024-09-18T00:19:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045843916999999998, 2024-09-18T00:19:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12632 2024-09-18T00:19:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:19:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:19:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403288+0000", 2024-09-18T00:19:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3630000000000001e-06 2024-09-18T00:19:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403886+0000", 2024-09-18T00:19:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00059840100000000003 2024-09-18T00:19:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403913+0000", 2024-09-18T00:19:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6376e-05 2024-09-18T00:19:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403927+0000", 2024-09-18T00:19:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4188e-05 2024-09-18T00:19:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443268+0000", 2024-09-18T00:19:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.039340591000000001 2024-09-18T00:19:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443488+0000", 2024-09-18T00:19:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000220545 2024-09-18T00:19:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443918+0000", 2024-09-18T00:19:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043021800000000001 2024-09-18T00:19:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.448558+0000", 2024-09-18T00:19:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046391449999999999 2024-09-18T00:19:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449015+0000", 2024-09-18T00:19:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045766600000000002 2024-09-18T00:19:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449085+0000", 2024-09-18T00:19:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0191999999999995e-05 2024-09-18T00:19:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449130+0000", 2024-09-18T00:19:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4231999999999999e-05 2024-09-18T00:19:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:19386 3.1d 3:bcd89fc1:::rbd_data.112a7fe29ab4.00000000000000c8:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:53.699994+0000", 2024-09-18T00:19:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.585261622999999, 2024-09-18T00:19:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050613197999999998, 2024-09-18T00:19:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 19386 2024-09-18T00:19:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.699994+0000", 2024-09-18T00:19:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.699994+0000", 2024-09-18T00:19:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.699996+0000", 2024-09-18T00:19:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2330000000000001e-06 2024-09-18T00:19:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700007+0000", 2024-09-18T00:19:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1199000000000001e-05 2024-09-18T00:19:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700039+0000", 2024-09-18T00:19:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1362999999999998e-05 2024-09-18T00:19:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700054+0000", 2024-09-18T00:19:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4970000000000001e-05 2024-09-18T00:19:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700229+0000", 2024-09-18T00:19:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017526600000000001 2024-09-18T00:19:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700488+0000", 2024-09-18T00:19:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025918099999999998 2024-09-18T00:19:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750393+0000", 2024-09-18T00:19:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049904733 2024-09-18T00:19:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750476+0000", 2024-09-18T00:19:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3413999999999998e-05 2024-09-18T00:19:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750607+0000", 2024-09-18T00:19:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130839 2024-09-18T00:19:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:19387 3.1d 3:bc19744b:::rbd_data.112a7fe29ab4.00000000000000c9:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:53.700113+0000", 2024-09-18T00:19:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.585142212999999, 2024-09-18T00:19:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050693338999999997, 2024-09-18T00:19:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 19387 2024-09-18T00:19:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700113+0000", 2024-09-18T00:19:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700113+0000", 2024-09-18T00:19:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700115+0000", 2024-09-18T00:19:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3599999999999999e-06 2024-09-18T00:19:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700130+0000", 2024-09-18T00:19:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5396999999999998e-05 2024-09-18T00:19:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700147+0000", 2024-09-18T00:19:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7255e-05 2024-09-18T00:19:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700158+0000", 2024-09-18T00:19:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.079e-05 2024-09-18T00:19:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.707161+0000", 2024-09-18T00:19:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070025820000000003 2024-09-18T00:19:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.707407+0000", 2024-09-18T00:19:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024667100000000001 2024-09-18T00:19:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750638+0000", 2024-09-18T00:19:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043230365 2024-09-18T00:19:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750704+0000", 2024-09-18T00:19:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6806999999999999e-05 2024-09-18T00:19:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750807+0000", 2024-09-18T00:19:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102112 2024-09-18T00:19:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:22270 3.14 3:2badd31c:::rbd_data.112a948d5956.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.908651+0000", 2024-09-18T00:19:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.376604506, 2024-09-18T00:19:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.081875197999999996, 2024-09-18T00:19:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 22270 2024-09-18T00:19:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908651+0000", 2024-09-18T00:19:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908651+0000", 2024-09-18T00:19:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908653+0000", 2024-09-18T00:19:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2290000000000002e-06 2024-09-18T00:19:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908665+0000", 2024-09-18T00:19:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.225e-05 2024-09-18T00:19:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908697+0000", 2024-09-18T00:19:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1306000000000001e-05 2024-09-18T00:19:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908712+0000", 2024-09-18T00:19:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5466000000000001e-05 2024-09-18T00:19:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908889+0000", 2024-09-18T00:19:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017653999999999999 2024-09-18T00:19:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.909023+0000", 2024-09-18T00:19:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013473700000000001 2024-09-18T00:19:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.913637+0000", 2024-09-18T00:19:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046132600000000001 2024-09-18T00:19:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990060+0000", 2024-09-18T00:19:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.076422834999999995 2024-09-18T00:19:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990206+0000", 2024-09-18T00:19:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014591499999999999 2024-09-18T00:19:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990391+0000", 2024-09-18T00:19:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018554300000000001 2024-09-18T00:19:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990526+0000", 2024-09-18T00:19:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000135117 2024-09-18T00:19:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:22270 3.14 e64/32 ondisk, result = 0)", 2024-09-18T00:19:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.926819+0000", 2024-09-18T00:19:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.358436242, 2024-09-18T00:19:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.063725764000000004, 2024-09-18T00:19:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926819+0000", 2024-09-18T00:19:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926819+0000", 2024-09-18T00:19:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926821+0000", 2024-09-18T00:19:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5540000000000001e-06 2024-09-18T00:19:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926828+0000", 2024-09-18T00:19:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9260000000000003e-06 2024-09-18T00:19:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926857+0000", 2024-09-18T00:19:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9357e-05 2024-09-18T00:19:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926878+0000", 2024-09-18T00:19:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0571999999999999e-05 2024-09-18T00:19:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990155+0000", 2024-09-18T00:19:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.063277618999999993 2024-09-18T00:19:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990187+0000", 2024-09-18T00:19:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1322999999999998e-05 2024-09-18T00:19:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990545+0000", 2024-09-18T00:19:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00035841300000000003 2024-09-18T00:19:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:22272 3.3 3:c617f123:::rbd_data.112a948d5956.0000000000000009:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.294157779000001, 2024-09-18T00:19:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10752577300000001, 2024-09-18T00:19:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 22272 2024-09-18T00:19:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991102+0000", 2024-09-18T00:19:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9369999999999997e-06 2024-09-18T00:19:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991118+0000", 2024-09-18T00:19:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6396e-05 2024-09-18T00:19:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991150+0000", 2024-09-18T00:19:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2132999999999999e-05 2024-09-18T00:19:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991170+0000", 2024-09-18T00:19:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0085000000000002e-05 2024-09-18T00:19:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991370+0000", 2024-09-18T00:19:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019963700000000001 2024-09-18T00:19:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991518+0000", 2024-09-18T00:19:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000148424 2024-09-18T00:19:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.998850+0000", 2024-09-18T00:19:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073315480000000002 2024-09-18T00:19:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098178+0000", 2024-09-18T00:19:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.099328067000000006 2024-09-18T00:19:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098314+0000", 2024-09-18T00:19:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000135678 2024-09-18T00:19:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098498+0000", 2024-09-18T00:19:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018479600000000001 2024-09-18T00:19:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098623+0000", 2024-09-18T00:19:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000125072 2024-09-18T00:19:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:22272 3.3 e64/23 ondisk, result = 0)", 2024-09-18T00:19:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:58.023226+0000", 2024-09-18T00:19:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.262029483999999, 2024-09-18T00:19:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.075416677000000001, 2024-09-18T00:19:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023226+0000", 2024-09-18T00:19:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023226+0000", 2024-09-18T00:19:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023228+0000", 2024-09-18T00:19:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0600000000000002e-06 2024-09-18T00:19:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023237+0000", 2024-09-18T00:19:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6430000000000001e-06 2024-09-18T00:19:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023271+0000", 2024-09-18T00:19:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4091999999999998e-05 2024-09-18T00:19:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023293+0000", 2024-09-18T00:19:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2103000000000002e-05 2024-09-18T00:19:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098272+0000", 2024-09-18T00:19:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.074979263000000004 2024-09-18T00:19:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098297+0000", 2024-09-18T00:19:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5162999999999999e-05 2024-09-18T00:19:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098643+0000", 2024-09-18T00:19:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034535300000000001 2024-09-18T00:19:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:23689 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-18T00:19:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.4328330009999997, 2024-09-18T00:19:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9562625809999998, 2024-09-18T00:19:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 23689 2024-09-18T00:19:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852423+0000", 2024-09-18T00:19:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9999999999999995e-07 2024-09-18T00:19:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852429+0000", 2024-09-18T00:19:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3279999999999996e-06 2024-09-18T00:19:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852442+0000", 2024-09-18T00:19:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3052e-05 2024-09-18T00:19:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852448+0000", 2024-09-18T00:19:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2779999999999999e-06 2024-09-18T00:19:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852523+0000", 2024-09-18T00:19:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4463999999999997e-05 2024-09-18T00:19:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852538+0000", 2024-09-18T00:19:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5466999999999999e-05 2024-09-18T00:19:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793478+0000", 2024-09-18T00:19:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.94094012699999996 2024-09-18T00:19:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793494+0000", 2024-09-18T00:19:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5807999999999998e-05 2024-09-18T00:19:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.804875+0000", 2024-09-18T00:19:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0113811939999999 2024-09-18T00:19:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805359+0000", 2024-09-18T00:19:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000484248 2024-09-18T00:19:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-18T00:19:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805872+0000", 2024-09-18T00:19:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00051218100000000001 2024-09-18T00:19:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.807790+0000", 2024-09-18T00:19:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001918763 2024-09-18T00:19:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808242+0000", 2024-09-18T00:19:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045126200000000002 2024-09-18T00:19:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808428+0000", 2024-09-18T00:19:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018633000000000001 2024-09-18T00:19:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808685+0000", 2024-09-18T00:19:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025707900000000001 2024-09-18T00:19:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.386 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:08.386 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:08.386 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:08.386 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:19:08.388 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:08.386+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:08.446 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:19:08.493 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:08.489+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:08.514 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:08.514 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:08.514 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:08.514 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:08.528 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:19:08.531 INFO:teuthology.orchestra.run.smithi097.stdout:4307 read 0x8317943 thru 0x83213c2 (0x9a80 bytes) 2024-09-18T00:19:08.547 INFO:teuthology.orchestra.run.smithi097.stdout:4309 writesame 0xa2904be thru 0xa2962bc data_size 0xd(0x5dff bytes) 2024-09-18T00:19:08.547 INFO:teuthology.orchestra.run.smithi097.stdout:4310 read 0x3f73a27 thru 0x3f79f06 (0x64e0 bytes) 2024-09-18T00:19:08.548 INFO:teuthology.orchestra.run.smithi097.stdout:4311 read 0x422d95 thru 0x428c32 (0x5e9e bytes) 2024-09-18T00:19:08.548 INFO:teuthology.orchestra.run.smithi097.stdout:4312 write 0x84c918e thru 0x84d3d4d (0xabc0 bytes) 2024-09-18T00:19:08.553 INFO:teuthology.orchestra.run.smithi097.stdout:4313 read 0x56c0c7e thru 0x56d08cf (0xfc52 bytes) 2024-09-18T00:19:08.554 INFO:teuthology.orchestra.run.smithi097.stdout:4314 write 0x3a16769 thru 0x3a24f45 (0xe7dd bytes) 2024-09-18T00:19:08.577 INFO:teuthology.orchestra.run.smithi097.stdout:4317 write 0xe4176af thru 0xe422259 (0xabab bytes) 2024-09-18T00:19:08.582 INFO:teuthology.orchestra.run.smithi097.stdout:4318 punch from 0x3596906 to 0x35993ab, (0x2aa5 bytes) 2024-09-18T00:19:08.583 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:08.583 INFO:teuthology.orchestra.run.smithi097.stdout:4320 writesame 0x4603e83 thru 0x4604578 data_size 0x21(0x6f6 bytes) 2024-09-18T00:19:08.585 INFO:teuthology.orchestra.run.smithi097.stdout:4322 write 0x914462e thru 0x914cb1d (0x84f0 bytes) 2024-09-18T00:19:08.590 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:08.586+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:08.593 INFO:teuthology.orchestra.run.smithi097.stdout:4323 writesame 0x5d715e4 thru 0x5d73d71 data_size 0x3d(0x278e bytes) 2024-09-18T00:19:08.595 INFO:teuthology.orchestra.run.smithi097.stdout:4324 write 0x432a0cb thru 0x432b032 (0xf68 bytes) 2024-09-18T00:19:08.599 INFO:teuthology.orchestra.run.smithi097.stdout:4325 punch from 0xc005f0f to 0xc00d303, (0x73f4 bytes) 2024-09-18T00:19:08.599 INFO:teuthology.orchestra.run.smithi097.stdout:4327 read 0xa60917a thru 0xa60aec1 (0x1d48 bytes) 2024-09-18T00:19:08.599 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:08.599 INFO:teuthology.orchestra.run.smithi097.stdout:4329 write 0x8de43e2 thru 0x8deaa3a (0x6659 bytes) 2024-09-18T00:19:08.600 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:08.601 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:08.601 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:08.603 INFO:teuthology.orchestra.run.smithi097.stdout:4330 writesame 0x5146064 thru 0x51558d7 data_size 0x2(0xf874 bytes) 2024-09-18T00:19:08.618 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:08.618 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:08.635 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:08.635 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:08.654 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:08.654 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:08.655 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:08.656 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:08.656 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-18T00:19:08.674 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:19:08.675 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:08.676 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:08.680 INFO:teuthology.orchestra.run.smithi097.stdout:4331 read 0xd45b40b thru 0xd45c184 (0xd7a bytes) 2024-09-18T00:19:08.680 INFO:teuthology.orchestra.run.smithi097.stdout:4332 write 0x66c5c90 thru 0x66c91d2 (0x3543 bytes) 2024-09-18T00:19:08.684 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:08.684 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:08.690 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:08.685+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:08.696 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:08.696 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:08.699 INFO:teuthology.orchestra.run.smithi097.stdout:4337 trunc from 0xe42225a to 0x3d08500 2024-09-18T00:19:08.703 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:08.703 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:08.717 INFO:teuthology.orchestra.run.smithi097.stdout:4342 punch from 0xaceea2 to 0xad4e0e, (0x5f6c bytes) 2024-09-18T00:19:08.717 INFO:teuthology.orchestra.run.smithi097.stdout:4343 read 0x3742d91 thru 0x3750dfe (0xe06e bytes) 2024-09-18T00:19:08.717 INFO:teuthology.orchestra.run.smithi097.stdout:4345 read 0x3e2919 thru 0x3ef212 (0xc8fa bytes) 2024-09-18T00:19:08.727 INFO:teuthology.orchestra.run.smithi097.stdout:4346 write 0x6165994 thru 0x6174e40 (0xf4ad bytes) 2024-09-18T00:19:08.738 INFO:teuthology.orchestra.run.smithi097.stdout:4347 trunc from 0x6174e41 to 0x2baff19 2024-09-18T00:19:08.752 INFO:teuthology.orchestra.run.smithi097.stdout:4349 punch from 0x2490b43 to 0x24962c5, (0x5782 bytes) 2024-09-18T00:19:08.773 INFO:teuthology.orchestra.run.smithi097.stdout:4355 write 0xa1edc6b thru 0xa1fcca6 (0xf03c bytes) 2024-09-18T00:19:08.777 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:08.791 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:08.785+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:08.795 INFO:teuthology.orchestra.run.smithi097.stdout:4357 writesame 0xdcfed55 thru 0xdd0d782 data_size 0x37(0xea2e bytes) 2024-09-18T00:19:08.795 INFO:teuthology.orchestra.run.smithi097.stdout:4358 writesame 0x9554742 thru 0x956041f data_size 0x19(0xbcde bytes) 2024-09-18T00:19:08.802 INFO:teuthology.orchestra.run.smithi097.stdout:4359 read 0x89548bd thru 0x8958e36 (0x457a bytes) 2024-09-18T00:19:08.803 INFO:teuthology.orchestra.run.smithi097.stdout:4360 write 0x90aa0df thru 0x90afe58 (0x5d7a bytes) 2024-09-18T00:19:08.808 INFO:teuthology.orchestra.run.smithi097.stdout:4361 write 0x7fd3cfe thru 0x7fd6917 (0x2c1a bytes) 2024-09-18T00:19:08.813 INFO:teuthology.orchestra.run.smithi097.stdout:4362 write 0xaa6b04 thru 0xaa8574 (0x1a71 bytes) 2024-09-18T00:19:08.819 INFO:teuthology.orchestra.run.smithi097.stdout:4364 punch from 0xc08fd8e to 0xc090e9a, (0x110c bytes) 2024-09-18T00:19:08.819 INFO:teuthology.orchestra.run.smithi097.stdout:4365 punch from 0xb580975 to 0xb58bb6a, (0xb1f5 bytes) 2024-09-18T00:19:08.819 INFO:teuthology.orchestra.run.smithi097.stdout:4366 read 0x36d92bb thru 0x36dffac (0x6cf2 bytes) 2024-09-18T00:19:08.819 INFO:teuthology.orchestra.run.smithi097.stdout:4367 read 0x60b84b4 thru 0x60c2700 (0xa24d bytes) 2024-09-18T00:19:08.819 INFO:teuthology.orchestra.run.smithi097.stdout:4368 writesame 0x81f14cc thru 0x81f8be6 data_size 0x1(0x771b bytes) 2024-09-18T00:19:08.850 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:08.850 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:08.867 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:19:08.897 INFO:teuthology.orchestra.run.smithi097.stdout:4369 write 0x807835a thru 0x80867ff (0xe4a6 bytes) 2024-09-18T00:19:08.910 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:08.910+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:08.938 INFO:teuthology.orchestra.run.smithi097.stdout:4370 trunc from 0xdd0d783 to 0x16732e4 2024-09-18T00:19:08.967 INFO:teuthology.orchestra.run.smithi097.stdout:4371 read 0xe9dbd0 thru 0xea4e3c (0x726d bytes) 2024-09-18T00:19:08.969 INFO:teuthology.orchestra.run.smithi097.stdout:4374 read 0x142a9e8 thru 0x1434bc1 (0xa1da bytes) 2024-09-18T00:19:08.970 INFO:teuthology.orchestra.run.smithi097.stdout:4375 punch from 0x11be2ac to 0x11c2a6a, (0x47be bytes) 2024-09-18T00:19:08.990 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:08.991 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:08.992 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:08.990+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:08.996 INFO:teuthology.orchestra.run.smithi097.stdout:4377 write 0x6f78522 thru 0x6f7ad46 (0x2825 bytes) 2024-09-18T00:19:09.005 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:19:09.008 INFO:teuthology.orchestra.run.smithi097.stdout:4380 trunc from 0x6f7ad47 to 0x44f0e2d 2024-09-18T00:19:09.028 INFO:teuthology.orchestra.run.smithi097.stdout:4381 write 0x2915689 thru 0x2920727 (0xb09f bytes) 2024-09-18T00:19:09.041 INFO:teuthology.orchestra.run.smithi097.stdout:4383 write 0x22ef0aa thru 0x22f082e (0x1785 bytes) 2024-09-18T00:19:09.081 INFO:teuthology.orchestra.run.smithi097.stdout:4384 writesame 0xd711ef4 thru 0xd720576 data_size 0x1(0xe683 bytes) 2024-09-18T00:19:09.093 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:09.089+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:09.106 INFO:teuthology.orchestra.run.smithi097.stdout:4385 trunc from 0xd720577 to 0x22c57ca 2024-09-18T00:19:09.153 INFO:teuthology.orchestra.run.smithi097.stdout:4387 writesame 0x9836277 thru 0x9843410 data_size 0x63(0xd19a bytes) 2024-09-18T00:19:09.153 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:09.170 INFO:teuthology.orchestra.run.smithi097.stdout:4389 write 0xb9f0819 thru 0xb9fc837 (0xc01f bytes) 2024-09-18T00:19:09.174 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:09.174 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:09.176 INFO:teuthology.orchestra.run.smithi097.stdout:4390 read 0x9399cd8 thru 0x939c950 (0x2c79 bytes) 2024-09-18T00:19:09.176 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:09.176 INFO:teuthology.orchestra.run.smithi097.stdout:4392 write 0xd47b83 thru 0xd47efb (0x379 bytes) 2024-09-18T00:19:09.182 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:09.188 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:19:09.193 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:09.189+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:09.194 INFO:teuthology.orchestra.run.smithi097.stdout:4394 trunc from 0xb9fc838 to 0x76b626b 2024-09-18T00:19:09.218 INFO:teuthology.orchestra.run.smithi097.stdout:4396 trunc from 0x76b626b to 0x9b27b0e 2024-09-18T00:19:09.226 INFO:teuthology.orchestra.run.smithi097.stdout:4397 read 0x57a1eea thru 0x57a9aea (0x7c01 bytes) 2024-09-18T00:19:09.242 INFO:teuthology.orchestra.run.smithi097.stdout:4398 write 0xc0ec0cc thru 0xc0f08b7 (0x47ec bytes) 2024-09-18T00:19:09.248 INFO:teuthology.orchestra.run.smithi097.stdout:4399 trunc from 0xc0f08b8 to 0xc924269 2024-09-18T00:19:09.257 INFO:teuthology.orchestra.run.smithi097.stdout:4400 read 0x4d9a78b thru 0x4d9b380 (0xbf6 bytes) 2024-09-18T00:19:09.257 INFO:teuthology.orchestra.run.smithi097.stdout:4401 read 0x3fdba06 thru 0x3fe1367 (0x5962 bytes) 2024-09-18T00:19:09.257 INFO:teuthology.orchestra.run.smithi097.stdout:4402 punch from 0x9903505 to 0x990cf51, (0x9a4c bytes) 2024-09-18T00:19:09.257 INFO:teuthology.orchestra.run.smithi097.stdout:4405 write 0xb5cbbc9 thru 0xb5d9bdc (0xe014 bytes) 2024-09-18T00:19:09.268 INFO:teuthology.orchestra.run.smithi097.stdout:4407 read 0x4282adb thru 0x428d63a (0xab60 bytes) 2024-09-18T00:19:09.269 INFO:teuthology.orchestra.run.smithi097.stdout:4408 read 0x19f5a85 thru 0x19f6121 (0x69d bytes) 2024-09-18T00:19:09.269 INFO:teuthology.orchestra.run.smithi097.stdout:4409 write 0x7e75a95 thru 0x7e77148 (0x16b4 bytes) 2024-09-18T00:19:09.273 INFO:teuthology.orchestra.run.smithi097.stdout:4410 write 0x389c724 thru 0x38a4d6b (0x8648 bytes) 2024-09-18T00:19:09.284 INFO:teuthology.orchestra.run.smithi097.stdout:4411 writesame 0x93eccda thru 0x93f6358 data_size 0x1(0x967f bytes) 2024-09-18T00:19:09.294 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:09.294+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:09.310 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:09.310 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:09.325 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:19:09.395 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:09.389+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:09.416 INFO:teuthology.orchestra.run.smithi097.stdout:4412 write 0x3b90e2a thru 0x3b99363 (0x853a bytes) 2024-09-18T00:19:09.424 INFO:teuthology.orchestra.run.smithi097.stdout:4413 read 0x881dd24 thru 0x88201d0 (0x24ad bytes) 2024-09-18T00:19:09.439 INFO:teuthology.orchestra.run.smithi097.stdout:4414 write 0xddb3c23 thru 0xddc2f94 (0xf372 bytes) 2024-09-18T00:19:09.448 INFO:teuthology.orchestra.run.smithi097.stdout:4415 punch from 0x79a6ef4 to 0x79b50aa, (0xe1b6 bytes) 2024-09-18T00:19:09.448 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:09.448 INFO:teuthology.orchestra.run.smithi097.stdout:4419 writesame 0x35b030 thru 0x3698c9 data_size 0x13(0xe89a bytes) 2024-09-18T00:19:09.458 INFO:teuthology.orchestra.run.smithi097.stdout:4420 write 0x3fbed62 thru 0x3fcbd32 (0xcfd1 bytes) 2024-09-18T00:19:09.460 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:09.460 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:09.471 INFO:teuthology.orchestra.run.smithi097.stdout:4421 write 0x63bc23a thru 0x63c86e8 (0xc4af bytes) 2024-09-18T00:19:09.477 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:19:09.482 INFO:teuthology.orchestra.run.smithi097.stdout:4422 punch from 0x91f5666 to 0x91f67fb, (0x1195 bytes) 2024-09-18T00:19:09.482 INFO:teuthology.orchestra.run.smithi097.stdout:4423 write 0x4e9bd76 thru 0x4eaa09a (0xe325 bytes) 2024-09-18T00:19:09.495 INFO:teuthology.orchestra.run.smithi097.stdout:4424 read 0xd38bd37 thru 0xd396821 (0xaaeb bytes) 2024-09-18T00:19:09.495 INFO:teuthology.orchestra.run.smithi097.stdout:4425 write 0x5a6e5f0 thru 0x5a743e8 (0x5df9 bytes) 2024-09-18T00:19:09.495 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:09.489+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:09.498 INFO:teuthology.orchestra.run.smithi097.stdout:4426 read 0x8b65403 thru 0x8b6ba8e (0x668c bytes) 2024-09-18T00:19:09.517 INFO:teuthology.orchestra.run.smithi097.stdout:4427 trunc from 0xddc2f95 to 0x3faed49 2024-09-18T00:19:09.552 INFO:teuthology.orchestra.run.smithi097.stdout:4428 writesame 0x689ca2c thru 0x68a0bf5 data_size 0x15(0x41ca bytes) 2024-09-18T00:19:09.552 INFO:teuthology.orchestra.run.smithi097.stdout:4430 punch from 0x30870e4 to 0x308b7f8, (0x4714 bytes) 2024-09-18T00:19:09.552 INFO:teuthology.orchestra.run.smithi097.stdout:4431 read 0x4d18f2b thru 0x4d1c835 (0x390b bytes) 2024-09-18T00:19:09.552 INFO:teuthology.orchestra.run.smithi097.stdout:4434 write 0xb8215 thru 0xba3fa (0x21e6 bytes) 2024-09-18T00:19:09.572 INFO:teuthology.orchestra.run.smithi097.stdout:4435 writesame 0x2e65614 thru 0x2e6c877 data_size 0x2(0x7264 bytes) 2024-09-18T00:19:09.592 INFO:teuthology.orchestra.run.smithi097.stdout:4437 punch from 0x2a4842e to 0x2a4b401, (0x2fd3 bytes) 2024-09-18T00:19:09.592 INFO:teuthology.orchestra.run.smithi097.stdout:4438 writesame 0x41a5b7b thru 0x41b403a data_size 0xb7(0xe4c0 bytes) 2024-09-18T00:19:09.596 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:09.594+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:09.615 INFO:teuthology.orchestra.run.smithi097.stdout:4440 write 0x7619ba6 thru 0x761f58a (0x59e5 bytes) 2024-09-18T00:19:09.620 INFO:teuthology.orchestra.run.smithi097.stdout:4441 punch from 0x2752d35 to 0x275ac93, (0x7f5e bytes) 2024-09-18T00:19:09.631 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:09.631 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:09.637 INFO:teuthology.orchestra.run.smithi097.stdout:4443 writesame 0xa5cfef1 thru 0xa5d6fd3 data_size 0x13(0x70e3 bytes) 2024-09-18T00:19:09.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:09.652 INFO:teuthology.orchestra.run.smithi097.stdout:4445 writesame 0xc25fe7e thru 0xc267dd0 data_size 0x35(0x7f53 bytes) 2024-09-18T00:19:09.655 INFO:teuthology.orchestra.run.smithi097.stdout:4446 trunc from 0xc267dd1 to 0xe2d2ba0 2024-09-18T00:19:09.666 INFO:teuthology.orchestra.run.smithi097.stdout:4447 write 0x15edf62 thru 0x15f92d1 (0xb370 bytes) 2024-09-18T00:19:09.677 INFO:teuthology.orchestra.run.smithi097.stdout:4448 punch from 0x712960a to 0x7137f25, (0xe91b bytes) 2024-09-18T00:19:09.677 INFO:teuthology.orchestra.run.smithi097.stdout:4449 write 0x8a679bc thru 0x8a6be72 (0x44b7 bytes) 2024-09-18T00:19:09.684 INFO:teuthology.orchestra.run.smithi097.stdout:4451 read 0xb78deb6 thru 0xb79672d (0x8878 bytes) 2024-09-18T00:19:09.684 INFO:teuthology.orchestra.run.smithi097.stdout:4452 punch from 0x5caa2a4 to 0x5cad178, (0x2ed4 bytes) 2024-09-18T00:19:09.685 INFO:teuthology.orchestra.run.smithi097.stdout:4454 punch from 0xde6de42 to 0xde7ab67, (0xcd25 bytes) 2024-09-18T00:19:09.685 INFO:teuthology.orchestra.run.smithi097.stdout:4456 read 0xae34d70 thru 0xae3f77c (0xaa0d bytes) 2024-09-18T00:19:09.685 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:09.697 INFO:teuthology.orchestra.run.smithi097.stdout:4459 write 0xe66ffea thru 0xe6709de (0x9f5 bytes) 2024-09-18T00:19:09.697 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:09.694+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:09.701 INFO:teuthology.orchestra.run.smithi097.stdout:4462 punch from 0xc3a9ac8 to 0xc3adf63, (0x449b bytes) 2024-09-18T00:19:09.701 INFO:teuthology.orchestra.run.smithi097.stdout:4464 punch from 0x4726eac to 0x472e0a8, (0x71fc bytes) 2024-09-18T00:19:09.702 INFO:teuthology.orchestra.run.smithi097.stdout:4465 write 0xc693d5a thru 0xc6991c8 (0x546f bytes) 2024-09-18T00:19:09.720 INFO:teuthology.orchestra.run.smithi097.stdout:4466 write 0xed5d362 thru 0xed5d816 (0x4b5 bytes) 2024-09-18T00:19:09.725 INFO:teuthology.orchestra.run.smithi097.stdout:4467 read 0x60268bf thru 0x60365e8 (0xfd2a bytes) 2024-09-18T00:19:09.725 INFO:teuthology.orchestra.run.smithi097.stdout:4469 write 0x47db6f4 thru 0x47df11e (0x3a2b bytes) 2024-09-18T00:19:09.729 INFO:teuthology.orchestra.run.smithi097.stdout:4471 writesame 0x43d027c thru 0x43d7bdb data_size 0x10(0x7960 bytes) 2024-09-18T00:19:09.735 INFO:teuthology.orchestra.run.smithi097.stdout:4472 read 0x33de10f thru 0x33e64b5 (0x83a7 bytes) 2024-09-18T00:19:09.735 INFO:teuthology.orchestra.run.smithi097.stdout:4473 write 0x9fe3ff6 thru 0x9feb49d (0x74a8 bytes) 2024-09-18T00:19:09.742 INFO:teuthology.orchestra.run.smithi097.stdout:4474 writesame 0xacd5516 thru 0xace5295 data_size 0x40(0xfd80 bytes) 2024-09-18T00:19:09.750 INFO:teuthology.orchestra.run.smithi097.stdout:4475 write 0x93f95e6 thru 0x940872c (0xf147 bytes) 2024-09-18T00:19:09.762 INFO:teuthology.orchestra.run.smithi097.stdout:4476 read 0xe08434b thru 0xe090bcc (0xc882 bytes) 2024-09-18T00:19:09.762 INFO:teuthology.orchestra.run.smithi097.stdout:4477 writesame 0xb8bedbc thru 0xb8ca6bc data_size 0x1(0xb901 bytes) 2024-09-18T00:19:09.780 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:09.780 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:09.795 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:19:09.798 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:09.798+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:09.874 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:19:09.910 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:09.910 INFO:teuthology.orchestra.run.smithi097.stdout:4479 writesame 0x62733d6 thru 0x62793cb data_size 0x47(0x5ff6 bytes) 2024-09-18T00:19:09.910 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:09.905+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:09.916 INFO:teuthology.orchestra.run.smithi097.stdout:4480 punch from 0x1ce075c to 0x1ce5adf, (0x5383 bytes) 2024-09-18T00:19:09.916 INFO:teuthology.orchestra.run.smithi097.stdout:4481 read 0x449d262 thru 0x44a53dc (0x817b bytes) 2024-09-18T00:19:09.916 INFO:teuthology.orchestra.run.smithi097.stdout:4482 read 0x6e67b42 thru 0x6e70bfd (0x90bc bytes) 2024-09-18T00:19:09.916 INFO:teuthology.orchestra.run.smithi097.stdout:4483 read 0x1bf5f2 thru 0x1c4296 (0x4ca5 bytes) 2024-09-18T00:19:09.917 INFO:teuthology.orchestra.run.smithi097.stdout:4484 read 0x8dfa65c thru 0x8dfe049 (0x39ee bytes) 2024-09-18T00:19:09.917 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:09.917 INFO:teuthology.orchestra.run.smithi097.stdout:4489 write 0x5970f8e thru 0x597be79 (0xaeec bytes) 2024-09-18T00:19:09.923 INFO:teuthology.orchestra.run.smithi097.stdout:4490 punch from 0x63abf5c to 0x63bb464, (0xf508 bytes) 2024-09-18T00:19:09.937 INFO:teuthology.orchestra.run.smithi097.stdout:4492 trunc from 0xed5d817 to 0x622df6b 2024-09-18T00:19:09.958 INFO:teuthology.orchestra.run.smithi097.stdout:4493 write 0x2efbf9c thru 0x2f05cdb (0x9d40 bytes) 2024-09-18T00:19:09.964 INFO:teuthology.orchestra.run.smithi097.stdout:4494 write 0x566071c thru 0x566baa1 (0xb386 bytes) 2024-09-18T00:19:09.967 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:09.967 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:09.983 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:19:09.985 INFO:teuthology.orchestra.run.smithi097.stdout:4496 trunc from 0x622df6b to 0xaef6abe 2024-09-18T00:19:09.996 INFO:teuthology.orchestra.run.smithi097.stdout:4501 writesame 0x599301d thru 0x59a0a61 data_size 0x1(0xda45 bytes) 2024-09-18T00:19:10.000 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:09.998+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:10.102 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:10.098+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:10.104 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:10.104 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:10.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:19:10.148 INFO:teuthology.orchestra.run.smithi097.stdout:4502 read 0x4cb67a thru 0x4d68f0 (0xb277 bytes) 2024-09-18T00:19:10.149 INFO:teuthology.orchestra.run.smithi097.stdout:4504 writesame 0x77d9cf5 thru 0x77dda6e data_size 0x2b(0x3d7a bytes) 2024-09-18T00:19:10.154 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:10.154 INFO:teuthology.orchestra.run.smithi097.stdout:4507 writesame 0x1d0b1f0 thru 0x1d177a8 data_size 0x7(0xc5b9 bytes) 2024-09-18T00:19:10.175 INFO:teuthology.orchestra.run.smithi097.stdout:4508 write 0x756e5df thru 0x757cb76 (0xe598 bytes) 2024-09-18T00:19:10.203 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:10.197+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:10.204 INFO:teuthology.orchestra.run.smithi097.stdout:4509 trunc from 0xaef6abe to 0xa2cc8c 2024-09-18T00:19:10.246 INFO:teuthology.orchestra.run.smithi097.stdout:4510 trunc from 0xa2cc8c to 0x78d4c4c 2024-09-18T00:19:10.255 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is unset 2024-09-18T00:19:10.287 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:10.287 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:10.287 INFO:teuthology.orchestra.run.smithi097.stdout:4511 write 0xd77755e thru 0xd77784c (0x2ef bytes) 2024-09-18T00:19:10.292 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:10.292 INFO:teuthology.orchestra.run.smithi097.stdout:4513 writesame 0xc277a54 thru 0xc27cfa9 data_size 0x21(0x5556 bytes) 2024-09-18T00:19:10.302 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:19:10.314 INFO:teuthology.orchestra.run.smithi097.stdout:4514 trunc from 0xd77784d to 0x2148416 2024-09-18T00:19:10.314 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:10.309+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:10.345 INFO:teuthology.orchestra.run.smithi097.stdout:4515 writesame 0x26e4d97 thru 0x26ea6ba data_size 0x23(0x5924 bytes) 2024-09-18T00:19:10.345 INFO:teuthology.orchestra.run.smithi097.stdout:4516 read 0x2048c27 thru 0x2048f7f (0x359 bytes) 2024-09-18T00:19:10.345 INFO:teuthology.orchestra.run.smithi097.stdout:4517 writesame 0x1c145c3 thru 0x1c162f9 data_size 0x9(0x1d37 bytes) 2024-09-18T00:19:10.372 INFO:teuthology.orchestra.run.smithi097.stdout:4518 writesame 0xa4ec5f7 thru 0xa4ed652 data_size 0x6(0x105c bytes) 2024-09-18T00:19:10.372 INFO:teuthology.orchestra.run.smithi097.stdout:4520 write 0xa4293a0 thru 0xa42d577 (0x41d8 bytes) 2024-09-18T00:19:10.376 INFO:teuthology.orchestra.run.smithi097.stdout:4522 read 0x17cb7d4 thru 0x17d68c4 (0xb0f1 bytes) 2024-09-18T00:19:10.376 INFO:teuthology.orchestra.run.smithi097.stdout:4523 punch from 0x3f60f94 to 0x3f6a97a, (0x99e6 bytes) 2024-09-18T00:19:10.376 INFO:teuthology.orchestra.run.smithi097.stdout:4524 read 0x58bf06e thru 0x58c092c (0x18bf bytes) 2024-09-18T00:19:10.376 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:10.376 INFO:teuthology.orchestra.run.smithi097.stdout:4528 write 0x3a0ea2b thru 0x3a151e8 (0x67be bytes) 2024-09-18T00:19:10.386 INFO:teuthology.orchestra.run.smithi097.stdout:4529 trunc from 0xa4ed653 to 0x5cd8554 2024-09-18T00:19:10.404 INFO:teuthology.orchestra.run.smithi097.stdout:4530 write 0x246c918 thru 0x24700c0 (0x37a9 bytes) 2024-09-18T00:19:10.404 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:10.397+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:10.427 INFO:teuthology.orchestra.run.smithi097.stdout:4532 writesame 0x812c276 thru 0x812ce43 data_size 0x1(0xbce bytes) 2024-09-18T00:19:10.429 INFO:teuthology.orchestra.run.smithi097.stdout:4534 trunc from 0x812ce44 to 0x6f82df7 2024-09-18T00:19:10.441 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:10.441 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:10.442 INFO:teuthology.orchestra.run.smithi097.stdout:4536 write 0x4cddd9e thru 0x4ce76ef (0x9952 bytes) 2024-09-18T00:19:10.451 INFO:teuthology.orchestra.run.smithi097.stdout:4537 writesame 0x50aa649 thru 0x50ac16b data_size 0x1(0x1b23 bytes) 2024-09-18T00:19:10.457 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:19:10.473 INFO:teuthology.orchestra.run.smithi097.stdout:4538 trunc from 0x6f82df7 to 0x1dcbc73 2024-09-18T00:19:10.516 INFO:teuthology.orchestra.run.smithi097.stdout:4539 writesame 0x5d70efc thru 0x5d74bb5 data_size 0x3(0x3cba bytes) 2024-09-18T00:19:10.517 INFO:teuthology.orchestra.run.smithi097.stdout:4540 punch from 0x1aaa93f to 0x1aac431, (0x1af2 bytes) 2024-09-18T00:19:10.517 INFO:teuthology.orchestra.run.smithi097.stdout:4541 clone 20 order 19 su 2048 sc 14 2024-09-18T00:19:10.517 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:10.513+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:10.597 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:10.597 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:10.606 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:10.602+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:10.612 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:19:10.705 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:19:10.706 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:10.701+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:10.776 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:10.777 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:10.792 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:19:10.838 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:10.838+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:10.847 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:10.847 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:10.848 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:10.864 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:10.864 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:10.880 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:10.897 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:10.898 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:10.898 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-18T00:19:10.907 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:10.901+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:10.915 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:10.915 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:10.925 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:10.926 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:10.926 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:10.926 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:10.932 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:10.933 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:10.936 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:10.937 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:10.940 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:19:11.008 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:11.001+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:11.105 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:11.105 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:11.105 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:11.106 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:11.106 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:11.109 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:11.106+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:11.116 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:19:11.209 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:11.206+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:11.288 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:11.288 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:11.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:11.288 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:11.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.289 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2418 3.1a 3:59a8b40d:::rbd_data.112aa7ba4578.00000000000001b0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:19:11.289 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:11.289 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.548674628999997, 2024-09-18T00:19:11.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10488562999999999, 2024-09-18T00:19:11.289 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.289 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.289 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:11.289 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.289 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.289 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.289 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2418 2024-09-18T00:19:11.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.289 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:11.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:11.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735592+0000", 2024-09-18T00:19:11.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5229999999999998e-06 2024-09-18T00:19:11.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735604+0000", 2024-09-18T00:19:11.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2143e-05 2024-09-18T00:19:11.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735629+0000", 2024-09-18T00:19:11.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.497e-05 2024-09-18T00:19:11.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735648+0000", 2024-09-18T00:19:11.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8896999999999999e-05 2024-09-18T00:19:11.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735775+0000", 2024-09-18T00:19:11.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126935 2024-09-18T00:19:11.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.736118+0000", 2024-09-18T00:19:11.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034330800000000001 2024-09-18T00:19:11.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-18T00:19:11.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.741633+0000", 2024-09-18T00:19:11.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055144139999999996 2024-09-18T00:19:11.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:11.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840134+0000", 2024-09-18T00:19:11.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.098501618999999999 2024-09-18T00:19:11.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:11.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840184+0000", 2024-09-18T00:19:11.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9953000000000001e-05 2024-09-18T00:19:11.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:11.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840336+0000", 2024-09-18T00:19:11.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151476 2024-09-18T00:19:11.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840475+0000", 2024-09-18T00:19:11.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013939200000000001 2024-09-18T00:19:11.294 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.294 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.294 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.294 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2418 3.1a e37/23 ondisk, result = 0)", 2024-09-18T00:19:11.294 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:19:11.295 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.530084786000003, 2024-09-18T00:19:11.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.085979978999999998, 2024-09-18T00:19:11.295 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.295 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.295 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.295 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:19:11.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:19:11.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754181+0000", 2024-09-18T00:19:11.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8530000000000001e-06 2024-09-18T00:19:11.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754194+0000", 2024-09-18T00:19:11.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.292e-05 2024-09-18T00:19:11.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754230+0000", 2024-09-18T00:19:11.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5602999999999998e-05 2024-09-18T00:19:11.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754249+0000", 2024-09-18T00:19:11.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8850000000000001e-05 2024-09-18T00:19:11.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840070+0000", 2024-09-18T00:19:11.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.085821123999999999 2024-09-18T00:19:11.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840098+0000", 2024-09-18T00:19:11.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8546000000000001e-05 2024-09-18T00:19:11.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840159+0000", 2024-09-18T00:19:11.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1082999999999998e-05 2024-09-18T00:19:11.298 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.298 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.299 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.299 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12545 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.299 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:19:11.299 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 34.889022023999999, 2024-09-18T00:19:11.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045427642999999997, 2024-09-18T00:19:11.299 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.299 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.299 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.299 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.299 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.299 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.300 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12545 2024-09-18T00:19:11.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.300 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:19:11.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:19:11.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395243+0000", 2024-09-18T00:19:11.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.206e-06 2024-09-18T00:19:11.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395249+0000", 2024-09-18T00:19:11.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:19:11.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395270+0000", 2024-09-18T00:19:11.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1223e-05 2024-09-18T00:19:11.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395285+0000", 2024-09-18T00:19:11.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4725999999999999e-05 2024-09-18T00:19:11.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440296+0000", 2024-09-18T00:19:11.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045011402999999998 2024-09-18T00:19:11.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440453+0000", 2024-09-18T00:19:11.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015742299999999999 2024-09-18T00:19:11.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440670+0000", 2024-09-18T00:19:11.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000216302 2024-09-18T00:19:11.303 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.303 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.303 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12546 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:19:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 34.888959816000003, 2024-09-18T00:19:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045915171999999997, 2024-09-18T00:19:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12546 2024-09-18T00:19:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:19:11.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:19:11.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395306+0000", 2024-09-18T00:19:11.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.206e-06 2024-09-18T00:19:11.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395330+0000", 2024-09-18T00:19:11.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4686e-05 2024-09-18T00:19:11.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395356+0000", 2024-09-18T00:19:11.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5418999999999998e-05 2024-09-18T00:19:11.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395387+0000", 2024-09-18T00:19:11.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0942000000000003e-05 2024-09-18T00:19:11.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440743+0000", 2024-09-18T00:19:11.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045356592000000001 2024-09-18T00:19:11.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440880+0000", 2024-09-18T00:19:11.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013641700000000001 2024-09-18T00:19:11.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441219+0000", 2024-09-18T00:19:11.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033991000000000002 2024-09-18T00:19:11.309 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.309 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.309 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.309 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12547 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 212992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.309 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:19:11.309 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 34.888850482999999, 2024-09-18T00:19:11.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046163691999999999, 2024-09-18T00:19:11.309 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.309 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.309 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.309 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.310 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.310 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.310 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12547 2024-09-18T00:19:11.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.310 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:19:11.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:19:11.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395415+0000", 2024-09-18T00:19:11.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.553e-06 2024-09-18T00:19:11.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395426+0000", 2024-09-18T00:19:11.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0682999999999999e-05 2024-09-18T00:19:11.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395449+0000", 2024-09-18T00:19:11.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3082999999999999e-05 2024-09-18T00:19:11.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395469+0000", 2024-09-18T00:19:11.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0173000000000002e-05 2024-09-18T00:19:11.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441289+0000", 2024-09-18T00:19:11.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045820195000000001 2024-09-18T00:19:11.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441426+0000", 2024-09-18T00:19:11.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013712699999999999 2024-09-18T00:19:11.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441577+0000", 2024-09-18T00:19:11.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015087799999999999 2024-09-18T00:19:11.313 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.313 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.313 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.313 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12548 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.313 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:19:11.314 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 34.888776722000003, 2024-09-18T00:19:11.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046293932000000003, 2024-09-18T00:19:11.314 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.314 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.314 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.314 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.314 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.314 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.314 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12548 2024-09-18T00:19:11.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.314 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:19:11.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:19:11.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395489+0000", 2024-09-18T00:19:11.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.22e-06 2024-09-18T00:19:11.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395512+0000", 2024-09-18T00:19:11.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3312e-05 2024-09-18T00:19:11.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395621+0000", 2024-09-18T00:19:11.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010859 2024-09-18T00:19:11.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395689+0000", 2024-09-18T00:19:11.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8232000000000007e-05 2024-09-18T00:19:11.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441610+0000", 2024-09-18T00:19:11.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045920751000000003 2024-09-18T00:19:11.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441695+0000", 2024-09-18T00:19:11.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5379999999999999e-05 2024-09-18T00:19:11.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441781+0000", 2024-09-18T00:19:11.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6446999999999994e-05 2024-09-18T00:19:11.318 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.318 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.318 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.318 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12549 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 294912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.318 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:19:11.318 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 34.888516070000001, 2024-09-18T00:19:11.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046410264999999999, 2024-09-18T00:19:11.318 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.318 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.318 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.318 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.319 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.319 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.319 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12549 2024-09-18T00:19:11.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.319 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:19:11.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:19:11.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395750+0000", 2024-09-18T00:19:11.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:19:11.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395758+0000", 2024-09-18T00:19:11.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5259999999999993e-06 2024-09-18T00:19:11.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395793+0000", 2024-09-18T00:19:11.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4459000000000003e-05 2024-09-18T00:19:11.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395809+0000", 2024-09-18T00:19:11.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6433e-05 2024-09-18T00:19:11.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441851+0000", 2024-09-18T00:19:11.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046041493000000003 2024-09-18T00:19:11.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442002+0000", 2024-09-18T00:19:11.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151433 2024-09-18T00:19:11.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442158+0000", 2024-09-18T00:19:11.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000156358 2024-09-18T00:19:11.322 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.322 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.322 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.322 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12550 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.322 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:19:11.322 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 34.888404581000003, 2024-09-18T00:19:11.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046498398000000003, 2024-09-18T00:19:11.323 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.323 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.323 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.323 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.323 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.323 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.323 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12550 2024-09-18T00:19:11.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.323 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:19:11.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:19:11.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395861+0000", 2024-09-18T00:19:11.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.234e-06 2024-09-18T00:19:11.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395872+0000", 2024-09-18T00:19:11.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1409e-05 2024-09-18T00:19:11.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395892+0000", 2024-09-18T00:19:11.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9680000000000001e-05 2024-09-18T00:19:11.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395911+0000", 2024-09-18T00:19:11.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9283000000000002e-05 2024-09-18T00:19:11.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442210+0000", 2024-09-18T00:19:11.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046298372999999997 2024-09-18T00:19:11.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442279+0000", 2024-09-18T00:19:11.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9055999999999997e-05 2024-09-18T00:19:11.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442358+0000", 2024-09-18T00:19:11.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9363000000000006e-05 2024-09-18T00:19:11.327 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.327 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.327 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.327 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12551 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.327 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:19:11.327 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 34.887732129, 2024-09-18T00:19:11.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046066902999999999, 2024-09-18T00:19:11.327 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.327 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.327 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.328 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.328 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.328 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.328 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12551 2024-09-18T00:19:11.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.328 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:19:11.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:19:11.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396534+0000", 2024-09-18T00:19:11.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6869999999999999e-06 2024-09-18T00:19:11.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396543+0000", 2024-09-18T00:19:11.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3430000000000002e-06 2024-09-18T00:19:11.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396563+0000", 2024-09-18T00:19:11.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0063000000000001e-05 2024-09-18T00:19:11.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396579+0000", 2024-09-18T00:19:11.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5610000000000001e-05 2024-09-18T00:19:11.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442434+0000", 2024-09-18T00:19:11.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045855094999999998 2024-09-18T00:19:11.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442515+0000", 2024-09-18T00:19:11.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0988000000000005e-05 2024-09-18T00:19:11.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442599+0000", 2024-09-18T00:19:11.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4116999999999994e-05 2024-09-18T00:19:11.331 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.331 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.331 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.332 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12552 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.332 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:19:11.332 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 34.887657326999999, 2024-09-18T00:19:11.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046210804000000001, 2024-09-18T00:19:11.332 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.332 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.332 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.332 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.332 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.332 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.332 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12552 2024-09-18T00:19:11.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.332 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:19:11.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:19:11.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396608+0000", 2024-09-18T00:19:11.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4729999999999999e-06 2024-09-18T00:19:11.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396614+0000", 2024-09-18T00:19:11.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1e-06 2024-09-18T00:19:11.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396633+0000", 2024-09-18T00:19:11.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9009e-05 2024-09-18T00:19:11.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396650+0000", 2024-09-18T00:19:11.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7045999999999999e-05 2024-09-18T00:19:11.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442662+0000", 2024-09-18T00:19:11.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046011170999999997 2024-09-18T00:19:11.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442737+0000", 2024-09-18T00:19:11.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5759000000000004e-05 2024-09-18T00:19:11.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442818+0000", 2024-09-18T00:19:11.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0246000000000004e-05 2024-09-18T00:19:11.336 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.336 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.336 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.336 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12553 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.336 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:19:11.336 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 34.887594622999998, 2024-09-18T00:19:11.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046339454000000002, 2024-09-18T00:19:11.337 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.337 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.337 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.337 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.337 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.337 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.337 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12553 2024-09-18T00:19:11.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.337 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.337 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.337 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:19:11.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:19:11.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396671+0000", 2024-09-18T00:19:11.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.314e-06 2024-09-18T00:19:11.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396676+0000", 2024-09-18T00:19:11.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6659999999999996e-06 2024-09-18T00:19:11.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396695+0000", 2024-09-18T00:19:11.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8383e-05 2024-09-18T00:19:11.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396709+0000", 2024-09-18T00:19:11.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4309e-05 2024-09-18T00:19:11.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442871+0000", 2024-09-18T00:19:11.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046162295999999998 2024-09-18T00:19:11.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442939+0000", 2024-09-18T00:19:11.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7607999999999994e-05 2024-09-18T00:19:11.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443009+0000", 2024-09-18T00:19:11.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9877999999999997e-05 2024-09-18T00:19:11.340 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.340 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.340 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.341 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12554 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.341 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:19:11.341 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 34.887536701000002, 2024-09-18T00:19:11.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046504050999999998, 2024-09-18T00:19:11.341 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.341 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.341 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.341 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.341 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.341 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.341 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12554 2024-09-18T00:19:11.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.342 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:19:11.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:19:11.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396729+0000", 2024-09-18T00:19:11.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2559999999999999e-06 2024-09-18T00:19:11.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396736+0000", 2024-09-18T00:19:11.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6140000000000003e-06 2024-09-18T00:19:11.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396755+0000", 2024-09-18T00:19:11.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8221999999999999e-05 2024-09-18T00:19:11.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396769+0000", 2024-09-18T00:19:11.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4372999999999999e-05 2024-09-18T00:19:11.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443078+0000", 2024-09-18T00:19:11.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046308850999999998 2024-09-18T00:19:11.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443152+0000", 2024-09-18T00:19:11.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4083000000000002e-05 2024-09-18T00:19:11.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443231+0000", 2024-09-18T00:19:11.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9652000000000002e-05 2024-09-18T00:19:11.345 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.345 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.345 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.345 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12632 3.3 3:c21fac02:::rbd_data.112a7b1589a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1490944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.345 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:19:11.345 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 34.880978415999998, 2024-09-18T00:19:11.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045843916999999998, 2024-09-18T00:19:11.346 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.346 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.346 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:11.346 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.346 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.346 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.346 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12632 2024-09-18T00:19:11.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.346 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:19:11.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:19:11.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403288+0000", 2024-09-18T00:19:11.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3630000000000001e-06 2024-09-18T00:19:11.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403886+0000", 2024-09-18T00:19:11.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00059840100000000003 2024-09-18T00:19:11.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403913+0000", 2024-09-18T00:19:11.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6376e-05 2024-09-18T00:19:11.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403927+0000", 2024-09-18T00:19:11.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4188e-05 2024-09-18T00:19:11.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443268+0000", 2024-09-18T00:19:11.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.039340591000000001 2024-09-18T00:19:11.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443488+0000", 2024-09-18T00:19:11.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000220545 2024-09-18T00:19:11.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:11.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443918+0000", 2024-09-18T00:19:11.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043021800000000001 2024-09-18T00:19:11.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:11.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.448558+0000", 2024-09-18T00:19:11.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046391449999999999 2024-09-18T00:19:11.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:11.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449015+0000", 2024-09-18T00:19:11.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045766600000000002 2024-09-18T00:19:11.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:11.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449085+0000", 2024-09-18T00:19:11.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0191999999999995e-05 2024-09-18T00:19:11.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449130+0000", 2024-09-18T00:19:11.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4231999999999999e-05 2024-09-18T00:19:11.351 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.351 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.351 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.352 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:19386 3.1d 3:bcd89fc1:::rbd_data.112a7fe29ab4.00000000000000c8:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:11.352 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:53.699994+0000", 2024-09-18T00:19:11.352 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.584270274000001, 2024-09-18T00:19:11.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050613197999999998, 2024-09-18T00:19:11.352 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.352 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.352 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:11.352 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.352 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.352 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.352 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 19386 2024-09-18T00:19:11.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.352 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.699994+0000", 2024-09-18T00:19:11.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.699994+0000", 2024-09-18T00:19:11.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.699996+0000", 2024-09-18T00:19:11.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2330000000000001e-06 2024-09-18T00:19:11.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700007+0000", 2024-09-18T00:19:11.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1199000000000001e-05 2024-09-18T00:19:11.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700039+0000", 2024-09-18T00:19:11.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1362999999999998e-05 2024-09-18T00:19:11.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700054+0000", 2024-09-18T00:19:11.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4970000000000001e-05 2024-09-18T00:19:11.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700229+0000", 2024-09-18T00:19:11.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017526600000000001 2024-09-18T00:19:11.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700488+0000", 2024-09-18T00:19:11.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025918099999999998 2024-09-18T00:19:11.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:11.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750393+0000", 2024-09-18T00:19:11.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049904733 2024-09-18T00:19:11.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:11.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750476+0000", 2024-09-18T00:19:11.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3413999999999998e-05 2024-09-18T00:19:11.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750607+0000", 2024-09-18T00:19:11.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130839 2024-09-18T00:19:11.356 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.357 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.357 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.357 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:19387 3.1d 3:bc19744b:::rbd_data.112a7fe29ab4.00000000000000c9:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:11.357 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:53.700113+0000", 2024-09-18T00:19:11.357 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.584150864000001, 2024-09-18T00:19:11.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050693338999999997, 2024-09-18T00:19:11.357 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.357 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.357 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:11.357 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.357 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.357 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.358 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 19387 2024-09-18T00:19:11.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.358 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700113+0000", 2024-09-18T00:19:11.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700113+0000", 2024-09-18T00:19:11.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700115+0000", 2024-09-18T00:19:11.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3599999999999999e-06 2024-09-18T00:19:11.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700130+0000", 2024-09-18T00:19:11.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5396999999999998e-05 2024-09-18T00:19:11.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700147+0000", 2024-09-18T00:19:11.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7255e-05 2024-09-18T00:19:11.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700158+0000", 2024-09-18T00:19:11.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.079e-05 2024-09-18T00:19:11.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.707161+0000", 2024-09-18T00:19:11.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070025820000000003 2024-09-18T00:19:11.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.707407+0000", 2024-09-18T00:19:11.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024667100000000001 2024-09-18T00:19:11.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:11.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750638+0000", 2024-09-18T00:19:11.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043230365 2024-09-18T00:19:11.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:11.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750704+0000", 2024-09-18T00:19:11.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6806999999999999e-05 2024-09-18T00:19:11.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750807+0000", 2024-09-18T00:19:11.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102112 2024-09-18T00:19:11.362 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.362 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.362 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.362 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:22270 3.14 3:2badd31c:::rbd_data.112a948d5956.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:11.362 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.908651+0000", 2024-09-18T00:19:11.362 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.375613157, 2024-09-18T00:19:11.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.081875197999999996, 2024-09-18T00:19:11.363 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.363 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.363 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:11.363 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.363 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.363 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.363 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 22270 2024-09-18T00:19:11.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.364 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908651+0000", 2024-09-18T00:19:11.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908651+0000", 2024-09-18T00:19:11.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908653+0000", 2024-09-18T00:19:11.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2290000000000002e-06 2024-09-18T00:19:11.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908665+0000", 2024-09-18T00:19:11.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.225e-05 2024-09-18T00:19:11.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908697+0000", 2024-09-18T00:19:11.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1306000000000001e-05 2024-09-18T00:19:11.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908712+0000", 2024-09-18T00:19:11.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5466000000000001e-05 2024-09-18T00:19:11.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908889+0000", 2024-09-18T00:19:11.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017653999999999999 2024-09-18T00:19:11.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.909023+0000", 2024-09-18T00:19:11.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013473700000000001 2024-09-18T00:19:11.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:11.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.913637+0000", 2024-09-18T00:19:11.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046132600000000001 2024-09-18T00:19:11.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:11.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990060+0000", 2024-09-18T00:19:11.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.076422834999999995 2024-09-18T00:19:11.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:11.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990206+0000", 2024-09-18T00:19:11.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014591499999999999 2024-09-18T00:19:11.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:11.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990391+0000", 2024-09-18T00:19:11.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018554300000000001 2024-09-18T00:19:11.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990526+0000", 2024-09-18T00:19:11.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000135117 2024-09-18T00:19:11.368 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.368 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.369 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.369 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:22270 3.14 e64/32 ondisk, result = 0)", 2024-09-18T00:19:11.369 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.926819+0000", 2024-09-18T00:19:11.369 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.357444893, 2024-09-18T00:19:11.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.063725764000000004, 2024-09-18T00:19:11.369 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.369 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.369 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.369 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926819+0000", 2024-09-18T00:19:11.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926819+0000", 2024-09-18T00:19:11.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926821+0000", 2024-09-18T00:19:11.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5540000000000001e-06 2024-09-18T00:19:11.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926828+0000", 2024-09-18T00:19:11.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9260000000000003e-06 2024-09-18T00:19:11.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926857+0000", 2024-09-18T00:19:11.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9357e-05 2024-09-18T00:19:11.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926878+0000", 2024-09-18T00:19:11.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0571999999999999e-05 2024-09-18T00:19:11.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990155+0000", 2024-09-18T00:19:11.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.063277618999999993 2024-09-18T00:19:11.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990187+0000", 2024-09-18T00:19:11.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1322999999999998e-05 2024-09-18T00:19:11.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990545+0000", 2024-09-18T00:19:11.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00035841300000000003 2024-09-18T00:19:11.373 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.373 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.373 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.373 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:22272 3.3 3:c617f123:::rbd_data.112a948d5956.0000000000000009:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:11.373 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:11.373 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.293166429999999, 2024-09-18T00:19:11.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10752577300000001, 2024-09-18T00:19:11.373 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.373 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.373 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:11.374 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.374 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.374 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.374 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 22272 2024-09-18T00:19:11.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.374 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:11.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:11.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991102+0000", 2024-09-18T00:19:11.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9369999999999997e-06 2024-09-18T00:19:11.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991118+0000", 2024-09-18T00:19:11.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6396e-05 2024-09-18T00:19:11.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991150+0000", 2024-09-18T00:19:11.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2132999999999999e-05 2024-09-18T00:19:11.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991170+0000", 2024-09-18T00:19:11.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0085000000000002e-05 2024-09-18T00:19:11.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991370+0000", 2024-09-18T00:19:11.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019963700000000001 2024-09-18T00:19:11.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991518+0000", 2024-09-18T00:19:11.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000148424 2024-09-18T00:19:11.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:11.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.998850+0000", 2024-09-18T00:19:11.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073315480000000002 2024-09-18T00:19:11.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:11.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098178+0000", 2024-09-18T00:19:11.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.099328067000000006 2024-09-18T00:19:11.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:11.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098314+0000", 2024-09-18T00:19:11.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000135678 2024-09-18T00:19:11.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:11.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098498+0000", 2024-09-18T00:19:11.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018479600000000001 2024-09-18T00:19:11.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098623+0000", 2024-09-18T00:19:11.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000125072 2024-09-18T00:19:11.380 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.380 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.380 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.380 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:22272 3.3 e64/23 ondisk, result = 0)", 2024-09-18T00:19:11.380 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:58.023226+0000", 2024-09-18T00:19:11.380 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.261038135, 2024-09-18T00:19:11.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.075416677000000001, 2024-09-18T00:19:11.380 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.380 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.380 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.380 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023226+0000", 2024-09-18T00:19:11.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023226+0000", 2024-09-18T00:19:11.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023228+0000", 2024-09-18T00:19:11.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0600000000000002e-06 2024-09-18T00:19:11.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023237+0000", 2024-09-18T00:19:11.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6430000000000001e-06 2024-09-18T00:19:11.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023271+0000", 2024-09-18T00:19:11.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4091999999999998e-05 2024-09-18T00:19:11.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023293+0000", 2024-09-18T00:19:11.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2103000000000002e-05 2024-09-18T00:19:11.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098272+0000", 2024-09-18T00:19:11.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.074979263000000004 2024-09-18T00:19:11.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098297+0000", 2024-09-18T00:19:11.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5162999999999999e-05 2024-09-18T00:19:11.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098643+0000", 2024-09-18T00:19:11.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034535300000000001 2024-09-18T00:19:11.384 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.384 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.384 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.384 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:23689 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-18T00:19:11.384 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:11.384 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.4318416519999992, 2024-09-18T00:19:11.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9562625809999998, 2024-09-18T00:19:11.384 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.384 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.385 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:11.385 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.385 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.385 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.385 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 23689 2024-09-18T00:19:11.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.385 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:11.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852423+0000", 2024-09-18T00:19:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9999999999999995e-07 2024-09-18T00:19:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852429+0000", 2024-09-18T00:19:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3279999999999996e-06 2024-09-18T00:19:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852442+0000", 2024-09-18T00:19:11.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3052e-05 2024-09-18T00:19:11.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852448+0000", 2024-09-18T00:19:11.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2779999999999999e-06 2024-09-18T00:19:11.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852523+0000", 2024-09-18T00:19:11.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4463999999999997e-05 2024-09-18T00:19:11.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:11.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852538+0000", 2024-09-18T00:19:11.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5466999999999999e-05 2024-09-18T00:19:11.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793478+0000", 2024-09-18T00:19:11.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.94094012699999996 2024-09-18T00:19:11.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:11.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793494+0000", 2024-09-18T00:19:11.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5807999999999998e-05 2024-09-18T00:19:11.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.804875+0000", 2024-09-18T00:19:11.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0113811939999999 2024-09-18T00:19:11.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805359+0000", 2024-09-18T00:19:11.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000484248 2024-09-18T00:19:11.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-18T00:19:11.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805872+0000", 2024-09-18T00:19:11.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00051218100000000001 2024-09-18T00:19:11.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:11.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.807790+0000", 2024-09-18T00:19:11.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001918763 2024-09-18T00:19:11.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:11.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808242+0000", 2024-09-18T00:19:11.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045126200000000002 2024-09-18T00:19:11.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:11.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808428+0000", 2024-09-18T00:19:11.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018633000000000001 2024-09-18T00:19:11.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808685+0000", 2024-09-18T00:19:11.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025707900000000001 2024-09-18T00:19:11.391 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.391 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.391 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.391 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.391 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.391 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:11.393 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:19:11.393 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:11.390+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:11.442 INFO:teuthology.orchestra.run.smithi097.stdout:truncating image image_client.0-clone19 from 0x5d74bb6 (overlap 0xa2cc8c) to 0x823d3c 2024-09-18T00:19:11.494 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:11.489+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:11.565 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:11.565 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:11.565 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:11.565 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:11.579 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:19:11.610 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-18T00:19:11.618 INFO:teuthology.orchestra.run.smithi097.stdout:4542 trunc from 0x5d74bb6 to 0x409819b 2024-09-18T00:19:11.622 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:11.622+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:11.696 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:11.694+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:11.708 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:11.708 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:11.708 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:11.708 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:11.708 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:11.709 INFO:teuthology.orchestra.run.smithi097.stdout:4543 trunc from 0x409819b to 0xce653cf 2024-09-18T00:19:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:4545 read 0xb143be4 thru 0xb14cb87 (0x8fa4 bytes) 2024-09-18T00:19:11.723 INFO:teuthology.orchestra.run.smithi097.stdout:4546 trunc from 0xce653cf to 0xd5f5ebb 2024-09-18T00:19:11.725 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:19:11.736 INFO:teuthology.orchestra.run.smithi097.stdout:4547 read 0x5975978 thru 0x597e66c (0x8cf5 bytes) 2024-09-18T00:19:11.744 INFO:teuthology.orchestra.run.smithi097.stdout:4548 trunc from 0xd5f5ebb to 0xaa8bcc4 2024-09-18T00:19:11.757 INFO:teuthology.orchestra.run.smithi097.stdout:4550 writesame 0x4393e0d thru 0x439c754 data_size 0x8(0x8948 bytes) 2024-09-18T00:19:11.768 INFO:teuthology.orchestra.run.smithi097.stdout:4551 write 0x5a8c8ea thru 0x5a97a04 (0xb11b bytes) 2024-09-18T00:19:11.797 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:11.793+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:11.801 INFO:teuthology.orchestra.run.smithi097.stdout:4552 punch from 0x5f99a2d to 0x5f9fbd6, (0x61a9 bytes) 2024-09-18T00:19:11.801 INFO:teuthology.orchestra.run.smithi097.stdout:4553 punch from 0xefa564 to 0xefbd1f, (0x17bb bytes) 2024-09-18T00:19:11.801 INFO:teuthology.orchestra.run.smithi097.stdout:4554 write 0x367ed17 thru 0x368e81e (0xfb08 bytes) 2024-09-18T00:19:11.835 INFO:teuthology.orchestra.run.smithi097.stdout:4555 write 0x38995af thru 0x38a58b4 (0xc306 bytes) 2024-09-18T00:19:11.866 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:11.866 INFO:teuthology.orchestra.run.smithi097.stdout:4557 write 0x6703d7b thru 0x670face (0xbd54 bytes) 2024-09-18T00:19:11.887 INFO:teuthology.orchestra.run.smithi097.stdout:4558 write 0x6713e56 thru 0x6717d2d (0x3ed8 bytes) 2024-09-18T00:19:11.890 INFO:teuthology.orchestra.run.smithi097.stdout:4559 write 0x611cf5e thru 0x612cc65 (0xfd08 bytes) 2024-09-18T00:19:11.896 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:11.896 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:11.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:11.897 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:11.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.897 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2229 3.10 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:11.897 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:11.897 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.631260834000003, 2024-09-18T00:19:11.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048730086999999998, 2024-09-18T00:19:11.897 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.897 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.897 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:11.897 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.897 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.897 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.898 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2229 2024-09-18T00:19:11.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.898 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:11.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:11.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263362+0000", 2024-09-18T00:19:11.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3869999999999999e-06 2024-09-18T00:19:11.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263381+0000", 2024-09-18T00:19:11.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9153000000000001e-05 2024-09-18T00:19:11.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263418+0000", 2024-09-18T00:19:11.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6346000000000001e-05 2024-09-18T00:19:11.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263435+0000", 2024-09-18T00:19:11.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7309999999999999e-05 2024-09-18T00:19:11.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263589+0000", 2024-09-18T00:19:11.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015451199999999999 2024-09-18T00:19:11.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263902+0000", 2024-09-18T00:19:11.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031291799999999997 2024-09-18T00:19:11.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:11.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.265995+0000", 2024-09-18T00:19:11.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020931809999999999 2024-09-18T00:19:11.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:11.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311788+0000", 2024-09-18T00:19:11.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045792078 2024-09-18T00:19:11.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:11.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311903+0000", 2024-09-18T00:19:11.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011510699999999999 2024-09-18T00:19:11.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:11.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312009+0000", 2024-09-18T00:19:11.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010630599999999999 2024-09-18T00:19:11.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312089+0000", 2024-09-18T00:19:11.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9789000000000005e-05 2024-09-18T00:19:11.903 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.903 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.903 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.903 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2229 3.10 e36/23 ondisk, result = 0)", 2024-09-18T00:19:11.903 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:11.903 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.612115175, 2024-09-18T00:19:11.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029600955000000002, 2024-09-18T00:19:11.903 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.903 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.903 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.904 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:11.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:11.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282507+0000", 2024-09-18T00:19:11.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9160000000000001e-06 2024-09-18T00:19:11.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282522+0000", 2024-09-18T00:19:11.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5e-05 2024-09-18T00:19:11.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282558+0000", 2024-09-18T00:19:11.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5694000000000001e-05 2024-09-18T00:19:11.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282580+0000", 2024-09-18T00:19:11.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2323e-05 2024-09-18T00:19:11.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311873+0000", 2024-09-18T00:19:11.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029293013999999999 2024-09-18T00:19:11.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311892+0000", 2024-09-18T00:19:11.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8556e-05 2024-09-18T00:19:11.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312105+0000", 2024-09-18T00:19:11.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021345199999999999 2024-09-18T00:19:11.907 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.907 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.907 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.907 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5714 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:11.907 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:11.907 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.584412671999999, 2024-09-18T00:19:11.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027448727999999999, 2024-09-18T00:19:11.908 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.908 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.908 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:11.908 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.908 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.908 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.908 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5714 2024-09-18T00:19:11.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.908 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:11.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:11.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310211+0000", 2024-09-18T00:19:11.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5639999999999998e-06 2024-09-18T00:19:11.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312678+0000", 2024-09-18T00:19:11.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002466761 2024-09-18T00:19:11.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312746+0000", 2024-09-18T00:19:11.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8046000000000005e-05 2024-09-18T00:19:11.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312774+0000", 2024-09-18T00:19:11.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7813e-05 2024-09-18T00:19:11.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312927+0000", 2024-09-18T00:19:11.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015328499999999999 2024-09-18T00:19:11.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313214+0000", 2024-09-18T00:19:11.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028683099999999998 2024-09-18T00:19:11.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:11.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313708+0000", 2024-09-18T00:19:11.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049379100000000004 2024-09-18T00:19:11.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:11.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336895+0000", 2024-09-18T00:19:11.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023186801 2024-09-18T00:19:11.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:11.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337454+0000", 2024-09-18T00:19:11.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055979399999999996 2024-09-18T00:19:11.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:11.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337571+0000", 2024-09-18T00:19:11.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116821 2024-09-18T00:19:11.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337656+0000", 2024-09-18T00:19:11.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4221000000000006e-05 2024-09-18T00:19:11.913 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.913 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.913 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.913 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6801 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:11.914 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:11.914 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.857576223999999, 2024-09-18T00:19:11.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051953602000000002, 2024-09-18T00:19:11.914 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.914 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.914 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:11.914 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.914 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.914 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.914 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6801 2024-09-18T00:19:11.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.914 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:11.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:11.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037046+0000", 2024-09-18T00:19:11.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.19e-06 2024-09-18T00:19:11.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037062+0000", 2024-09-18T00:19:11.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586999999999998e-05 2024-09-18T00:19:11.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037096+0000", 2024-09-18T00:19:11.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4112000000000001e-05 2024-09-18T00:19:11.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037120+0000", 2024-09-18T00:19:11.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.355e-05 2024-09-18T00:19:11.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037246+0000", 2024-09-18T00:19:11.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012652200000000001 2024-09-18T00:19:11.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037385+0000", 2024-09-18T00:19:11.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013924599999999999 2024-09-18T00:19:11.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:11.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.040881+0000", 2024-09-18T00:19:11.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034954830000000002 2024-09-18T00:19:11.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:11.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088794+0000", 2024-09-18T00:19:11.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047912897000000003 2024-09-18T00:19:11.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:11.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088828+0000", 2024-09-18T00:19:11.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3880000000000001e-05 2024-09-18T00:19:11.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:11.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088916+0000", 2024-09-18T00:19:11.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.865e-05 2024-09-18T00:19:11.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088997+0000", 2024-09-18T00:19:11.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0484999999999999e-05 2024-09-18T00:19:11.919 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.919 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.919 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.919 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:6801 3.10 e42/23 ondisk, result = 0)", 2024-09-18T00:19:11.919 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:11.919 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.843768814000001, 2024-09-18T00:19:11.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037960394000000001, 2024-09-18T00:19:11.920 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.920 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.920 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.920 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:11.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:11.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050852+0000", 2024-09-18T00:19:11.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:19:11.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050860+0000", 2024-09-18T00:19:11.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7729999999999999e-06 2024-09-18T00:19:11.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050891+0000", 2024-09-18T00:19:11.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0617000000000002e-05 2024-09-18T00:19:11.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050910+0000", 2024-09-18T00:19:11.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.944e-05 2024-09-18T00:19:11.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088766+0000", 2024-09-18T00:19:11.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037855633999999999 2024-09-18T00:19:11.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088784+0000", 2024-09-18T00:19:11.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7767000000000001e-05 2024-09-18T00:19:11.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088811+0000", 2024-09-18T00:19:11.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7382999999999999e-05 2024-09-18T00:19:11.923 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.923 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.923 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.923 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12594 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.924 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:11.924 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.502992276000001, 2024-09-18T00:19:11.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026492600000000002, 2024-09-18T00:19:11.924 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.924 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.924 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.924 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.924 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.924 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.924 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12594 2024-09-18T00:19:11.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.924 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:11.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:11.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391629+0000", 2024-09-18T00:19:11.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.86e-06 2024-09-18T00:19:11.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391637+0000", 2024-09-18T00:19:11.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3999999999999992e-06 2024-09-18T00:19:11.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391664+0000", 2024-09-18T00:19:11.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6536000000000001e-05 2024-09-18T00:19:11.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391681+0000", 2024-09-18T00:19:11.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6563e-05 2024-09-18T00:19:11.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417791+0000", 2024-09-18T00:19:11.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026110936000000001 2024-09-18T00:19:11.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417933+0000", 2024-09-18T00:19:11.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014112099999999999 2024-09-18T00:19:11.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418120+0000", 2024-09-18T00:19:11.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018718399999999999 2024-09-18T00:19:11.928 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.928 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.928 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.928 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12595 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.928 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:11.928 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.502916077999998, 2024-09-18T00:19:11.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026819428999999999, 2024-09-18T00:19:11.928 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.928 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.928 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.928 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.929 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.929 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.929 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12595 2024-09-18T00:19:11.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.929 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:11.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:11.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391705+0000", 2024-09-18T00:19:11.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3999999999999999e-06 2024-09-18T00:19:11.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391711+0000", 2024-09-18T00:19:11.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7459999999999997e-06 2024-09-18T00:19:11.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391733+0000", 2024-09-18T00:19:11.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2923000000000001e-05 2024-09-18T00:19:11.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391748+0000", 2024-09-18T00:19:11.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4790000000000001e-05 2024-09-18T00:19:11.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418180+0000", 2024-09-18T00:19:11.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026431888000000001 2024-09-18T00:19:11.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418328+0000", 2024-09-18T00:19:11.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000147725 2024-09-18T00:19:11.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418523+0000", 2024-09-18T00:19:11.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000194957 2024-09-18T00:19:11.932 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.932 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.932 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.932 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12596 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.932 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:11.933 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.502853109999997, 2024-09-18T00:19:11.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027220296000000001, 2024-09-18T00:19:11.933 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.933 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.933 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.933 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.933 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.933 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.933 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12596 2024-09-18T00:19:11.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.933 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:11.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:11.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391768+0000", 2024-09-18T00:19:11.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.283e-06 2024-09-18T00:19:11.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391773+0000", 2024-09-18T00:19:11.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3970000000000002e-06 2024-09-18T00:19:11.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391795+0000", 2024-09-18T00:19:11.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1835e-05 2024-09-18T00:19:11.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391809+0000", 2024-09-18T00:19:11.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4107000000000001e-05 2024-09-18T00:19:11.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418582+0000", 2024-09-18T00:19:11.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026772918999999999 2024-09-18T00:19:11.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418780+0000", 2024-09-18T00:19:11.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000198042 2024-09-18T00:19:11.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418987+0000", 2024-09-18T00:19:11.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000206713 2024-09-18T00:19:11.936 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.937 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.937 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.937 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12597 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.937 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:11.937 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.502792591999999, 2024-09-18T00:19:11.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027612714, 2024-09-18T00:19:11.937 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.937 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.937 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.937 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.937 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.937 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.937 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12597 2024-09-18T00:19:11.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.938 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:11.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:11.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391828+0000", 2024-09-18T00:19:11.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.203e-06 2024-09-18T00:19:11.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391834+0000", 2024-09-18T00:19:11.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6529999999999999e-06 2024-09-18T00:19:11.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391856+0000", 2024-09-18T00:19:11.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2195999999999999e-05 2024-09-18T00:19:11.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391870+0000", 2024-09-18T00:19:11.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4063000000000001e-05 2024-09-18T00:19:11.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419049+0000", 2024-09-18T00:19:11.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027178751000000001 2024-09-18T00:19:11.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419239+0000", 2024-09-18T00:19:11.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019039400000000001 2024-09-18T00:19:11.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419440+0000", 2024-09-18T00:19:11.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000200454 2024-09-18T00:19:11.941 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.941 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.941 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.941 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12598 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.941 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:11.941 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.502729541000001, 2024-09-18T00:19:11.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028269056000000001, 2024-09-18T00:19:11.942 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.942 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.942 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.942 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.942 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.942 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.942 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12598 2024-09-18T00:19:11.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.942 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:11.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:11.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391891+0000", 2024-09-18T00:19:11.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.257e-06 2024-09-18T00:19:11.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391897+0000", 2024-09-18T00:19:11.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9560000000000002e-06 2024-09-18T00:19:11.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391920+0000", 2024-09-18T00:19:11.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3025999999999999e-05 2024-09-18T00:19:11.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391935+0000", 2024-09-18T00:19:11.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4853e-05 2024-09-18T00:19:11.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419527+0000", 2024-09-18T00:19:11.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027592378000000001 2024-09-18T00:19:11.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419681+0000", 2024-09-18T00:19:11.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015316 2024-09-18T00:19:11.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420159+0000", 2024-09-18T00:19:11.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000478426 2024-09-18T00:19:11.946 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.946 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.946 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.946 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12599 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.946 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:11.946 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.502665686, 2024-09-18T00:19:11.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028649857000000001, 2024-09-18T00:19:11.946 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.946 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.946 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.947 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.947 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.947 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.947 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12599 2024-09-18T00:19:11.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.947 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:11.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:11.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391955+0000", 2024-09-18T00:19:11.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-06 2024-09-18T00:19:11.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391961+0000", 2024-09-18T00:19:11.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7100000000000004e-06 2024-09-18T00:19:11.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391982+0000", 2024-09-18T00:19:11.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1492e-05 2024-09-18T00:19:11.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391996+0000", 2024-09-18T00:19:11.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4194e-05 2024-09-18T00:19:11.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420237+0000", 2024-09-18T00:19:11.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028240818000000001 2024-09-18T00:19:11.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420430+0000", 2024-09-18T00:19:11.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019228299999999999 2024-09-18T00:19:11.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420604+0000", 2024-09-18T00:19:11.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000174117 2024-09-18T00:19:11.951 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.951 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.951 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.952 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12600 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 270336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.952 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:11.952 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.502604691999998, 2024-09-18T00:19:11.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028964586, 2024-09-18T00:19:11.952 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.952 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.952 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.952 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.952 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.953 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.953 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12600 2024-09-18T00:19:11.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.953 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:11.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:11.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392016+0000", 2024-09-18T00:19:11.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.31e-06 2024-09-18T00:19:11.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392022+0000", 2024-09-18T00:19:11.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5600000000000001e-06 2024-09-18T00:19:11.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392045+0000", 2024-09-18T00:19:11.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3595999999999999e-05 2024-09-18T00:19:11.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392060+0000", 2024-09-18T00:19:11.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4727e-05 2024-09-18T00:19:11.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420669+0000", 2024-09-18T00:19:11.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028608855999999998 2024-09-18T00:19:11.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420813+0000", 2024-09-18T00:19:11.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014429500000000001 2024-09-18T00:19:11.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420979+0000", 2024-09-18T00:19:11.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016624200000000001 2024-09-18T00:19:11.956 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.956 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.956 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.957 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12601 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.957 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:11.957 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.502540787000001, 2024-09-18T00:19:11.957 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029292832000000001, 2024-09-18T00:19:11.957 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.957 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.957 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.957 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.957 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.957 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.957 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12601 2024-09-18T00:19:11.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.958 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:11.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:11.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392080+0000", 2024-09-18T00:19:11.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:11.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392086+0000", 2024-09-18T00:19:11.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7570000000000001e-06 2024-09-18T00:19:11.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392105+0000", 2024-09-18T00:19:11.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9415999999999998e-05 2024-09-18T00:19:11.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392132+0000", 2024-09-18T00:19:11.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7019e-05 2024-09-18T00:19:11.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.960 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421049+0000", 2024-09-18T00:19:11.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028916659000000001 2024-09-18T00:19:11.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.960 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421204+0000", 2024-09-18T00:19:11.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155224 2024-09-18T00:19:11.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.961 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.961 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.961 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421372+0000", 2024-09-18T00:19:11.961 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016754699999999999 2024-09-18T00:19:11.961 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.961 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.961 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.961 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.961 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12602 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.961 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:11.961 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.502468155999999, 2024-09-18T00:19:11.961 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029612833000000002, 2024-09-18T00:19:11.961 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.962 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.962 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.962 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.962 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.962 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.962 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12602 2024-09-18T00:19:11.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.962 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.962 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:11.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:11.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392152+0000", 2024-09-18T00:19:11.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0470000000000001e-06 2024-09-18T00:19:11.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392158+0000", 2024-09-18T00:19:11.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:19:11.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392181+0000", 2024-09-18T00:19:11.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2629e-05 2024-09-18T00:19:11.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392222+0000", 2024-09-18T00:19:11.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0772000000000002e-05 2024-09-18T00:19:11.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421441+0000", 2024-09-18T00:19:11.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029219327 2024-09-18T00:19:11.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.965 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.965 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.965 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421608+0000", 2024-09-18T00:19:11.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000166813 2024-09-18T00:19:11.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.965 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.965 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.965 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421764+0000", 2024-09-18T00:19:11.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015628500000000001 2024-09-18T00:19:11.965 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.965 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.965 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.966 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12603 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.966 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:11.966 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.502377762000002, 2024-09-18T00:19:11.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029902524, 2024-09-18T00:19:11.966 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.966 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.966 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.966 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.966 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.966 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.966 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12603 2024-09-18T00:19:11.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.967 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:11.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:11.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392243+0000", 2024-09-18T00:19:11.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:11.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392260+0000", 2024-09-18T00:19:11.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6860000000000001e-05 2024-09-18T00:19:11.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392282+0000", 2024-09-18T00:19:11.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2713000000000001e-05 2024-09-18T00:19:11.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.969 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.969 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392297+0000", 2024-09-18T00:19:11.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4375e-05 2024-09-18T00:19:11.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.969 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.969 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421830+0000", 2024-09-18T00:19:11.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029532668000000002 2024-09-18T00:19:11.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.969 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.970 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421985+0000", 2024-09-18T00:19:11.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155613 2024-09-18T00:19:11.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.970 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.970 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422144+0000", 2024-09-18T00:19:11.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000159102 2024-09-18T00:19:11.970 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.970 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.970 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.970 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12604 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.971 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:11.971 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.502302710999999, 2024-09-18T00:19:11.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030228695, 2024-09-18T00:19:11.971 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.971 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.971 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.971 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.971 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.971 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.971 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12604 2024-09-18T00:19:11.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.971 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:11.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:11.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392318+0000", 2024-09-18T00:19:11.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:11.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392337+0000", 2024-09-18T00:19:11.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:19:11.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392364+0000", 2024-09-18T00:19:11.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7288999999999999e-05 2024-09-18T00:19:11.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.974 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.974 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392379+0000", 2024-09-18T00:19:11.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:19:11.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.974 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.974 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422209+0000", 2024-09-18T00:19:11.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029829050999999999 2024-09-18T00:19:11.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422388+0000", 2024-09-18T00:19:11.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017988699999999999 2024-09-18T00:19:11.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422545+0000", 2024-09-18T00:19:11.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015701199999999999 2024-09-18T00:19:11.975 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.975 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.975 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.976 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12605 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.976 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:11.976 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.502218149999997, 2024-09-18T00:19:11.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030488674, 2024-09-18T00:19:11.976 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.976 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.976 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.976 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.976 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.976 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.976 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12605 2024-09-18T00:19:11.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.976 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:11.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:11.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392403+0000", 2024-09-18T00:19:11.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.883e-06 2024-09-18T00:19:11.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392421+0000", 2024-09-18T00:19:11.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7526000000000001e-05 2024-09-18T00:19:11.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392450+0000", 2024-09-18T00:19:11.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8972e-05 2024-09-18T00:19:11.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.979 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392488+0000", 2024-09-18T00:19:11.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8522000000000001e-05 2024-09-18T00:19:11.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.979 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422614+0000", 2024-09-18T00:19:11.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030125945000000001 2024-09-18T00:19:11.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.979 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422752+0000", 2024-09-18T00:19:11.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000138178 2024-09-18T00:19:11.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422890+0000", 2024-09-18T00:19:11.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013764800000000001 2024-09-18T00:19:11.980 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.980 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.980 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.980 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12606 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:11.980 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:11.981 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.502109490000002, 2024-09-18T00:19:11.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033727073000000003, 2024-09-18T00:19:11.981 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.981 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.981 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.981 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.981 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.981 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.981 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12606 2024-09-18T00:19:11.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.981 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:11.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:11.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392511+0000", 2024-09-18T00:19:11.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3090000000000001e-06 2024-09-18T00:19:11.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.983 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392517+0000", 2024-09-18T00:19:11.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.993e-06 2024-09-18T00:19:11.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.983 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392543+0000", 2024-09-18T00:19:11.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6152999999999998e-05 2024-09-18T00:19:11.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.983 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392558+0000", 2024-09-18T00:19:11.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4610000000000001e-05 2024-09-18T00:19:11.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.425959+0000", 2024-09-18T00:19:11.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033400484000000001 2024-09-18T00:19:11.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426106+0000", 2024-09-18T00:19:11.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014779800000000001 2024-09-18T00:19:11.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426237+0000", 2024-09-18T00:19:11.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013072600000000001 2024-09-18T00:19:11.985 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.985 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.985 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.985 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17562 3.a 3:5578626c:::rbd_data.112a6efb7564.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T00:19:11.985 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:11.985 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 21.936644329, 2024-09-18T00:19:11.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052733453999999999, 2024-09-18T00:19:11.985 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.986 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.986 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:11.986 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:11.986 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:11.986 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:11.986 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17562 2024-09-18T00:19:11.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.986 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:11.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:11.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957977+0000", 2024-09-18T00:19:11.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.119e-06 2024-09-18T00:19:11.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957988+0000", 2024-09-18T00:19:11.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0557000000000001e-05 2024-09-18T00:19:11.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958017+0000", 2024-09-18T00:19:11.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9156e-05 2024-09-18T00:19:11.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958036+0000", 2024-09-18T00:19:11.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8919e-05 2024-09-18T00:19:11.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958144+0000", 2024-09-18T00:19:11.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107905 2024-09-18T00:19:11.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958284+0000", 2024-09-18T00:19:11.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140216 2024-09-18T00:19:11.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:11.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.960572+0000", 2024-09-18T00:19:11.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022881860000000002 2024-09-18T00:19:11.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:11.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010537+0000", 2024-09-18T00:19:11.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049964664999999998 2024-09-18T00:19:11.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:11.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010590+0000", 2024-09-18T00:19:11.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3094999999999999e-05 2024-09-18T00:19:11.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:11.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010642+0000", 2024-09-18T00:19:11.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2073000000000001e-05 2024-09-18T00:19:11.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010709+0000", 2024-09-18T00:19:11.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6563000000000006e-05 2024-09-18T00:19:11.991 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.991 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.991 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.991 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17562 3.a e60/23 ondisk, result = 0)", 2024-09-18T00:19:11.991 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:11.991 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 21.926440958000001, 2024-09-18T00:19:11.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042539307999999998, 2024-09-18T00:19:11.991 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:11.992 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:11.992 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:11.992 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:11.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:11.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:11.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:11.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:11.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:11.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:11.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968180+0000", 2024-09-18T00:19:11.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-18T00:19:11.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:11.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968188+0000", 2024-09-18T00:19:11.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7419999999999997e-06 2024-09-18T00:19:11.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:11.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968217+0000", 2024-09-18T00:19:11.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8926999999999999e-05 2024-09-18T00:19:11.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:11.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968231+0000", 2024-09-18T00:19:11.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4168999999999999e-05 2024-09-18T00:19:11.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:11.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010575+0000", 2024-09-18T00:19:11.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042343923999999998 2024-09-18T00:19:11.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:11.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010584+0000", 2024-09-18T00:19:11.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-18T00:19:11.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:11.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:11.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:11.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010718+0000", 2024-09-18T00:19:11.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013344499999999999 2024-09-18T00:19:11.995 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.995 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.995 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.995 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:11.995 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:11.995 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:11.996 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:19:11.998 INFO:teuthology.orchestra.run.smithi097.stdout:4561 writesame 0x39d3b36 thru 0x39d6940 data_size 0x1(0x2e0b bytes) 2024-09-18T00:19:11.998 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:11.998 INFO:teuthology.orchestra.run.smithi097.stdout:4563 write 0x4bc975c thru 0x4bcf33d (0x5be2 bytes) 2024-09-18T00:19:11.998 INFO:teuthology.orchestra.run.smithi097.stdout:4564 write 0x9d4d550 thru 0x9d50c8e (0x373f bytes) 2024-09-18T00:19:11.998 INFO:teuthology.orchestra.run.smithi097.stdout:4565 read 0x3789c8c thru 0x3799415 (0xf78a bytes) 2024-09-18T00:19:11.998 INFO:teuthology.orchestra.run.smithi097.stdout:4567 writesame 0xd27eca thru 0xd2e377 data_size 0x31(0x64ae bytes) 2024-09-18T00:19:11.999 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:19:11.999 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:11.993+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:12.054 INFO:teuthology.orchestra.run.smithi097.stdout:4569 read 0x346111f thru 0x3467ce5 (0x6bc7 bytes) 2024-09-18T00:19:12.055 INFO:teuthology.orchestra.run.smithi097.stdout:4570 read 0x3599acd thru 0x35a1566 (0x7a9a bytes) 2024-09-18T00:19:12.056 INFO:teuthology.orchestra.run.smithi097.stdout:4571 read 0xde6037 thru 0xdf3b3f (0xdb09 bytes) 2024-09-18T00:19:12.057 INFO:teuthology.orchestra.run.smithi097.stdout:4572 punch from 0x3fcede0 to 0x3fddd9c, (0xefbc bytes) 2024-09-18T00:19:12.057 INFO:teuthology.orchestra.run.smithi097.stdout:4574 writesame 0x362b444 thru 0x362fd3d data_size 0x1(0x48fa bytes) 2024-09-18T00:19:12.062 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:12.074 INFO:teuthology.orchestra.run.smithi097.stdout:4576 trunc from 0xaa8bcc4 to 0x250dd8c 2024-09-18T00:19:12.097 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:12.093+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:12.122 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:12.122 INFO:teuthology.orchestra.run.smithi097.stdout:4578 write 0x1dfa170 thru 0x1e0854c (0xe3dd bytes) 2024-09-18T00:19:12.145 INFO:teuthology.orchestra.run.smithi097.stdout:4579 trunc from 0x250dd8c to 0x2e2bb7e 2024-09-18T00:19:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:no valid command found; 10 closest matches: 2024-09-18T00:19:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:0 2024-09-18T00:19:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:1 2024-09-18T00:19:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:2 2024-09-18T00:19:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:abort 2024-09-18T00:19:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:assert 2024-09-18T00:19:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:bluestore allocator dump block 2024-09-18T00:19:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:bluestore allocator fragmentation block 2024-09-18T00:19:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:bluestore allocator fragmentation histogram block [] [] 2024-09-18T00:19:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:bluestore allocator score block 2024-09-18T00:19:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:config diff 2024-09-18T00:19:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:admin_socket: invalid command 2024-09-18T00:19:12.173 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T00:19:12.173 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-09-18T00:19:12.175 INFO:teuthology.orchestra.run.smithi097.stdout:4580 writesame 0xee24f57 thru 0xee31c65 data_size 0x1(0xcd0f bytes) 2024-09-18T00:19:12.175 INFO:teuthology.orchestra.run.smithi097.stdout:4581 read 0x75b1ad6 thru 0x75bcf3f (0xb46a bytes) 2024-09-18T00:19:12.175 INFO:teuthology.orchestra.run.smithi097.stdout:4582 read 0x345a9d6 thru 0x345b209 (0x834 bytes) 2024-09-18T00:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout:4583 read 0xc370d43 thru 0xc3733ec (0x26aa bytes) 2024-09-18T00:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout:4586 read 0xdf000c4 thru 0xdf084c3 (0x8400 bytes) 2024-09-18T00:19:12.176 INFO:teuthology.orchestra.run.smithi097.stdout:4587 writesame 0xcdcec79 thru 0xcdda592 data_size 0x3a(0xb91a bytes) 2024-09-18T00:19:12.184 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:12.184 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:12.184 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:12.184 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout:4590 read 0x125b504 thru 0x125cda7 (0x18a4 bytes) 2024-09-18T00:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout:4591 read 0x9b01353 thru 0x9b06630 (0x52de bytes) 2024-09-18T00:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout:4594 punch from 0xc033ac3 to 0xc039e4e, (0x638b bytes) 2024-09-18T00:19:12.194 INFO:teuthology.orchestra.run.smithi097.stdout:4595 punch from 0xa1e1626 to 0xa1e5c9f, (0x4679 bytes) 2024-09-18T00:19:12.195 INFO:teuthology.orchestra.run.smithi097.stdout:4596 read 0x1a16eff thru 0x1a1d151 (0x6253 bytes) 2024-09-18T00:19:12.199 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:12.198+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:12.200 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:19:12.208 INFO:teuthology.orchestra.run.smithi097.stdout:4597 trunc from 0xee31c66 to 0x6c929a2 2024-09-18T00:19:12.223 INFO:teuthology.orchestra.run.smithi097.stdout:4599 write 0x1093931 thru 0x109d9b6 (0xa086 bytes) 2024-09-18T00:19:12.255 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:12.270 INFO:teuthology.orchestra.run.smithi097.stdout:4602 write 0xad77b2d thru 0xad7ab79 (0x304d bytes) 2024-09-18T00:19:12.287 INFO:teuthology.orchestra.run.smithi097.stdout:4603 writesame 0x3e01e3f thru 0x3e09e1d data_size 0x1(0x7fdf bytes) 2024-09-18T00:19:12.287 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:19:12.298 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:12.298+0000 7f73da6c37c0 -1 osd.5 55 log_to_monitors true 2024-09-18T00:19:12.311 INFO:teuthology.orchestra.run.smithi097.stdout:4604 punch from 0x2ece947 to 0x2edc9d2, (0xe08b bytes) 2024-09-18T00:19:12.311 INFO:teuthology.orchestra.run.smithi097.stdout:4607 punch from 0x735e3c1 to 0x736164f, (0x328e bytes) 2024-09-18T00:19:12.312 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:12.305+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:12.317 INFO:teuthology.orchestra.run.smithi097.stdout:4608 trunc from 0xad7ab7a to 0xdd9084b 2024-09-18T00:19:12.323 INFO:teuthology.orchestra.run.smithi097.stdout:4609 write 0x164b749 thru 0x164d5a5 (0x1e5d bytes) 2024-09-18T00:19:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:4610 punch from 0x579bb38 to 0x57aa17a, (0xe642 bytes) 2024-09-18T00:19:12.337 INFO:teuthology.orchestra.run.smithi097.stdout:4611 write 0x6db06a thru 0x6e9ac9 (0xea60 bytes) 2024-09-18T00:19:12.388 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:12.388 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:12.388 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:12.388 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:12.388 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:12.399 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:12.398+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:12.410 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:19:12.500 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:12.498+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:12.575 INFO:teuthology.orchestra.run.smithi097.stdout:4612 read 0x18330cc thru 0x183aad6 (0x7a0b bytes) 2024-09-18T00:19:12.576 INFO:teuthology.orchestra.run.smithi097.stdout:4613 write 0x7ede51d thru 0x7edead9 (0x5bd bytes) 2024-09-18T00:19:12.579 INFO:teuthology.orchestra.run.smithi097.stdout:4614 writesame 0x12ea167 thru 0x12ec854 data_size 0x21(0x26ee bytes) 2024-09-18T00:19:12.583 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:12.583 INFO:teuthology.orchestra.run.smithi097.stdout:4616 write 0x3ce47e4 thru 0x3ce8718 (0x3f35 bytes) 2024-09-18T00:19:12.585 INFO:teuthology.orchestra.run.smithi097.stdout:4618 write 0x12a4733 thru 0x12a5172 (0xa40 bytes) 2024-09-18T00:19:12.586 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2228 3.15 3:aa2fc059:::rbd_data.112aa7ba4578.000000000000014a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.321121058000003, 2024-09-18T00:19:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.068966377999999995, 2024-09-18T00:19:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2228 2024-09-18T00:19:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263099+0000", 2024-09-18T00:19:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2359999999999999e-06 2024-09-18T00:19:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263115+0000", 2024-09-18T00:19:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5996999999999999e-05 2024-09-18T00:19:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263156+0000", 2024-09-18T00:19:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:19:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263183+0000", 2024-09-18T00:19:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7233000000000002e-05 2024-09-18T00:19:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263304+0000", 2024-09-18T00:19:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121209 2024-09-18T00:19:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263528+0000", 2024-09-18T00:19:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022401800000000001 2024-09-18T00:19:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:19:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.266458+0000", 2024-09-18T00:19:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029296980000000001 2024-09-18T00:19:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331628+0000", 2024-09-18T00:19:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.065170045999999995 2024-09-18T00:19:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331819+0000", 2024-09-18T00:19:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000190819 2024-09-18T00:19:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331971+0000", 2024-09-18T00:19:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151972 2024-09-18T00:19:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332063+0000", 2024-09-18T00:19:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2083999999999995e-05 2024-09-18T00:19:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2228 3.15 e36/32 ondisk, result = 0)", 2024-09-18T00:19:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:19:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.308067946000001, 2024-09-18T00:19:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055930148999999998, 2024-09-18T00:19:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:19:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:19:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276152+0000", 2024-09-18T00:19:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0200000000000001e-06 2024-09-18T00:19:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276162+0000", 2024-09-18T00:19:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0767e-05 2024-09-18T00:19:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276202+0000", 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000003e-05 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276226+0000", 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.351e-05 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331782+0000", 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055556296999999998 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331801+0000", 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9357000000000001e-05 2024-09-18T00:19:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332080+0000", 2024-09-18T00:19:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027849800000000002 2024-09-18T00:19:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2854 3.f 3:f02b0bca:::rbd_data.112aa7ba4578.0000000000000135:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:19:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.714148567999999, 2024-09-18T00:19:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.082582802999999996, 2024-09-18T00:19:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2854 2024-09-18T00:19:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870071+0000", 2024-09-18T00:19:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.227e-06 2024-09-18T00:19:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870083+0000", 2024-09-18T00:19:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1909999999999999e-05 2024-09-18T00:19:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870114+0000", 2024-09-18T00:19:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0576e-05 2024-09-18T00:19:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870129+0000", 2024-09-18T00:19:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.485e-05 2024-09-18T00:19:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870256+0000", 2024-09-18T00:19:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012691300000000001 2024-09-18T00:19:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870493+0000", 2024-09-18T00:19:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023737399999999999 2024-09-18T00:19:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.874933+0000", 2024-09-18T00:19:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044398299999999996 2024-09-18T00:19:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952452+0000", 2024-09-18T00:19:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.077518918000000006 2024-09-18T00:19:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952486+0000", 2024-09-18T00:19:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4035999999999997e-05 2024-09-18T00:19:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952575+0000", 2024-09-18T00:19:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.954e-05 2024-09-18T00:19:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952652+0000", 2024-09-18T00:19:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6629000000000001e-05 2024-09-18T00:19:12.601 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2854 3.f e38/32 ondisk, result = 0)", 2024-09-18T00:19:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.696617752999998, 2024-09-18T00:19:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064868625999999999, 2024-09-18T00:19:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:12.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887602+0000", 2024-09-18T00:19:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.677e-06 2024-09-18T00:19:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887609+0000", 2024-09-18T00:19:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6729999999999992e-06 2024-09-18T00:19:12.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887637+0000", 2024-09-18T00:19:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8206000000000001e-05 2024-09-18T00:19:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887651+0000", 2024-09-18T00:19:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3387000000000001e-05 2024-09-18T00:19:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952422+0000", 2024-09-18T00:19:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064771366999999996 2024-09-18T00:19:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952441+0000", 2024-09-18T00:19:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8737e-05 2024-09-18T00:19:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952469+0000", 2024-09-18T00:19:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7579000000000001e-05 2024-09-18T00:19:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.605 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5676 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 12255232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:19:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 47.285941895999997, 2024-09-18T00:19:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047571816000000003, 2024-09-18T00:19:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5676 2024-09-18T00:19:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:19:12.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:19:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298277+0000", 2024-09-18T00:19:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.454e-06 2024-09-18T00:19:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298282+0000", 2024-09-18T00:19:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6299999999999997e-06 2024-09-18T00:19:12.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298299+0000", 2024-09-18T00:19:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6959000000000001e-05 2024-09-18T00:19:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298313+0000", 2024-09-18T00:19:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.376e-05 2024-09-18T00:19:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345610+0000", 2024-09-18T00:19:12.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047297128000000001 2024-09-18T00:19:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345708+0000", 2024-09-18T00:19:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8245999999999994e-05 2024-09-18T00:19:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345848+0000", 2024-09-18T00:19:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000139639 2024-09-18T00:19:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.609 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5677 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 12779520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:19:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 47.285888383, 2024-09-18T00:19:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047810679000000002, 2024-09-18T00:19:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5677 2024-09-18T00:19:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:19:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:19:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298330+0000", 2024-09-18T00:19:12.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.217e-06 2024-09-18T00:19:12.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298354+0000", 2024-09-18T00:19:12.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3753000000000001e-05 2024-09-18T00:19:12.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298388+0000", 2024-09-18T00:19:12.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4276999999999997e-05 2024-09-18T00:19:12.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298415+0000", 2024-09-18T00:19:12.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6403000000000001e-05 2024-09-18T00:19:12.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345896+0000", 2024-09-18T00:19:12.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047481007999999998 2024-09-18T00:19:12.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345992+0000", 2024-09-18T00:19:12.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6141e-05 2024-09-18T00:19:12.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346140+0000", 2024-09-18T00:19:12.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014788 2024-09-18T00:19:12.614 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.614 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.614 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.614 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5678 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:12.614 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:19:12.614 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 47.285782204, 2024-09-18T00:19:12.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047985466999999997, 2024-09-18T00:19:12.614 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.615 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.615 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:12.615 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:12.615 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:12.615 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:12.615 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5678 2024-09-18T00:19:12.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.615 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:19:12.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:19:12.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298437+0000", 2024-09-18T00:19:12.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1799999999999999e-06 2024-09-18T00:19:12.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298443+0000", 2024-09-18T00:19:12.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1140000000000002e-06 2024-09-18T00:19:12.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298460+0000", 2024-09-18T00:19:12.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7615999999999999e-05 2024-09-18T00:19:12.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298474+0000", 2024-09-18T00:19:12.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4017e-05 2024-09-18T00:19:12.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346180+0000", 2024-09-18T00:19:12.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047705801999999999 2024-09-18T00:19:12.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346274+0000", 2024-09-18T00:19:12.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3814999999999995e-05 2024-09-18T00:19:12.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346421+0000", 2024-09-18T00:19:12.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000146923 2024-09-18T00:19:12.618 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.619 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.619 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.619 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5679 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:12.619 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:19:12.619 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 47.285723160000003, 2024-09-18T00:19:12.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048190771, 2024-09-18T00:19:12.619 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.619 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.619 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:12.619 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:12.620 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:12.620 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:12.620 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5679 2024-09-18T00:19:12.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.620 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:19:12.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:19:12.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298496+0000", 2024-09-18T00:19:12.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4699999999999999e-06 2024-09-18T00:19:12.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298501+0000", 2024-09-18T00:19:12.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:19:12.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298519+0000", 2024-09-18T00:19:12.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7329e-05 2024-09-18T00:19:12.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298532+0000", 2024-09-18T00:19:12.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3083e-05 2024-09-18T00:19:12.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346462+0000", 2024-09-18T00:19:12.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047930449999999999 2024-09-18T00:19:12.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346556+0000", 2024-09-18T00:19:12.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3222999999999998e-05 2024-09-18T00:19:12.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346685+0000", 2024-09-18T00:19:12.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000129696 2024-09-18T00:19:12.623 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.623 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.623 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.623 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5680 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:12.623 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:19:12.623 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 47.285665751000003, 2024-09-18T00:19:12.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048535847, 2024-09-18T00:19:12.624 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.624 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.624 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:12.624 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:12.624 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:12.624 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:12.624 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5680 2024-09-18T00:19:12.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.624 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:19:12.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:19:12.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298553+0000", 2024-09-18T00:19:12.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0759999999999999e-06 2024-09-18T00:19:12.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298575+0000", 2024-09-18T00:19:12.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2019999999999999e-05 2024-09-18T00:19:12.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298593+0000", 2024-09-18T00:19:12.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7900000000000001e-05 2024-09-18T00:19:12.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298607+0000", 2024-09-18T00:19:12.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3844e-05 2024-09-18T00:19:12.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346791+0000", 2024-09-18T00:19:12.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048184449999999997 2024-09-18T00:19:12.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346921+0000", 2024-09-18T00:19:12.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012994699999999999 2024-09-18T00:19:12.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347088+0000", 2024-09-18T00:19:12.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016661000000000001 2024-09-18T00:19:12.627 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.627 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.627 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.628 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5681 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:12.628 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:19:12.628 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 47.285593341000002, 2024-09-18T00:19:12.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048743337999999997, 2024-09-18T00:19:12.628 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.628 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.628 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:12.628 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:12.628 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:12.628 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:12.628 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5681 2024-09-18T00:19:12.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.629 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:19:12.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:19:12.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298626+0000", 2024-09-18T00:19:12.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4899999999999999e-06 2024-09-18T00:19:12.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298631+0000", 2024-09-18T00:19:12.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0300000000000001e-06 2024-09-18T00:19:12.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298650+0000", 2024-09-18T00:19:12.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9499000000000001e-05 2024-09-18T00:19:12.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298664+0000", 2024-09-18T00:19:12.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3767e-05 2024-09-18T00:19:12.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347139+0000", 2024-09-18T00:19:12.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048475348000000001 2024-09-18T00:19:12.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347228+0000", 2024-09-18T00:19:12.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8843000000000003e-05 2024-09-18T00:19:12.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347368+0000", 2024-09-18T00:19:12.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013936100000000001 2024-09-18T00:19:12.632 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.632 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.632 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.633 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5682 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:12.633 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:19:12.633 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 47.285536526000001, 2024-09-18T00:19:12.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048916864999999997, 2024-09-18T00:19:12.633 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.633 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.633 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:12.633 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:12.633 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:12.633 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:12.633 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5682 2024-09-18T00:19:12.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.633 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:19:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:19:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298682+0000", 2024-09-18T00:19:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:19:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298687+0000", 2024-09-18T00:19:12.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9239999999999998e-06 2024-09-18T00:19:12.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298704+0000", 2024-09-18T00:19:12.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6569999999999999e-05 2024-09-18T00:19:12.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298731+0000", 2024-09-18T00:19:12.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6752999999999999e-05 2024-09-18T00:19:12.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347416+0000", 2024-09-18T00:19:12.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048685056999999997 2024-09-18T00:19:12.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347506+0000", 2024-09-18T00:19:12.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9936000000000001e-05 2024-09-18T00:19:12.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347598+0000", 2024-09-18T00:19:12.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2335000000000003e-05 2024-09-18T00:19:12.637 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.637 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.637 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.637 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15879 3.16 3:692b47b6:::rbd_object_map.112a13ec2d6c.000000000000000b:head [call] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:19:12.637 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:12.637 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.929453846000001, 2024-09-18T00:19:12.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.011184018, 2024-09-18T00:19:12.638 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.638 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.638 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:12.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:12.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:12.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:12.638 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15879 2024-09-18T00:19:12.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.638 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:12.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:12.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654768+0000", 2024-09-18T00:19:12.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9129999999999996e-06 2024-09-18T00:19:12.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654786+0000", 2024-09-18T00:19:12.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8695999999999999e-05 2024-09-18T00:19:12.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654832+0000", 2024-09-18T00:19:12.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5735000000000002e-05 2024-09-18T00:19:12.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654860+0000", 2024-09-18T00:19:12.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7789000000000001e-05 2024-09-18T00:19:12.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655048+0000", 2024-09-18T00:19:12.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018781900000000001 2024-09-18T00:19:12.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:19:12.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655070+0000", 2024-09-18T00:19:12.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1916000000000002e-05 2024-09-18T00:19:12.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663335+0000", 2024-09-18T00:19:12.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0082651210000000003 2024-09-18T00:19:12.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:12.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663358+0000", 2024-09-18T00:19:12.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2986e-05 2024-09-18T00:19:12.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663739+0000", 2024-09-18T00:19:12.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0003814820000001 2024-09-18T00:19:12.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663920+0000", 2024-09-18T00:19:12.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018116900000000001 2024-09-18T00:19:12.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.665948+0000", 2024-09-18T00:19:12.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020273919999999998 2024-09-18T00:19:12.643 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.643 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.643 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.643 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15881 3.16 3:6cc14920:::rbd_header.112a3285400b:head [watch reconnect cookie 140685950184608 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:19:12.643 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:12.644 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.929292851, 2024-09-18T00:19:12.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0138723599999999, 2024-09-18T00:19:12.644 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.644 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.644 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:12.644 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:12.644 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:12.644 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:12.644 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15881 2024-09-18T00:19:12.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.644 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:12.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:12.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654926+0000", 2024-09-18T00:19:12.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6470000000000001e-06 2024-09-18T00:19:12.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654935+0000", 2024-09-18T00:19:12.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8859999999999993e-06 2024-09-18T00:19:12.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654960+0000", 2024-09-18T00:19:12.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4689999999999999e-05 2024-09-18T00:19:12.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654976+0000", 2024-09-18T00:19:12.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6319000000000001e-05 2024-09-18T00:19:12.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655238+0000", 2024-09-18T00:19:12.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026145999999999998 2024-09-18T00:19:12.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:12.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655255+0000", 2024-09-18T00:19:12.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7249999999999999e-05 2024-09-18T00:19:12.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663416+0000", 2024-09-18T00:19:12.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081607959999999997 2024-09-18T00:19:12.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:12.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663436+0000", 2024-09-18T00:19:12.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0384e-05 2024-09-18T00:19:12.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666184+0000", 2024-09-18T00:19:12.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002748121 2024-09-18T00:19:12.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666452+0000", 2024-09-18T00:19:12.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026801700000000001 2024-09-18T00:19:12.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.668797+0000", 2024-09-18T00:19:12.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023447899999999998 2024-09-18T00:19:12.649 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.649 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.649 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.650 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17089 3.f 3:f7681386:::rbd_data.112a6efb7564.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:19:12.650 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:12.650 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.704365755000001, 2024-09-18T00:19:12.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.12573889399999999, 2024-09-18T00:19:12.650 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.650 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.650 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:12.650 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:12.650 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:12.650 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:12.650 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17089 2024-09-18T00:19:12.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.650 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:12.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:12.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879854+0000", 2024-09-18T00:19:12.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:19:12.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879867+0000", 2024-09-18T00:19:12.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3103e-05 2024-09-18T00:19:12.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879898+0000", 2024-09-18T00:19:12.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0103e-05 2024-09-18T00:19:12.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879912+0000", 2024-09-18T00:19:12.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4636e-05 2024-09-18T00:19:12.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879992+0000", 2024-09-18T00:19:12.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9814000000000005e-05 2024-09-18T00:19:12.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.880101+0000", 2024-09-18T00:19:12.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109172 2024-09-18T00:19:12.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:12.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.887269+0000", 2024-09-18T00:19:12.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071679300000000003 2024-09-18T00:19:12.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:12.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005269+0000", 2024-09-18T00:19:12.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.117999621 2024-09-18T00:19:12.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:12.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005317+0000", 2024-09-18T00:19:12.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8279000000000003e-05 2024-09-18T00:19:12.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:12.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005464+0000", 2024-09-18T00:19:12.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014661200000000001 2024-09-18T00:19:12.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005591+0000", 2024-09-18T00:19:12.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000127107 2024-09-18T00:19:12.655 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.655 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.655 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.655 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17089 3.f e59/32 ondisk, result = 0)", 2024-09-18T00:19:12.655 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:12.656 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.680258890000001, 2024-09-18T00:19:12.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101334403, 2024-09-18T00:19:12.656 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.656 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.656 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:12.656 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:12.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:12.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903960+0000", 2024-09-18T00:19:12.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.33e-06 2024-09-18T00:19:12.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903966+0000", 2024-09-18T00:19:12.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5929999999999996e-06 2024-09-18T00:19:12.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903986+0000", 2024-09-18T00:19:12.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.018e-05 2024-09-18T00:19:12.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903997+0000", 2024-09-18T00:19:12.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0988999999999999e-05 2024-09-18T00:19:12.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005171+0000", 2024-09-18T00:19:12.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101174267 2024-09-18T00:19:12.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005200+0000", 2024-09-18T00:19:12.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8540000000000001e-05 2024-09-18T00:19:12.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005293+0000", 2024-09-18T00:19:12.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504000000000005e-05 2024-09-18T00:19:12.659 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.659 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.659 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.660 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24529 3.16 3:6857ce1a:::rbd_data.112a990280ef.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:12.660 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:12.660 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.947447479, 2024-09-18T00:19:12.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10693805200000001, 2024-09-18T00:19:12.660 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.660 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.660 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:12.660 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:12.660 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:12.660 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:12.660 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24529 2024-09-18T00:19:12.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.660 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:12.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:12.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636772+0000", 2024-09-18T00:19:12.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1100000000000001e-06 2024-09-18T00:19:12.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636783+0000", 2024-09-18T00:19:12.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1005999999999999e-05 2024-09-18T00:19:12.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636814+0000", 2024-09-18T00:19:12.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1047000000000003e-05 2024-09-18T00:19:12.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636829+0000", 2024-09-18T00:19:12.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4363e-05 2024-09-18T00:19:12.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636980+0000", 2024-09-18T00:19:12.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015181099999999999 2024-09-18T00:19:12.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.637237+0000", 2024-09-18T00:19:12.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000256352 2024-09-18T00:19:12.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:12.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743536+0000", 2024-09-18T00:19:12.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.106298825 2024-09-18T00:19:12.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:12.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743599+0000", 2024-09-18T00:19:12.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3449999999999997e-05 2024-09-18T00:19:12.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743708+0000", 2024-09-18T00:19:12.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109088 2024-09-18T00:19:12.664 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.664 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.664 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.664 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24805 3.16 3:6de5e15d:::rbd_data.112a990280ef.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:12.665 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:12.665 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.3616442470000001, 2024-09-18T00:19:12.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14775107700000001, 2024-09-18T00:19:12.665 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.665 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.665 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:12.665 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:12.665 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:12.665 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:12.665 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24805 2024-09-18T00:19:12.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.665 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:12.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:12.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222574+0000", 2024-09-18T00:19:12.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0210000000000001e-06 2024-09-18T00:19:12.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222581+0000", 2024-09-18T00:19:12.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.144e-06 2024-09-18T00:19:12.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222595+0000", 2024-09-18T00:19:12.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4567e-05 2024-09-18T00:19:12.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222603+0000", 2024-09-18T00:19:12.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6529999999999994e-06 2024-09-18T00:19:12.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222687+0000", 2024-09-18T00:19:12.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4587999999999997e-05 2024-09-18T00:19:12.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222828+0000", 2024-09-18T00:19:12.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140178 2024-09-18T00:19:12.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:12.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370231+0000", 2024-09-18T00:19:12.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14740378700000001 2024-09-18T00:19:12.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:12.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370269+0000", 2024-09-18T00:19:12.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7886999999999998e-05 2024-09-18T00:19:12.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370324+0000", 2024-09-18T00:19:12.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5251999999999999e-05 2024-09-18T00:19:12.669 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.669 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.669 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.670 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24852 3.13 3:cc64f0fd:::rbd_data.112a990280ef.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:12.670 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:12.670 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.1289047769999998, 2024-09-18T00:19:12.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.066392943999999995, 2024-09-18T00:19:12.670 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.670 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.670 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:12.670 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:12.670 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:12.670 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:12.670 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24852 2024-09-18T00:19:12.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.671 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:12.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:12.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455315+0000", 2024-09-18T00:19:12.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.705e-06 2024-09-18T00:19:12.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455324+0000", 2024-09-18T00:19:12.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8120000000000002e-06 2024-09-18T00:19:12.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455349+0000", 2024-09-18T00:19:12.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4250999999999999e-05 2024-09-18T00:19:12.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455361+0000", 2024-09-18T00:19:12.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2805e-05 2024-09-18T00:19:12.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455425+0000", 2024-09-18T00:19:12.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3981999999999998e-05 2024-09-18T00:19:12.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455480+0000", 2024-09-18T00:19:12.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4792e-05 2024-09-18T00:19:12.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:12.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.460671+0000", 2024-09-18T00:19:12.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051907300000000002 2024-09-18T00:19:12.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:12.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521355+0000", 2024-09-18T00:19:12.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060684208000000003 2024-09-18T00:19:12.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:12.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521493+0000", 2024-09-18T00:19:12.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013803200000000001 2024-09-18T00:19:12.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:12.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521590+0000", 2024-09-18T00:19:12.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6339e-05 2024-09-18T00:19:12.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521706+0000", 2024-09-18T00:19:12.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116288 2024-09-18T00:19:12.675 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.675 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.675 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.675 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:25269 3.f 3:f6adfd48:::rbd_data.112aa34d8ec3.0000000000000145:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T00:19:12.675 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:12.675 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.979314816, 2024-09-18T00:19:12.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.075209085999999994, 2024-09-18T00:19:12.675 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.676 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.676 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:12.676 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:12.676 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:12.676 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:12.676 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 25269 2024-09-18T00:19:12.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.676 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:12.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:12.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604904+0000", 2024-09-18T00:19:12.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:19:12.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604910+0000", 2024-09-18T00:19:12.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.964e-06 2024-09-18T00:19:12.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604925+0000", 2024-09-18T00:19:12.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4894e-05 2024-09-18T00:19:12.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604932+0000", 2024-09-18T00:19:12.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-18T00:19:12.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604990+0000", 2024-09-18T00:19:12.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8143000000000002e-05 2024-09-18T00:19:12.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.605104+0000", 2024-09-18T00:19:12.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011404399999999999 2024-09-18T00:19:12.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:12.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.608755+0000", 2024-09-18T00:19:12.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003651081 2024-09-18T00:19:12.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:12.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679920+0000", 2024-09-18T00:19:12.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.071164543999999996 2024-09-18T00:19:12.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:12.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679978+0000", 2024-09-18T00:19:12.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8585000000000002e-05 2024-09-18T00:19:12.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:12.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680036+0000", 2024-09-18T00:19:12.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7978e-05 2024-09-18T00:19:12.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680112+0000", 2024-09-18T00:19:12.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5527999999999999e-05 2024-09-18T00:19:12.680 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.680 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.680 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.681 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:25269 3.f e72/32 ondisk, result = 0)", 2024-09-18T00:19:12.681 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:12.681 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.9661044309999998, 2024-09-18T00:19:12.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062014761000000002, 2024-09-18T00:19:12.681 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:12.681 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:12.681 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:12.681 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:12.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:12.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:12.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:12.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:12.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:12.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:12.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618114+0000", 2024-09-18T00:19:12.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1650000000000001e-06 2024-09-18T00:19:12.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:12.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618119+0000", 2024-09-18T00:19:12.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-18T00:19:12.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:12.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618134+0000", 2024-09-18T00:19:12.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5770999999999999e-05 2024-09-18T00:19:12.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:12.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618143+0000", 2024-09-18T00:19:12.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.551e-06 2024-09-18T00:19:12.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:12.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679967+0000", 2024-09-18T00:19:12.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061823937000000002 2024-09-18T00:19:12.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:12.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679975+0000", 2024-09-18T00:19:12.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7340000000000008e-06 2024-09-18T00:19:12.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:12.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:12.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:12.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680128+0000", 2024-09-18T00:19:12.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015338299999999999 2024-09-18T00:19:12.684 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.684 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.684 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.684 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:12.684 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:12.684 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:12.686 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:12.682+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:12.687 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:19:12.688 INFO:teuthology.orchestra.run.smithi097.stdout:4619 read 0x5f13816 thru 0x5f21317 (0xdb02 bytes) 2024-09-18T00:19:12.688 INFO:teuthology.orchestra.run.smithi097.stdout:4620 writesame 0xa398a3a thru 0xa3a20dc data_size 0x7(0x96a3 bytes) 2024-09-18T00:19:12.688 INFO:teuthology.orchestra.run.smithi097.stdout:4621 punch from 0x7abe533 to 0x7ac65ec, (0x80b9 bytes) 2024-09-18T00:19:12.688 INFO:teuthology.orchestra.run.smithi097.stdout:4622 punch from 0x679c9fd to 0x679f407, (0x2a0a bytes) 2024-09-18T00:19:12.688 INFO:teuthology.orchestra.run.smithi097.stdout:4623 read 0x1b66417 thru 0x1b6d5e4 (0x71ce bytes) 2024-09-18T00:19:12.688 INFO:teuthology.orchestra.run.smithi097.stdout:4624 write 0x210db27 thru 0x2115819 (0x7cf3 bytes) 2024-09-18T00:19:12.688 INFO:teuthology.orchestra.run.smithi097.stdout:4625 read 0x3ba888d thru 0x3bafd3d (0x74b1 bytes) 2024-09-18T00:19:12.688 INFO:teuthology.orchestra.run.smithi097.stdout:4626 read 0x96aeb79 thru 0x96b23e8 (0x3870 bytes) 2024-09-18T00:19:12.688 INFO:teuthology.orchestra.run.smithi097.stdout:4627 read 0xbcc4ecc thru 0xbcce072 (0x91a7 bytes) 2024-09-18T00:19:12.688 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:12.688 INFO:teuthology.orchestra.run.smithi097.stdout:4629 trunc from 0xdd9084b to 0x45ce71d 2024-09-18T00:19:12.688 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:12.688 INFO:teuthology.orchestra.run.smithi097.stdout:4632 write 0x491440 thru 0x498702 (0x72c3 bytes) 2024-09-18T00:19:12.689 INFO:teuthology.orchestra.run.smithi097.stdout:4633 trunc from 0x45ce71d to 0x66478ef 2024-09-18T00:19:12.689 INFO:teuthology.orchestra.run.smithi097.stdout:4635 writesame 0xdc115b0 thru 0xdc1a416 data_size 0x17(0x8e67 bytes) 2024-09-18T00:19:12.689 INFO:teuthology.orchestra.run.smithi097.stdout:4636 write 0xaabb4bb thru 0xaac0b80 (0x56c6 bytes) 2024-09-18T00:19:12.695 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:12.695 INFO:teuthology.orchestra.run.smithi097.stdout:4638 write 0x91ecf36 thru 0x91f9a2c (0xcaf7 bytes) 2024-09-18T00:19:12.728 INFO:teuthology.orchestra.run.smithi097.stdout:4639 writesame 0xe96008d thru 0xe96956c data_size 0x10(0x94e0 bytes) 2024-09-18T00:19:12.728 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:12.728 INFO:teuthology.orchestra.run.smithi097.stdout:4642 writesame 0xd5c8c04 thru 0xd5d5dc1 data_size 0x26(0xd1be bytes) 2024-09-18T00:19:12.746 INFO:teuthology.orchestra.run.smithi097.stdout:4643 punch from 0x8cd3826 to 0x8cd844c, (0x4c26 bytes) 2024-09-18T00:19:12.746 INFO:teuthology.orchestra.run.smithi097.stdout:4644 writesame 0xa14429c thru 0xa1500ed data_size 0x11(0xbe52 bytes) 2024-09-18T00:19:12.765 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:12.765 INFO:teuthology.orchestra.run.smithi097.stdout:4647 read 0xb9b2c3d thru 0xb9bf8e8 (0xccac bytes) 2024-09-18T00:19:12.765 INFO:teuthology.orchestra.run.smithi097.stdout:4648 punch from 0xda01eb3 to 0xda0ecde, (0xce2b bytes) 2024-09-18T00:19:12.765 INFO:teuthology.orchestra.run.smithi097.stdout:4650 read 0x1999676 thru 0x199adae (0x1739 bytes) 2024-09-18T00:19:12.766 INFO:teuthology.orchestra.run.smithi097.stdout:4652 read 0xa0eaacb thru 0xa0f3f23 (0x9459 bytes) 2024-09-18T00:19:12.766 INFO:teuthology.orchestra.run.smithi097.stdout:4653 read 0xa291935 thru 0xa294c80 (0x334c bytes) 2024-09-18T00:19:12.767 INFO:teuthology.orchestra.run.smithi097.stdout:4654 read 0xca806ee thru 0xca8df91 (0xd8a4 bytes) 2024-09-18T00:19:12.767 INFO:teuthology.orchestra.run.smithi097.stdout:4656 punch from 0xdbac9e4 to 0xdbb8194, (0xb7b0 bytes) 2024-09-18T00:19:12.767 INFO:teuthology.orchestra.run.smithi097.stdout:4658 read 0xe252bef thru 0xe260f5f (0xe371 bytes) 2024-09-18T00:19:12.777 INFO:teuthology.orchestra.run.smithi097.stdout:4660 writesame 0xebdeff3 thru 0xebe6561 data_size 0xb(0x756f bytes) 2024-09-18T00:19:12.777 INFO:teuthology.orchestra.run.smithi097.stdout:4661 read 0x6da0661 thru 0x6da26b9 (0x2059 bytes) 2024-09-18T00:19:12.777 INFO:teuthology.orchestra.run.smithi097.stdout:4662 read 0xb0a5032 thru 0xb0acb11 (0x7ae0 bytes) 2024-09-18T00:19:12.778 INFO:teuthology.orchestra.run.smithi097.stdout:4663 writesame 0x66a8af2 thru 0x66a9bad data_size 0x1(0x10bc bytes) 2024-09-18T00:19:12.783 INFO:teuthology.orchestra.run.smithi097.stdout:4665 writesame 0x13dde00 thru 0x13eaf65 data_size 0x31(0xd166 bytes) 2024-09-18T00:19:12.786 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:12.786+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:12.789 INFO:teuthology.orchestra.run.smithi097.stdout:4666 trunc from 0xebe6562 to 0xec318e8 2024-09-18T00:19:12.797 INFO:teuthology.orchestra.run.smithi097.stdout:4667 write 0x844b379 thru 0x844d5c6 (0x224e bytes) 2024-09-18T00:19:12.807 INFO:teuthology.orchestra.run.smithi097.stdout:4668 write 0x2606543 thru 0x26091ea (0x2ca8 bytes) 2024-09-18T00:19:12.811 INFO:teuthology.orchestra.run.smithi097.stdout:4671 write 0xa8e2f8b thru 0xa8e46c6 (0x173c bytes) 2024-09-18T00:19:12.814 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:12.814 INFO:teuthology.orchestra.run.smithi097.stdout:4674 write 0x239ee53 thru 0x23ac83b (0xd9e9 bytes) 2024-09-18T00:19:12.823 INFO:teuthology.orchestra.run.smithi097.stdout:4675 punch from 0x3911c5 to 0x3a0f42, (0xfd7d bytes) 2024-09-18T00:19:12.823 INFO:teuthology.orchestra.run.smithi097.stdout:4676 read 0x83b524f thru 0x83ba8b1 (0x5663 bytes) 2024-09-18T00:19:12.824 INFO:teuthology.orchestra.run.smithi097.stdout:4677 write 0x789200c thru 0x78930e9 (0x10de bytes) 2024-09-18T00:19:12.826 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:12.826 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:12.826 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:12.826 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:12.839 INFO:teuthology.orchestra.run.smithi097.stdout:4678 trunc from 0xec318e8 to 0xa60d7e1 2024-09-18T00:19:12.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:19:12.866 INFO:teuthology.orchestra.run.smithi097.stdout:4679 read 0x979b19 thru 0x98012d (0x6615 bytes) 2024-09-18T00:19:12.868 INFO:teuthology.orchestra.run.smithi097.stdout:4680 punch from 0x243d02e to 0x24433fe, (0x63d0 bytes) 2024-09-18T00:19:12.868 INFO:teuthology.orchestra.run.smithi097.stdout:4681 read 0x443cd7e thru 0x44464e0 (0x9763 bytes) 2024-09-18T00:19:12.883 INFO:teuthology.orchestra.run.smithi097.stdout:4683 write 0xdead33e thru 0xdeb5e9c (0x8b5f bytes) 2024-09-18T00:19:12.886 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:12.886+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:12.905 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:12.905 INFO:teuthology.orchestra.run.smithi097.stdout:4685 writesame 0x92ce1bc thru 0x92d1f77 data_size 0x12(0x3dbc bytes) 2024-09-18T00:19:12.909 INFO:teuthology.orchestra.run.smithi097.stdout:4686 writesame 0xb4555bc thru 0xb45ae9e data_size 0x29(0x58e3 bytes) 2024-09-18T00:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout:4687 read 0x6ae023e thru 0x6ae29cb (0x278e bytes) 2024-09-18T00:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout:4688 read 0x2c053ef thru 0x2c10f67 (0xbb79 bytes) 2024-09-18T00:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout:4689 punch from 0xaf57acf to 0xaf5d4d2, (0x5a03 bytes) 2024-09-18T00:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout:4690 read 0x5caa5fb thru 0x5cb0a62 (0x6468 bytes) 2024-09-18T00:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout:4692 read 0x2e6d330 thru 0x2e73fb1 (0x6c82 bytes) 2024-09-18T00:19:12.927 INFO:teuthology.orchestra.run.smithi097.stdout:4693 punch from 0x5f2fed7 to 0x5f3082d, (0x956 bytes) 2024-09-18T00:19:12.934 INFO:teuthology.orchestra.run.smithi097.stdout:4694 trunc from 0xdeb5e9d to 0x90ebd0b 2024-09-18T00:19:12.937 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:19:12.967 INFO:teuthology.orchestra.run.smithi097.stdout:4695 punch from 0x6dfac0d to 0x6dfe5d3, (0x39c6 bytes) 2024-09-18T00:19:12.967 INFO:teuthology.orchestra.run.smithi097.stdout:4696 read 0x4ed6c8f thru 0x4ee323f (0xc5b1 bytes) 2024-09-18T00:19:12.967 INFO:teuthology.orchestra.run.smithi097.stdout:4697 read 0x86c83b0 thru 0x86d2cae (0xa8ff bytes) 2024-09-18T00:19:12.967 INFO:teuthology.orchestra.run.smithi097.stdout:4699 write 0x4c02975 thru 0x4c09ff9 (0x7685 bytes) 2024-09-18T00:19:12.987 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:12.986+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:13.000 INFO:teuthology.orchestra.run.smithi097.stdout:4700 trunc from 0x90ebd0b to 0x1313c06 2024-09-18T00:19:13.014 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:13.014 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:13.015 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:13.015 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:13.036 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:13.061 INFO:teuthology.orchestra.run.smithi097.stdout:4701 write 0x4a21dde thru 0x4a2ff66 (0xe189 bytes) 2024-09-18T00:19:13.088 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:13.086+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:13.089 INFO:teuthology.orchestra.run.smithi097.stdout:4702 writesame 0x4f10a2f thru 0x4f1bb71 data_size 0x1(0xb143 bytes) 2024-09-18T00:19:13.093 INFO:teuthology.orchestra.run.smithi097.stdout:4703 trunc from 0x4f1bb72 to 0x20f7b10 2024-09-18T00:19:13.105 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:13.105 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:13.113 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:13.123 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:13.124 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:13.132 INFO:teuthology.orchestra.run.smithi097.stdout:4705 write 0x91d7e83 thru 0x91d8b2a (0xca8 bytes) 2024-09-18T00:19:13.142 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:13.142 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:13.143 INFO:teuthology.orchestra.run.smithi097.stdout:4707 trunc from 0x91d8b2b to 0x9ee6c71 2024-09-18T00:19:13.164 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:13.164 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:13.164 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-18T00:19:13.168 INFO:teuthology.orchestra.run.smithi097.stdout:4708 write 0xc2c31b1 thru 0xc2cc7c3 (0x9613 bytes) 2024-09-18T00:19:13.189 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:13.185+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:13.189 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:13.189 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:13.196 INFO:teuthology.orchestra.run.smithi097.stdout:4709 trunc from 0xc2cc7c4 to 0x853bc98 2024-09-18T00:19:13.197 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:13.197 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:13.213 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:13.213 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:13.214 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:19:13.216 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:13.216 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:13.218 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:13.229 INFO:teuthology.orchestra.run.smithi097.stdout:4711 write 0x9569ca0 thru 0x957211d (0x847e bytes) 2024-09-18T00:19:13.250 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:13.250 INFO:teuthology.orchestra.run.smithi097.stdout:4714 read 0x937292e thru 0x937fe66 (0xd539 bytes) 2024-09-18T00:19:13.252 INFO:teuthology.orchestra.run.smithi097.stdout:4715 punch from 0xf87791 to 0xf91229, (0x9a98 bytes) 2024-09-18T00:19:13.252 INFO:teuthology.orchestra.run.smithi097.stdout:4716 read 0x4d0fe29 thru 0x4d171b9 (0x7391 bytes) 2024-09-18T00:19:13.267 INFO:teuthology.orchestra.run.smithi097.stdout:4721 write 0xca75483 thru 0xca803e8 (0xaf66 bytes) 2024-09-18T00:19:13.284 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is unset 2024-09-18T00:19:13.285 INFO:teuthology.orchestra.run.smithi097.stdout:4722 punch from 0x9a1da34 to 0x9a20c16, (0x31e2 bytes) 2024-09-18T00:19:13.285 INFO:teuthology.orchestra.run.smithi097.stdout:4723 punch from 0x72905af to 0x7292732, (0x2183 bytes) 2024-09-18T00:19:13.286 INFO:teuthology.orchestra.run.smithi097.stdout:4724 writesame 0x9b72f53 thru 0x9b8164d data_size 0x1(0xe6fb bytes) 2024-09-18T00:19:13.289 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:13.286+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:13.321 INFO:teuthology.orchestra.run.smithi097.stdout:4725 trunc from 0xca803e9 to 0xc8aed10 2024-09-18T00:19:13.327 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:13.326+0000 7f73ce4a1640 -1 osd.5 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:19:13.335 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:13.337 INFO:teuthology.orchestra.run.smithi097.stdout:4729 trunc from 0xc8aed10 to 0xb00d615 2024-09-18T00:19:13.349 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:13.349 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:13.353 INFO:teuthology.orchestra.run.smithi097.stdout:4731 read 0x1e14472 thru 0x1e1c28b (0x7e1a bytes) 2024-09-18T00:19:13.353 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:13.353 INFO:teuthology.orchestra.run.smithi097.stdout:4734 read 0x3663f54 thru 0x36727a7 (0xe854 bytes) 2024-09-18T00:19:13.353 INFO:teuthology.orchestra.run.smithi097.stdout:4735 write 0x3ec8b19 thru 0x3ed4fb0 (0xc498 bytes) 2024-09-18T00:19:13.363 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:19:13.373 INFO:teuthology.orchestra.run.smithi097.stdout:4736 read 0xa8a0b6 thru 0xa8cbad (0x2af8 bytes) 2024-09-18T00:19:13.383 INFO:teuthology.orchestra.run.smithi097.stdout:4737 trunc from 0xb00d615 to 0x606f1d 2024-09-18T00:19:13.390 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:13.390+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:13.424 INFO:teuthology.orchestra.run.smithi097.stdout:4740 read 0x3eeebc thru 0x3f12a1 (0x23e6 bytes) 2024-09-18T00:19:13.425 INFO:teuthology.orchestra.run.smithi097.stdout:4741 read 0x4bf0a3 thru 0x4c5324 (0x6282 bytes) 2024-09-18T00:19:13.427 INFO:teuthology.orchestra.run.smithi097.stdout:4742 read 0x51459 thru 0x609e5 (0xf58d bytes) 2024-09-18T00:19:13.445 INFO:teuthology.orchestra.run.smithi097.stdout:4744 writesame 0x561310a thru 0x5613ba6 data_size 0xd(0xa9d bytes) 2024-09-18T00:19:13.445 INFO:teuthology.orchestra.run.smithi097.stdout:4745 write 0x32ca86b thru 0x32cc91f (0x20b5 bytes) 2024-09-18T00:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout:4747 writesame 0xe8116c0 thru 0xe813cea data_size 0x1(0x262b bytes) 2024-09-18T00:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout:4748 read 0xa2310de thru 0xa2355ae (0x44d1 bytes) 2024-09-18T00:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout:4750 punch from 0xa482895 to 0xa48e5ae, (0xbd19 bytes) 2024-09-18T00:19:13.483 INFO:teuthology.orchestra.run.smithi097.stdout:4751 write 0x1178a19 thru 0x117ff6d (0x7555 bytes) 2024-09-18T00:19:13.490 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:13.490+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:13.500 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:13.500 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:13.504 INFO:teuthology.orchestra.run.smithi097.stdout:4752 write 0xbcb448d thru 0xbcb6db9 (0x292d bytes) 2024-09-18T00:19:13.515 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:19:13.516 INFO:teuthology.orchestra.run.smithi097.stdout:4754 writesame 0x7f6fb29 thru 0x7f7b73b data_size 0xb(0xbc13 bytes) 2024-09-18T00:19:13.535 INFO:teuthology.orchestra.run.smithi097.stdout:4755 writesame 0x88a42e6 thru 0x88acfe5 data_size 0x8d(0x8d00 bytes) 2024-09-18T00:19:13.558 INFO:teuthology.orchestra.run.smithi097.stdout:4756 write 0xdaa1fd9 thru 0xdaad486 (0xb4ae bytes) 2024-09-18T00:19:13.579 INFO:teuthology.orchestra.run.smithi097.stdout:4758 read 0xd824617 thru 0xd8285cf (0x3fb9 bytes) 2024-09-18T00:19:13.579 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:13.581 INFO:teuthology.orchestra.run.smithi097.stdout:4760 trunc from 0xe813ceb to 0xcdeac95 2024-09-18T00:19:13.591 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:13.585+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:13.600 INFO:teuthology.orchestra.run.smithi097.stdout:4763 write 0x291a2ec thru 0x2921060 (0x6d75 bytes) 2024-09-18T00:19:13.618 INFO:teuthology.orchestra.run.smithi097.stdout:4765 writesame 0x9f4c38f thru 0x9f51a1c data_size 0x9(0x568e bytes) 2024-09-18T00:19:13.635 INFO:teuthology.orchestra.run.smithi097.stdout:4766 write 0x247f864 thru 0x248cece (0xd66b bytes) 2024-09-18T00:19:13.642 INFO:teuthology.orchestra.run.smithi097.stdout:4769 punch from 0xabe1d to 0xb1ef3, (0x60d6 bytes) 2024-09-18T00:19:13.643 INFO:teuthology.orchestra.run.smithi097.stdout:4771 read 0x85f3296 thru 0x85fa63f (0x73aa bytes) 2024-09-18T00:19:13.645 INFO:teuthology.orchestra.run.smithi097.stdout:4772 writesame 0x8efcdf4 thru 0x8f01c29 data_size 0x47(0x4e36 bytes) 2024-09-18T00:19:13.661 INFO:teuthology.orchestra.run.smithi097.stdout:4773 write 0x476bd30 thru 0x476e997 (0x2c68 bytes) 2024-09-18T00:19:13.665 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:13.666 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:13.676 INFO:teuthology.orchestra.run.smithi097.stdout:4774 writesame 0x16cc9db thru 0x16d1a68 data_size 0x15(0x508e bytes) 2024-09-18T00:19:13.682 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:19:13.692 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:13.690+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:13.694 INFO:teuthology.orchestra.run.smithi097.stdout:4775 read 0x9da1b61 thru 0x9da4d31 (0x31d1 bytes) 2024-09-18T00:19:13.696 INFO:teuthology.orchestra.run.smithi097.stdout:4776 writesame 0xcba09db thru 0xcba9329 data_size 0x1(0x894f bytes) 2024-09-18T00:19:13.719 INFO:teuthology.orchestra.run.smithi097.stdout:4777 write 0x7cd63c1 thru 0x7ce4649 (0xe289 bytes) 2024-09-18T00:19:13.742 INFO:teuthology.orchestra.run.smithi097.stdout:4778 writesame 0x107c443 thru 0x1085418 data_size 0x6(0x8fd6 bytes) 2024-09-18T00:19:13.752 INFO:teuthology.orchestra.run.smithi097.stdout:4779 trunc from 0xcdeac95 to 0xb52a3e4 2024-09-18T00:19:13.786 INFO:teuthology.orchestra.run.smithi097.stdout:4780 writesame 0xcdc18e5 thru 0xcdc25e1 data_size 0x7(0xcfd bytes) 2024-09-18T00:19:13.786 INFO:teuthology.orchestra.run.smithi097.stdout:4781 writesame 0xbe28a98 thru 0xbe32a7d data_size 0x2(0x9fe6 bytes) 2024-09-18T00:19:13.793 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:13.789+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:13.823 INFO:teuthology.orchestra.run.smithi097.stdout:4784 trunc from 0xcdc25e2 to 0x76c761 2024-09-18T00:19:13.824 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:13.824 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:13.843 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:19:13.903 INFO:teuthology.orchestra.run.smithi097.stdout:4785 trunc from 0x76c761 to 0xc7ac07b 2024-09-18T00:19:13.903 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:13.897+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:13.913 INFO:teuthology.orchestra.run.smithi097.stdout:4786 punch from 0x3b6efc9 to 0x3b733f5, (0x442c bytes) 2024-09-18T00:19:13.913 INFO:teuthology.orchestra.run.smithi097.stdout:4789 punch from 0x7bc4a8e to 0x7bc8161, (0x36d3 bytes) 2024-09-18T00:19:13.913 INFO:teuthology.orchestra.run.smithi097.stdout:4790 writesame 0xa7477c2 thru 0xa74b7c6 data_size 0x9(0x4005 bytes) 2024-09-18T00:19:13.929 INFO:teuthology.orchestra.run.smithi097.stdout:4792 read 0x33dde1b thru 0x33ed457 (0xf63d bytes) 2024-09-18T00:19:13.929 INFO:teuthology.orchestra.run.smithi097.stdout:4793 write 0x5370bd3 thru 0x537336d (0x279b bytes) 2024-09-18T00:19:13.941 INFO:teuthology.orchestra.run.smithi097.stdout:4794 punch from 0xa436068 to 0xa442527, (0xc4bf bytes) 2024-09-18T00:19:13.941 INFO:teuthology.orchestra.run.smithi097.stdout:4795 punch from 0xafc4df6 to 0xafd3379, (0xe583 bytes) 2024-09-18T00:19:13.941 INFO:teuthology.orchestra.run.smithi097.stdout:4796 read 0x231d720 thru 0x23220f5 (0x49d6 bytes) 2024-09-18T00:19:13.941 INFO:teuthology.orchestra.run.smithi097.stdout:4797 write 0x5ba058c thru 0x5bad099 (0xcb0e bytes) 2024-09-18T00:19:13.972 INFO:teuthology.orchestra.run.smithi097.stdout:4798 write 0xd064ff1 thru 0xd073764 (0xe774 bytes) 2024-09-18T00:19:13.994 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:13.994+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:13.996 INFO:teuthology.orchestra.run.smithi097.stdout:4799 write 0x6546539 thru 0x6547903 (0x13cb bytes) 2024-09-18T00:19:14.002 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:14.002 INFO:teuthology.orchestra.run.smithi097.stdout:4801 writesame 0x8a2c884 thru 0x8a398a1 data_size 0x11(0xd01e bytes) 2024-09-18T00:19:14.013 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:14.013 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:14.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:19:14.035 INFO:teuthology.orchestra.run.smithi097.stdout:4802 trunc from 0xd073765 to 0x3fa4d4b 2024-09-18T00:19:14.083 INFO:teuthology.orchestra.run.smithi097.stdout:4803 writesame 0x4888983 thru 0x488dc10 data_size 0x1(0x528e bytes) 2024-09-18T00:19:14.095 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:14.094+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:14.104 INFO:teuthology.orchestra.run.smithi097.stdout:4804 writesame 0xbdbaafd thru 0xbdc83fa data_size 0x65(0xd8fe bytes) 2024-09-18T00:19:14.104 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:14.104 INFO:teuthology.orchestra.run.smithi097.stdout:4807 write 0x6c06a62 thru 0x6c163c9 (0xf968 bytes) 2024-09-18T00:19:14.147 INFO:teuthology.orchestra.run.smithi097.stdout:4809 writesame 0xd908dad thru 0xd910717 data_size 0xd(0x796b bytes) 2024-09-18T00:19:14.147 INFO:teuthology.orchestra.run.smithi097.stdout:4811 write 0x2c01469 thru 0x2c01896 (0x42e bytes) 2024-09-18T00:19:14.154 INFO:teuthology.orchestra.run.smithi097.stdout:4812 read 0x717aa86 thru 0x718a35c (0xf8d7 bytes) 2024-09-18T00:19:14.155 INFO:teuthology.orchestra.run.smithi097.stdout:4813 punch from 0x3b1a188 to 0x3b28e44, (0xecbc bytes) 2024-09-18T00:19:14.155 INFO:teuthology.orchestra.run.smithi097.stdout:4814 writesame 0x870c8b7 thru 0x8714e59 data_size 0x1(0x85a3 bytes) 2024-09-18T00:19:14.170 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:14.170 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:14.186 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:19:14.191 INFO:teuthology.orchestra.run.smithi097.stdout:4816 read 0x91aabfb thru 0x91b2f6c (0x8372 bytes) 2024-09-18T00:19:14.192 INFO:teuthology.orchestra.run.smithi097.stdout:4817 trunc from 0xd910718 to 0xd2feb05 2024-09-18T00:19:14.208 INFO:teuthology.orchestra.run.smithi097.stdout:4818 trunc from 0xd2feb05 to 0x9faf091 2024-09-18T00:19:14.217 INFO:teuthology.orchestra.run.smithi097.stdout:4819 write 0x566e0de thru 0x5673b3e (0x5a61 bytes) 2024-09-18T00:19:14.234 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:14.234+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:14.250 INFO:teuthology.orchestra.run.smithi097.stdout:4822 read 0x4393bf thru 0x440050 (0x6c92 bytes) 2024-09-18T00:19:14.252 INFO:teuthology.orchestra.run.smithi097.stdout:4824 writesame 0x121ebe9 thru 0x122bef3 data_size 0x57(0xd30b bytes) 2024-09-18T00:19:14.297 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:14.293+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:14.328 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:14.328 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:14.342 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:19:14.430 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:14.430+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:14.498 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:14.494+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:14.508 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:14.508 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:14.522 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:19:14.614 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:14.614+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:14.689 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:14.689 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:14.700 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:14.698+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:14.703 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:19:14.801 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:14.798+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:14.867 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:14.868 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:14.881 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:19:14.902 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:14.901+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:15.003 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:15.001+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:15.045 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:15.045 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:15.060 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:19:15.104 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:15.097+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:15.185 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:15.185 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:15.199 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:19:15.205 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:15.201+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:15.217 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:19:15.305 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:15.302+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:15.311 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:19:15.330 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:15.330 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:15.345 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:19:15.378 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:15.379 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:15.379 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:15.396 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:15.396 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:15.406 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:15.401+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:15.412 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:15.413 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:15.430 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:15.430 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:15.453 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:15.454 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-18T00:19:15.473 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:15.473 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:15.493 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:15.494 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:15.497 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:15.497 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:15.508 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:15.506+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:15.517 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:15.517 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:15.517 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:15.517 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:15.517 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:15.531 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:19:15.618 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:15.618+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:15.700 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2228 3.15 3:aa2fc059:::rbd_data.112aa7ba4578.000000000000014a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.433280316999998, 2024-09-18T00:19:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.068966377999999995, 2024-09-18T00:19:15.700 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2228 2024-09-18T00:19:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263099+0000", 2024-09-18T00:19:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2359999999999999e-06 2024-09-18T00:19:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263115+0000", 2024-09-18T00:19:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5996999999999999e-05 2024-09-18T00:19:15.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263156+0000", 2024-09-18T00:19:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:19:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263183+0000", 2024-09-18T00:19:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7233000000000002e-05 2024-09-18T00:19:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263304+0000", 2024-09-18T00:19:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121209 2024-09-18T00:19:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263528+0000", 2024-09-18T00:19:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022401800000000001 2024-09-18T00:19:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:19:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.266458+0000", 2024-09-18T00:19:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029296980000000001 2024-09-18T00:19:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331628+0000", 2024-09-18T00:19:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.065170045999999995 2024-09-18T00:19:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331819+0000", 2024-09-18T00:19:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000190819 2024-09-18T00:19:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331971+0000", 2024-09-18T00:19:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151972 2024-09-18T00:19:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332063+0000", 2024-09-18T00:19:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2083999999999995e-05 2024-09-18T00:19:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.705 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2228 3.15 e36/32 ondisk, result = 0)", 2024-09-18T00:19:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:19:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.420227205000003, 2024-09-18T00:19:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055930148999999998, 2024-09-18T00:19:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:19:15.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276150+0000", 2024-09-18T00:19:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276152+0000", 2024-09-18T00:19:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0200000000000001e-06 2024-09-18T00:19:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276162+0000", 2024-09-18T00:19:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0767e-05 2024-09-18T00:19:15.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276202+0000", 2024-09-18T00:19:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000003e-05 2024-09-18T00:19:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.276226+0000", 2024-09-18T00:19:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.351e-05 2024-09-18T00:19:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331782+0000", 2024-09-18T00:19:15.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055556296999999998 2024-09-18T00:19:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331801+0000", 2024-09-18T00:19:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9357000000000001e-05 2024-09-18T00:19:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332080+0000", 2024-09-18T00:19:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027849800000000002 2024-09-18T00:19:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2854 3.f 3:f02b0bca:::rbd_data.112aa7ba4578.0000000000000135:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:19:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 53.826307827000001, 2024-09-18T00:19:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.082582802999999996, 2024-09-18T00:19:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2854 2024-09-18T00:19:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870071+0000", 2024-09-18T00:19:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.227e-06 2024-09-18T00:19:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870083+0000", 2024-09-18T00:19:15.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1909999999999999e-05 2024-09-18T00:19:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870114+0000", 2024-09-18T00:19:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0576e-05 2024-09-18T00:19:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870129+0000", 2024-09-18T00:19:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.485e-05 2024-09-18T00:19:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870256+0000", 2024-09-18T00:19:15.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012691300000000001 2024-09-18T00:19:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870493+0000", 2024-09-18T00:19:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023737399999999999 2024-09-18T00:19:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.874933+0000", 2024-09-18T00:19:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044398299999999996 2024-09-18T00:19:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952452+0000", 2024-09-18T00:19:15.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.077518918000000006 2024-09-18T00:19:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952486+0000", 2024-09-18T00:19:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4035999999999997e-05 2024-09-18T00:19:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952575+0000", 2024-09-18T00:19:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.954e-05 2024-09-18T00:19:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952652+0000", 2024-09-18T00:19:15.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6629000000000001e-05 2024-09-18T00:19:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2854 3.f e38/32 ondisk, result = 0)", 2024-09-18T00:19:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 53.808777012, 2024-09-18T00:19:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064868625999999999, 2024-09-18T00:19:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887602+0000", 2024-09-18T00:19:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.677e-06 2024-09-18T00:19:15.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887609+0000", 2024-09-18T00:19:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6729999999999992e-06 2024-09-18T00:19:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887637+0000", 2024-09-18T00:19:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8206000000000001e-05 2024-09-18T00:19:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887651+0000", 2024-09-18T00:19:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3387000000000001e-05 2024-09-18T00:19:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952422+0000", 2024-09-18T00:19:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064771366999999996 2024-09-18T00:19:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952441+0000", 2024-09-18T00:19:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8737e-05 2024-09-18T00:19:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952469+0000", 2024-09-18T00:19:15.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7579000000000001e-05 2024-09-18T00:19:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5676 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 12255232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:19:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.398101154999999, 2024-09-18T00:19:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047571816000000003, 2024-09-18T00:19:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:15.719 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5676 2024-09-18T00:19:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:19:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298276+0000", 2024-09-18T00:19:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298277+0000", 2024-09-18T00:19:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.454e-06 2024-09-18T00:19:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298282+0000", 2024-09-18T00:19:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6299999999999997e-06 2024-09-18T00:19:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298299+0000", 2024-09-18T00:19:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6959000000000001e-05 2024-09-18T00:19:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298313+0000", 2024-09-18T00:19:15.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.376e-05 2024-09-18T00:19:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345610+0000", 2024-09-18T00:19:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047297128000000001 2024-09-18T00:19:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345708+0000", 2024-09-18T00:19:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8245999999999994e-05 2024-09-18T00:19:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345848+0000", 2024-09-18T00:19:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000139639 2024-09-18T00:19:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5677 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 12779520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:19:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.398047642000002, 2024-09-18T00:19:15.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047810679000000002, 2024-09-18T00:19:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5677 2024-09-18T00:19:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:19:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298329+0000", 2024-09-18T00:19:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298330+0000", 2024-09-18T00:19:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.217e-06 2024-09-18T00:19:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298354+0000", 2024-09-18T00:19:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3753000000000001e-05 2024-09-18T00:19:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298388+0000", 2024-09-18T00:19:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4276999999999997e-05 2024-09-18T00:19:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298415+0000", 2024-09-18T00:19:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6403000000000001e-05 2024-09-18T00:19:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345896+0000", 2024-09-18T00:19:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047481007999999998 2024-09-18T00:19:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.345992+0000", 2024-09-18T00:19:15.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6141e-05 2024-09-18T00:19:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346140+0000", 2024-09-18T00:19:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014788 2024-09-18T00:19:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5678 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:19:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.397941463000002, 2024-09-18T00:19:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047985466999999997, 2024-09-18T00:19:15.727 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5678 2024-09-18T00:19:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:19:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298435+0000", 2024-09-18T00:19:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298437+0000", 2024-09-18T00:19:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1799999999999999e-06 2024-09-18T00:19:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298443+0000", 2024-09-18T00:19:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1140000000000002e-06 2024-09-18T00:19:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298460+0000", 2024-09-18T00:19:15.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7615999999999999e-05 2024-09-18T00:19:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298474+0000", 2024-09-18T00:19:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4017e-05 2024-09-18T00:19:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346180+0000", 2024-09-18T00:19:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047705801999999999 2024-09-18T00:19:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346274+0000", 2024-09-18T00:19:15.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3814999999999995e-05 2024-09-18T00:19:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346421+0000", 2024-09-18T00:19:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000146923 2024-09-18T00:19:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5679 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 14352384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:19:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.397882418999998, 2024-09-18T00:19:15.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048190771, 2024-09-18T00:19:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5679 2024-09-18T00:19:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:19:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298494+0000", 2024-09-18T00:19:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298496+0000", 2024-09-18T00:19:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4699999999999999e-06 2024-09-18T00:19:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298501+0000", 2024-09-18T00:19:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5199999999999997e-06 2024-09-18T00:19:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298519+0000", 2024-09-18T00:19:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7329e-05 2024-09-18T00:19:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298532+0000", 2024-09-18T00:19:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3083e-05 2024-09-18T00:19:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346462+0000", 2024-09-18T00:19:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047930449999999999 2024-09-18T00:19:15.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346556+0000", 2024-09-18T00:19:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3222999999999998e-05 2024-09-18T00:19:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346685+0000", 2024-09-18T00:19:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000129696 2024-09-18T00:19:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.735 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5680 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:19:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.397825009999998, 2024-09-18T00:19:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048535847, 2024-09-18T00:19:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5680 2024-09-18T00:19:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:19:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298552+0000", 2024-09-18T00:19:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298553+0000", 2024-09-18T00:19:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0759999999999999e-06 2024-09-18T00:19:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298575+0000", 2024-09-18T00:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2019999999999999e-05 2024-09-18T00:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298593+0000", 2024-09-18T00:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7900000000000001e-05 2024-09-18T00:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298607+0000", 2024-09-18T00:19:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3844e-05 2024-09-18T00:19:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346791+0000", 2024-09-18T00:19:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048184449999999997 2024-09-18T00:19:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.346921+0000", 2024-09-18T00:19:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012994699999999999 2024-09-18T00:19:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347088+0000", 2024-09-18T00:19:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016661000000000001 2024-09-18T00:19:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5681 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:19:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.397752599999997, 2024-09-18T00:19:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048743337999999997, 2024-09-18T00:19:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:15.740 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5681 2024-09-18T00:19:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:19:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298624+0000", 2024-09-18T00:19:15.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298626+0000", 2024-09-18T00:19:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4899999999999999e-06 2024-09-18T00:19:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298631+0000", 2024-09-18T00:19:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0300000000000001e-06 2024-09-18T00:19:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298650+0000", 2024-09-18T00:19:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9499000000000001e-05 2024-09-18T00:19:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298664+0000", 2024-09-18T00:19:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3767e-05 2024-09-18T00:19:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347139+0000", 2024-09-18T00:19:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048475348000000001 2024-09-18T00:19:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347228+0000", 2024-09-18T00:19:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8843000000000003e-05 2024-09-18T00:19:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347368+0000", 2024-09-18T00:19:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013936100000000001 2024-09-18T00:19:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5682 3.f 3:f154d4d5:::rbd_data.112a960351d8.0000000000000002:6 [sparse-read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:15.744 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:19:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.397695785000003, 2024-09-18T00:19:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048916864999999997, 2024-09-18T00:19:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5682 2024-09-18T00:19:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:19:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298681+0000", 2024-09-18T00:19:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298682+0000", 2024-09-18T00:19:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2899999999999999e-06 2024-09-18T00:19:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298687+0000", 2024-09-18T00:19:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9239999999999998e-06 2024-09-18T00:19:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298704+0000", 2024-09-18T00:19:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6569999999999999e-05 2024-09-18T00:19:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.298731+0000", 2024-09-18T00:19:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6752999999999999e-05 2024-09-18T00:19:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347416+0000", 2024-09-18T00:19:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048685056999999997 2024-09-18T00:19:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347506+0000", 2024-09-18T00:19:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9936000000000001e-05 2024-09-18T00:19:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.347598+0000", 2024-09-18T00:19:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2335000000000003e-05 2024-09-18T00:19:15.748 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15879 3.16 3:692b47b6:::rbd_object_map.112a13ec2d6c.000000000000000b:head [call] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:19:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.041613105, 2024-09-18T00:19:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.011184018, 2024-09-18T00:19:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:15.749 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15879 2024-09-18T00:19:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654768+0000", 2024-09-18T00:19:15.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9129999999999996e-06 2024-09-18T00:19:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654786+0000", 2024-09-18T00:19:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8695999999999999e-05 2024-09-18T00:19:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654832+0000", 2024-09-18T00:19:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5735000000000002e-05 2024-09-18T00:19:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654860+0000", 2024-09-18T00:19:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7789000000000001e-05 2024-09-18T00:19:15.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655048+0000", 2024-09-18T00:19:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018781900000000001 2024-09-18T00:19:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:19:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655070+0000", 2024-09-18T00:19:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1916000000000002e-05 2024-09-18T00:19:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663335+0000", 2024-09-18T00:19:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0082651210000000003 2024-09-18T00:19:15.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663358+0000", 2024-09-18T00:19:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2986e-05 2024-09-18T00:19:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663739+0000", 2024-09-18T00:19:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0003814820000001 2024-09-18T00:19:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663920+0000", 2024-09-18T00:19:15.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018116900000000001 2024-09-18T00:19:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.665948+0000", 2024-09-18T00:19:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020273919999999998 2024-09-18T00:19:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15881 3.16 3:6cc14920:::rbd_header.112a3285400b:head [watch reconnect cookie 140685950184608 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:19:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.041452110000002, 2024-09-18T00:19:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0138723599999999, 2024-09-18T00:19:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.754 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15881 2024-09-18T00:19:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654926+0000", 2024-09-18T00:19:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6470000000000001e-06 2024-09-18T00:19:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654935+0000", 2024-09-18T00:19:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8859999999999993e-06 2024-09-18T00:19:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654960+0000", 2024-09-18T00:19:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4689999999999999e-05 2024-09-18T00:19:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654976+0000", 2024-09-18T00:19:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6319000000000001e-05 2024-09-18T00:19:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655238+0000", 2024-09-18T00:19:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026145999999999998 2024-09-18T00:19:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655255+0000", 2024-09-18T00:19:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7249999999999999e-05 2024-09-18T00:19:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663416+0000", 2024-09-18T00:19:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081607959999999997 2024-09-18T00:19:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663436+0000", 2024-09-18T00:19:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0384e-05 2024-09-18T00:19:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666184+0000", 2024-09-18T00:19:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002748121 2024-09-18T00:19:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666452+0000", 2024-09-18T00:19:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026801700000000001 2024-09-18T00:19:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.668797+0000", 2024-09-18T00:19:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023447899999999998 2024-09-18T00:19:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.759 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17089 3.f 3:f7681386:::rbd_data.112a6efb7564.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:19:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.816525014, 2024-09-18T00:19:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.12573889399999999, 2024-09-18T00:19:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:15.760 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17089 2024-09-18T00:19:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879854+0000", 2024-09-18T00:19:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:19:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879867+0000", 2024-09-18T00:19:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3103e-05 2024-09-18T00:19:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879898+0000", 2024-09-18T00:19:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0103e-05 2024-09-18T00:19:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879912+0000", 2024-09-18T00:19:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4636e-05 2024-09-18T00:19:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879992+0000", 2024-09-18T00:19:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9814000000000005e-05 2024-09-18T00:19:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.880101+0000", 2024-09-18T00:19:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109172 2024-09-18T00:19:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.887269+0000", 2024-09-18T00:19:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071679300000000003 2024-09-18T00:19:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005269+0000", 2024-09-18T00:19:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.117999621 2024-09-18T00:19:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005317+0000", 2024-09-18T00:19:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8279000000000003e-05 2024-09-18T00:19:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005464+0000", 2024-09-18T00:19:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014661200000000001 2024-09-18T00:19:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005591+0000", 2024-09-18T00:19:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000127107 2024-09-18T00:19:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.765 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17089 3.f e59/32 ondisk, result = 0)", 2024-09-18T00:19:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.792418149, 2024-09-18T00:19:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101334403, 2024-09-18T00:19:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903960+0000", 2024-09-18T00:19:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.33e-06 2024-09-18T00:19:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903966+0000", 2024-09-18T00:19:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5929999999999996e-06 2024-09-18T00:19:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903986+0000", 2024-09-18T00:19:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.018e-05 2024-09-18T00:19:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903997+0000", 2024-09-18T00:19:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0988999999999999e-05 2024-09-18T00:19:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005171+0000", 2024-09-18T00:19:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101174267 2024-09-18T00:19:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005200+0000", 2024-09-18T00:19:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8540000000000001e-05 2024-09-18T00:19:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005293+0000", 2024-09-18T00:19:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504000000000005e-05 2024-09-18T00:19:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.769 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24529 3.16 3:6857ce1a:::rbd_data.112a990280ef.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.0596067379999994, 2024-09-18T00:19:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10693805200000001, 2024-09-18T00:19:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24529 2024-09-18T00:19:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636772+0000", 2024-09-18T00:19:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1100000000000001e-06 2024-09-18T00:19:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636783+0000", 2024-09-18T00:19:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1005999999999999e-05 2024-09-18T00:19:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636814+0000", 2024-09-18T00:19:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1047000000000003e-05 2024-09-18T00:19:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636829+0000", 2024-09-18T00:19:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4363e-05 2024-09-18T00:19:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636980+0000", 2024-09-18T00:19:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015181099999999999 2024-09-18T00:19:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.637237+0000", 2024-09-18T00:19:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000256352 2024-09-18T00:19:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743536+0000", 2024-09-18T00:19:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.106298825 2024-09-18T00:19:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:15.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743599+0000", 2024-09-18T00:19:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3449999999999997e-05 2024-09-18T00:19:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743708+0000", 2024-09-18T00:19:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109088 2024-09-18T00:19:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24805 3.16 3:6de5e15d:::rbd_data.112a990280ef.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:15.774 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.4738035059999994, 2024-09-18T00:19:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14775107700000001, 2024-09-18T00:19:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24805 2024-09-18T00:19:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:15.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:15.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222574+0000", 2024-09-18T00:19:15.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0210000000000001e-06 2024-09-18T00:19:15.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222581+0000", 2024-09-18T00:19:15.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.144e-06 2024-09-18T00:19:15.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222595+0000", 2024-09-18T00:19:15.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4567e-05 2024-09-18T00:19:15.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222603+0000", 2024-09-18T00:19:15.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6529999999999994e-06 2024-09-18T00:19:15.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222687+0000", 2024-09-18T00:19:15.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4587999999999997e-05 2024-09-18T00:19:15.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222828+0000", 2024-09-18T00:19:15.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140178 2024-09-18T00:19:15.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:15.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370231+0000", 2024-09-18T00:19:15.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14740378700000001 2024-09-18T00:19:15.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:15.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370269+0000", 2024-09-18T00:19:15.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7886999999999998e-05 2024-09-18T00:19:15.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370324+0000", 2024-09-18T00:19:15.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5251999999999999e-05 2024-09-18T00:19:15.779 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.779 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.779 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.780 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24852 3.13 3:cc64f0fd:::rbd_data.112a990280ef.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:15.780 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:15.780 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.2410640359999991, 2024-09-18T00:19:15.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.066392943999999995, 2024-09-18T00:19:15.780 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.780 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.780 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:15.780 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:15.780 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:15.780 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:15.780 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24852 2024-09-18T00:19:15.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.780 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:15.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:15.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455315+0000", 2024-09-18T00:19:15.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.705e-06 2024-09-18T00:19:15.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455324+0000", 2024-09-18T00:19:15.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8120000000000002e-06 2024-09-18T00:19:15.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455349+0000", 2024-09-18T00:19:15.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4250999999999999e-05 2024-09-18T00:19:15.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455361+0000", 2024-09-18T00:19:15.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2805e-05 2024-09-18T00:19:15.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455425+0000", 2024-09-18T00:19:15.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3981999999999998e-05 2024-09-18T00:19:15.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455480+0000", 2024-09-18T00:19:15.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4792e-05 2024-09-18T00:19:15.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:15.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.460671+0000", 2024-09-18T00:19:15.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051907300000000002 2024-09-18T00:19:15.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:15.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521355+0000", 2024-09-18T00:19:15.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060684208000000003 2024-09-18T00:19:15.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:15.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521493+0000", 2024-09-18T00:19:15.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013803200000000001 2024-09-18T00:19:15.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:15.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521590+0000", 2024-09-18T00:19:15.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6339e-05 2024-09-18T00:19:15.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521706+0000", 2024-09-18T00:19:15.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116288 2024-09-18T00:19:15.785 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.785 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.785 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.785 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:25269 3.f 3:f6adfd48:::rbd_data.112aa34d8ec3.0000000000000145:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T00:19:15.785 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:15.785 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.0914740749999998, 2024-09-18T00:19:15.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.075209085999999994, 2024-09-18T00:19:15.786 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.786 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.786 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:15.786 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:15.786 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:15.786 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:15.786 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 25269 2024-09-18T00:19:15.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.786 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:15.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:15.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604904+0000", 2024-09-18T00:19:15.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:19:15.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604910+0000", 2024-09-18T00:19:15.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.964e-06 2024-09-18T00:19:15.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604925+0000", 2024-09-18T00:19:15.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4894e-05 2024-09-18T00:19:15.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604932+0000", 2024-09-18T00:19:15.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-18T00:19:15.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604990+0000", 2024-09-18T00:19:15.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8143000000000002e-05 2024-09-18T00:19:15.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.605104+0000", 2024-09-18T00:19:15.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011404399999999999 2024-09-18T00:19:15.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:15.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.608755+0000", 2024-09-18T00:19:15.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003651081 2024-09-18T00:19:15.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:15.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679920+0000", 2024-09-18T00:19:15.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.071164543999999996 2024-09-18T00:19:15.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:15.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679978+0000", 2024-09-18T00:19:15.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8585000000000002e-05 2024-09-18T00:19:15.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:15.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680036+0000", 2024-09-18T00:19:15.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7978e-05 2024-09-18T00:19:15.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680112+0000", 2024-09-18T00:19:15.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5527999999999999e-05 2024-09-18T00:19:15.791 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.791 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.791 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.791 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:25269 3.f e72/32 ondisk, result = 0)", 2024-09-18T00:19:15.791 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:15.791 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.07826369, 2024-09-18T00:19:15.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062014761000000002, 2024-09-18T00:19:15.791 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:15.792 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:15.792 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:15.792 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:15.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:15.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:15.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:15.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:15.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:15.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:15.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618114+0000", 2024-09-18T00:19:15.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1650000000000001e-06 2024-09-18T00:19:15.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:15.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618119+0000", 2024-09-18T00:19:15.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-18T00:19:15.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:15.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618134+0000", 2024-09-18T00:19:15.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5770999999999999e-05 2024-09-18T00:19:15.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:15.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618143+0000", 2024-09-18T00:19:15.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.551e-06 2024-09-18T00:19:15.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:15.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679967+0000", 2024-09-18T00:19:15.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061823937000000002 2024-09-18T00:19:15.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:15.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679975+0000", 2024-09-18T00:19:15.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7340000000000008e-06 2024-09-18T00:19:15.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:15.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:15.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:15.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680128+0000", 2024-09-18T00:19:15.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015338299999999999 2024-09-18T00:19:15.795 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.795 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.795 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.795 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:15.795 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:15.795 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:15.797 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:19:15.797 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:15.793+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:15.898 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:15.893+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:15.927 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:15.927 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:15.927 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:15.928 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:15.941 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:19:15.999 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:15.998+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:16.100 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:16.093+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:16.107 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:16.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:16.201 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:16.198+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:16.293 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:16.293 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:16.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2230 3.17 3:ef492888:::rbd_data.112aa7ba4578.000000000000014f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 56.024144839000002, 2024-09-18T00:19:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075065283999999996, 2024-09-18T00:19:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.294 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2230 2024-09-18T00:19:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266014+0000", 2024-09-18T00:19:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:19:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266029+0000", 2024-09-18T00:19:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4616e-05 2024-09-18T00:19:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266053+0000", 2024-09-18T00:19:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4369999999999999e-05 2024-09-18T00:19:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266067+0000", 2024-09-18T00:19:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4243999999999999e-05 2024-09-18T00:19:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266157+0000", 2024-09-18T00:19:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9909999999999998e-05 2024-09-18T00:19:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266327+0000", 2024-09-18T00:19:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016981200000000001 2024-09-18T00:19:16.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:19:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269180+0000", 2024-09-18T00:19:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028526559999999999 2024-09-18T00:19:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340955+0000", 2024-09-18T00:19:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.071774904 2024-09-18T00:19:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340974+0000", 2024-09-18T00:19:16.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8824000000000001e-05 2024-09-18T00:19:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341028+0000", 2024-09-18T00:19:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4456999999999997e-05 2024-09-18T00:19:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341076+0000", 2024-09-18T00:19:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8284000000000001e-05 2024-09-18T00:19:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2230 3.17 e36/23 ondisk, result = 0)", 2024-09-18T00:19:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 56.006801258000003, 2024-09-18T00:19:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057609645000000001, 2024-09-18T00:19:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283356+0000", 2024-09-18T00:19:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.44e-06 2024-09-18T00:19:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283362+0000", 2024-09-18T00:19:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2539999999999998e-06 2024-09-18T00:19:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283382+0000", 2024-09-18T00:19:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9896000000000001e-05 2024-09-18T00:19:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283398+0000", 2024-09-18T00:19:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5489999999999999e-05 2024-09-18T00:19:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340936+0000", 2024-09-18T00:19:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057538649999999997 2024-09-18T00:19:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340947+0000", 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0981e-05 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340964+0000", 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6934000000000001e-05 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2419 3.17 3:ed94b133:::rbd_data.112aa7ba4578.00000000000001b5:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.552679785000002, 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.064761812000000002, 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.303 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2419 2024-09-18T00:19:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737479+0000", 2024-09-18T00:19:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6089999999999999e-06 2024-09-18T00:19:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737487+0000", 2024-09-18T00:19:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7900000000000005e-06 2024-09-18T00:19:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737507+0000", 2024-09-18T00:19:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9936e-05 2024-09-18T00:19:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737518+0000", 2024-09-18T00:19:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1046999999999999e-05 2024-09-18T00:19:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737599+0000", 2024-09-18T00:19:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0697000000000003e-05 2024-09-18T00:19:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737697+0000", 2024-09-18T00:19:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8058000000000002e-05 2024-09-18T00:19:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:19:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.740445+0000", 2024-09-18T00:19:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027481279999999999 2024-09-18T00:19:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802023+0000", 2024-09-18T00:19:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.061577321999999997 2024-09-18T00:19:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802060+0000", 2024-09-18T00:19:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7586000000000002e-05 2024-09-18T00:19:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802158+0000", 2024-09-18T00:19:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7583000000000004e-05 2024-09-18T00:19:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802238+0000", 2024-09-18T00:19:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0055999999999994e-05 2024-09-18T00:19:16.308 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2419 3.17 e37/23 ondisk, result = 0)", 2024-09-18T00:19:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.540454838999999, 2024-09-18T00:19:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052342948, 2024-09-18T00:19:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749702+0000", 2024-09-18T00:19:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:19:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749710+0000", 2024-09-18T00:19:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7279999999999995e-06 2024-09-18T00:19:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749736+0000", 2024-09-18T00:19:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5797999999999999e-05 2024-09-18T00:19:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749751+0000", 2024-09-18T00:19:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5238999999999999e-05 2024-09-18T00:19:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801745+0000", 2024-09-18T00:19:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051994022000000001 2024-09-18T00:19:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801770+0000", 2024-09-18T00:19:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5103e-05 2024-09-18T00:19:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802044+0000", 2024-09-18T00:19:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027395799999999998 2024-09-18T00:19:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6503 3.1b 3:d9387a1f:::rbd_data.112a3a39fc45.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 50.725781869000002, 2024-09-18T00:19:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050510547000000003, 2024-09-18T00:19:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6503 2024-09-18T00:19:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564376+0000", 2024-09-18T00:19:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-06 2024-09-18T00:19:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564386+0000", 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0312999999999999e-05 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564408+0000", 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2001000000000002e-05 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564421+0000", 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2271e-05 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564491+0000", 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0630000000000006e-05 2024-09-18T00:19:16.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564606+0000", 2024-09-18T00:19:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011449 2024-09-18T00:19:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.566743+0000", 2024-09-18T00:19:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021373830000000001 2024-09-18T00:19:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614738+0000", 2024-09-18T00:19:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047995359000000001 2024-09-18T00:19:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614759+0000", 2024-09-18T00:19:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1010999999999999e-05 2024-09-18T00:19:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614826+0000", 2024-09-18T00:19:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6654000000000003e-05 2024-09-18T00:19:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614884+0000", 2024-09-18T00:19:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8332999999999999e-05 2024-09-18T00:19:16.317 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6503 3.1b e42/23 ondisk, result = 0)", 2024-09-18T00:19:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:19:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 50.716522400999999, 2024-09-18T00:19:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041115407, 2024-09-18T00:19:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:19:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:19:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573635+0000", 2024-09-18T00:19:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5689999999999999e-06 2024-09-18T00:19:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573643+0000", 2024-09-18T00:19:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9699999999999999e-06 2024-09-18T00:19:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573668+0000", 2024-09-18T00:19:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5536000000000001e-05 2024-09-18T00:19:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573683+0000", 2024-09-18T00:19:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4025e-05 2024-09-18T00:19:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614717+0000", 2024-09-18T00:19:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041034517 2024-09-18T00:19:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614730+0000", 2024-09-18T00:19:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2945999999999999e-05 2024-09-18T00:19:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614749+0000", 2024-09-18T00:19:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8844000000000001e-05 2024-09-18T00:19:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.322 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6518 3.1 3:8061bf9f:::rbd_data.112a3a39fc45.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:16.322 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:16.322 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 50.639343595, 2024-09-18T00:19:16.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062303123000000002, 2024-09-18T00:19:16.322 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.322 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.322 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:16.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.322 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6518 2024-09-18T00:19:16.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.322 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:16.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:16.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650814+0000", 2024-09-18T00:19:16.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.629e-06 2024-09-18T00:19:16.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650823+0000", 2024-09-18T00:19:16.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1800000000000002e-06 2024-09-18T00:19:16.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650843+0000", 2024-09-18T00:19:16.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9836000000000001e-05 2024-09-18T00:19:16.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650854+0000", 2024-09-18T00:19:16.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1564e-05 2024-09-18T00:19:16.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650917+0000", 2024-09-18T00:19:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3076000000000004e-05 2024-09-18T00:19:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.651029+0000", 2024-09-18T00:19:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111154 2024-09-18T00:19:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.655531+0000", 2024-09-18T00:19:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045027629999999999 2024-09-18T00:19:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712924+0000", 2024-09-18T00:19:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057393006000000003 2024-09-18T00:19:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712988+0000", 2024-09-18T00:19:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000005e-05 2024-09-18T00:19:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713056+0000", 2024-09-18T00:19:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7904999999999994e-05 2024-09-18T00:19:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713115+0000", 2024-09-18T00:19:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9339999999999998e-05 2024-09-18T00:19:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6518 3.1 e42/23 ondisk, result = 0)", 2024-09-18T00:19:16.327 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 50.626918512000003, 2024-09-18T00:19:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049887304, 2024-09-18T00:19:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:16.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663239+0000", 2024-09-18T00:19:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:19:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663246+0000", 2024-09-18T00:19:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4189999999999996e-06 2024-09-18T00:19:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663265+0000", 2024-09-18T00:19:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8323000000000001e-05 2024-09-18T00:19:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663277+0000", 2024-09-18T00:19:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2065999999999999e-05 2024-09-18T00:19:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712965+0000", 2024-09-18T00:19:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049688317000000003 2024-09-18T00:19:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712977+0000", 2024-09-18T00:19:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.206e-05 2024-09-18T00:19:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713125+0000", 2024-09-18T00:19:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014749899999999999 2024-09-18T00:19:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:8831 3.12 3:49cac8cf:::rbd_data.112a9f4259df.0000000000000130:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T00:19:16.331 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 44.009058263, 2024-09-18T00:19:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104125047, 2024-09-18T00:19:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 8831 2024-09-18T00:19:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:16.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281099+0000", 2024-09-18T00:19:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.029e-06 2024-09-18T00:19:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281103+0000", 2024-09-18T00:19:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7759999999999997e-06 2024-09-18T00:19:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281114+0000", 2024-09-18T00:19:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1090000000000001e-05 2024-09-18T00:19:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281120+0000", 2024-09-18T00:19:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.604e-06 2024-09-18T00:19:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281164+0000", 2024-09-18T00:19:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3696999999999999e-05 2024-09-18T00:19:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281270+0000", 2024-09-18T00:19:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106041 2024-09-18T00:19:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.285596+0000", 2024-09-18T00:19:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043266659999999998 2024-09-18T00:19:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385028+0000", 2024-09-18T00:19:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.099431808999999996 2024-09-18T00:19:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385094+0000", 2024-09-18T00:19:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5740000000000004e-05 2024-09-18T00:19:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385166+0000", 2024-09-18T00:19:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1637000000000005e-05 2024-09-18T00:19:16.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385223+0000", 2024-09-18T00:19:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6957999999999999e-05 2024-09-18T00:19:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:8831 3.12 e48/32 ondisk, result = 0)", 2024-09-18T00:19:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 43.992544152999997, 2024-09-18T00:19:16.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087621558000000002, 2024-09-18T00:19:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297613+0000", 2024-09-18T00:19:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3400000000000001e-06 2024-09-18T00:19:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297620+0000", 2024-09-18T00:19:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5869999999999998e-06 2024-09-18T00:19:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297639+0000", 2024-09-18T00:19:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9888999999999999e-05 2024-09-18T00:19:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297661+0000", 2024-09-18T00:19:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1772000000000001e-05 2024-09-18T00:19:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385073+0000", 2024-09-18T00:19:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087411303999999995 2024-09-18T00:19:16.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385086+0000", 2024-09-18T00:19:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3040000000000001e-05 2024-09-18T00:19:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385233+0000", 2024-09-18T00:19:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014762600000000001 2024-09-18T00:19:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22269 3.17 3:e9a1eb35:::rbd_data.112a948d5956.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.381012327000001, 2024-09-18T00:19:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.063462693000000001, 2024-09-18T00:19:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22269 2024-09-18T00:19:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909146+0000", 2024-09-18T00:19:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3700000000000002e-06 2024-09-18T00:19:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909160+0000", 2024-09-18T00:19:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3825000000000001e-05 2024-09-18T00:19:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909178+0000", 2024-09-18T00:19:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8567999999999999e-05 2024-09-18T00:19:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909189+0000", 2024-09-18T00:19:16.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0881e-05 2024-09-18T00:19:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909270+0000", 2024-09-18T00:19:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0846999999999994e-05 2024-09-18T00:19:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909383+0000", 2024-09-18T00:19:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112704 2024-09-18T00:19:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972310+0000", 2024-09-18T00:19:16.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062927081999999995 2024-09-18T00:19:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972433+0000", 2024-09-18T00:19:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-18T00:19:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972606+0000", 2024-09-18T00:19:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017317799999999999 2024-09-18T00:19:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22379 3.17 3:e9a1eb35:::rbd_data.112a948d5956.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.886937029999999, 2024-09-18T00:19:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.12560279099999999, 2024-09-18T00:19:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22379 2024-09-18T00:19:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.347 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403221+0000", 2024-09-18T00:19:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3269999999999999e-06 2024-09-18T00:19:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403232+0000", 2024-09-18T00:19:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1392e-05 2024-09-18T00:19:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403264+0000", 2024-09-18T00:19:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1146999999999998e-05 2024-09-18T00:19:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403279+0000", 2024-09-18T00:19:16.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5109000000000001e-05 2024-09-18T00:19:16.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403424+0000", 2024-09-18T00:19:16.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014516999999999999 2024-09-18T00:19:16.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403567+0000", 2024-09-18T00:19:16.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142824 2024-09-18T00:19:16.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:16.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528723+0000", 2024-09-18T00:19:16.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.12515647299999999 2024-09-18T00:19:16.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:16.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528764+0000", 2024-09-18T00:19:16.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0624000000000003e-05 2024-09-18T00:19:16.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528822+0000", 2024-09-18T00:19:16.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7725e-05 2024-09-18T00:19:16.351 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.352 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.352 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:27243 3.1b 3:df1bd230:::rbd_data.112aa34d8ec3.0000000000000029:16 [read 16384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:16.352 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:11.971339+0000", 2024-09-18T00:19:16.352 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.318816591, 2024-09-18T00:19:16.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.039816139, 2024-09-18T00:19:16.352 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.352 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.352 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.352 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.352 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.352 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.353 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27243 2024-09-18T00:19:16.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.353 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971339+0000", 2024-09-18T00:19:16.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971339+0000", 2024-09-18T00:19:16.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971340+0000", 2024-09-18T00:19:16.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2600000000000002e-07 2024-09-18T00:19:16.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971343+0000", 2024-09-18T00:19:16.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.994e-06 2024-09-18T00:19:16.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971351+0000", 2024-09-18T00:19:16.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8769999999999993e-06 2024-09-18T00:19:16.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971357+0000", 2024-09-18T00:19:16.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1040000000000003e-06 2024-09-18T00:19:16.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.006500+0000", 2024-09-18T00:19:16.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035143488000000001 2024-09-18T00:19:16.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.006572+0000", 2024-09-18T00:19:16.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2268999999999995e-05 2024-09-18T00:19:16.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.011155+0000", 2024-09-18T00:19:16.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045828809999999996 2024-09-18T00:19:16.356 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.356 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.356 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.356 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:27293 3.1b 3:df1bd230:::rbd_data.112aa34d8ec3.0000000000000029:16 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:16.356 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:11.971960+0000", 2024-09-18T00:19:16.356 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.3181953310000001, 2024-09-18T00:19:16.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.03820109, 2024-09-18T00:19:16.356 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.356 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.356 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.356 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.357 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.357 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.357 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27293 2024-09-18T00:19:16.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.357 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971960+0000", 2024-09-18T00:19:16.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971960+0000", 2024-09-18T00:19:16.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971961+0000", 2024-09-18T00:19:16.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-18T00:19:16.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971972+0000", 2024-09-18T00:19:16.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.079e-05 2024-09-18T00:19:16.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971980+0000", 2024-09-18T00:19:16.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5059999999999995e-06 2024-09-18T00:19:16.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971987+0000", 2024-09-18T00:19:16.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2609999999999999e-06 2024-09-18T00:19:16.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.009999+0000", 2024-09-18T00:19:16.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038012137000000001 2024-09-18T00:19:16.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010078+0000", 2024-09-18T00:19:16.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9580999999999996e-05 2024-09-18T00:19:16.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010162+0000", 2024-09-18T00:19:16.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3131000000000001e-05 2024-09-18T00:19:16.360 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.360 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.360 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.360 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:27295 3.1b 3:df1bd230:::rbd_data.112aa34d8ec3.0000000000000029:16 [read 75776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:16.360 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:11.971996+0000", 2024-09-18T00:19:16.360 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.3181599840000002, 2024-09-18T00:19:16.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038329568000000001, 2024-09-18T00:19:16.360 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.361 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.361 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.361 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.361 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.361 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.361 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27295 2024-09-18T00:19:16.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.361 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971996+0000", 2024-09-18T00:19:16.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971996+0000", 2024-09-18T00:19:16.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971997+0000", 2024-09-18T00:19:16.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2399999999999997e-07 2024-09-18T00:19:16.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971999+0000", 2024-09-18T00:19:16.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7760000000000002e-06 2024-09-18T00:19:16.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972007+0000", 2024-09-18T00:19:16.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7449999999999995e-06 2024-09-18T00:19:16.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972013+0000", 2024-09-18T00:19:16.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7869999999999998e-06 2024-09-18T00:19:16.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010186+0000", 2024-09-18T00:19:16.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038172999999999999 2024-09-18T00:19:16.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010259+0000", 2024-09-18T00:19:16.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2725999999999995e-05 2024-09-18T00:19:16.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010325+0000", 2024-09-18T00:19:16.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6810000000000006e-05 2024-09-18T00:19:16.364 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.364 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.364 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.365 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:27297 3.1b 3:df1bd230:::rbd_data.112aa34d8ec3.0000000000000029:16 [read 104448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:16.365 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:11.972021+0000", 2024-09-18T00:19:16.365 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.3181346200000004, 2024-09-18T00:19:16.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038465829, 2024-09-18T00:19:16.365 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.365 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.365 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.365 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27297 2024-09-18T00:19:16.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.366 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972021+0000", 2024-09-18T00:19:16.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972021+0000", 2024-09-18T00:19:16.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972022+0000", 2024-09-18T00:19:16.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1399999999999997e-07 2024-09-18T00:19:16.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972025+0000", 2024-09-18T00:19:16.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.717e-06 2024-09-18T00:19:16.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972032+0000", 2024-09-18T00:19:16.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6480000000000003e-06 2024-09-18T00:19:16.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972038+0000", 2024-09-18T00:19:16.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9549999999999997e-06 2024-09-18T00:19:16.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010349+0000", 2024-09-18T00:19:16.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038311031000000002 2024-09-18T00:19:16.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010421+0000", 2024-09-18T00:19:16.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1365999999999994e-05 2024-09-18T00:19:16.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010487+0000", 2024-09-18T00:19:16.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6497999999999999e-05 2024-09-18T00:19:16.368 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.368 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.369 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.369 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:27298 3.1b 3:df1bd230:::rbd_data.112aa34d8ec3.0000000000000029:16 [read 153600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:16.369 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:11.972047+0000", 2024-09-18T00:19:16.369 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.3181092549999995, 2024-09-18T00:19:16.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038602364, 2024-09-18T00:19:16.369 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.369 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.369 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.369 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.369 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.369 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.369 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27298 2024-09-18T00:19:16.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.370 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972047+0000", 2024-09-18T00:19:16.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972047+0000", 2024-09-18T00:19:16.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972047+0000", 2024-09-18T00:19:16.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2e-07 2024-09-18T00:19:16.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972050+0000", 2024-09-18T00:19:16.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0699999999999998e-06 2024-09-18T00:19:16.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972058+0000", 2024-09-18T00:19:16.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7379999999999994e-06 2024-09-18T00:19:16.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972071+0000", 2024-09-18T00:19:16.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3196999999999999e-05 2024-09-18T00:19:16.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010510+0000", 2024-09-18T00:19:16.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038439373999999998 2024-09-18T00:19:16.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010581+0000", 2024-09-18T00:19:16.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0773999999999997e-05 2024-09-18T00:19:16.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010649+0000", 2024-09-18T00:19:16.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7691e-05 2024-09-18T00:19:16.372 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.372 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.373 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.373 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:27300 3.1b 3:df1bd230:::rbd_data.112aa34d8ec3.0000000000000029:16 [read 182272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:16.373 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:11.972080+0000", 2024-09-18T00:19:16.373 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.3180755629999998, 2024-09-18T00:19:16.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038744935000000001, 2024-09-18T00:19:16.373 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.373 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.373 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.373 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.373 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.373 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.373 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27300 2024-09-18T00:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972080+0000", 2024-09-18T00:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972080+0000", 2024-09-18T00:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972081+0000", 2024-09-18T00:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5700000000000002e-07 2024-09-18T00:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972089+0000", 2024-09-18T00:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4570000000000005e-06 2024-09-18T00:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972098+0000", 2024-09-18T00:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7320000000000004e-06 2024-09-18T00:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972104+0000", 2024-09-18T00:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.207e-06 2024-09-18T00:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010672+0000", 2024-09-18T00:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038568189000000003 2024-09-18T00:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010743+0000", 2024-09-18T00:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0535000000000001e-05 2024-09-18T00:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010825+0000", 2024-09-18T00:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2257999999999997e-05 2024-09-18T00:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:27302 3.1b 3:df1bd230:::rbd_data.112aa34d8ec3.0000000000000029:16 [read 210944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:11.972113+0000", 2024-09-18T00:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.3180425830000004, 2024-09-18T00:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038872974999999997, 2024-09-18T00:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.377 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27302 2024-09-18T00:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972113+0000", 2024-09-18T00:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972113+0000", 2024-09-18T00:19:16.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972114+0000", 2024-09-18T00:19:16.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-07 2024-09-18T00:19:16.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972116+0000", 2024-09-18T00:19:16.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.633e-06 2024-09-18T00:19:16.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972124+0000", 2024-09-18T00:19:16.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.374e-06 2024-09-18T00:19:16.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972130+0000", 2024-09-18T00:19:16.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7799999999999997e-06 2024-09-18T00:19:16.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010848+0000", 2024-09-18T00:19:16.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038718283999999999 2024-09-18T00:19:16.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010917+0000", 2024-09-18T00:19:16.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9622000000000005e-05 2024-09-18T00:19:16.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010986+0000", 2024-09-18T00:19:16.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8707000000000005e-05 2024-09-18T00:19:16.381 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.381 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.381 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.381 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:27304 3.1b 3:df1bd230:::rbd_data.112aa34d8ec3.0000000000000029:16 [read 239616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:16.381 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:11.972138+0000", 2024-09-18T00:19:16.381 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.3180180080000001, 2024-09-18T00:19:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.039007344999999999, 2024-09-18T00:19:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27304 2024-09-18T00:19:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972138+0000", 2024-09-18T00:19:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972138+0000", 2024-09-18T00:19:16.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972138+0000", 2024-09-18T00:19:16.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2900000000000004e-07 2024-09-18T00:19:16.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972141+0000", 2024-09-18T00:19:16.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.74e-06 2024-09-18T00:19:16.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972149+0000", 2024-09-18T00:19:16.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7270000000000007e-06 2024-09-18T00:19:16.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972155+0000", 2024-09-18T00:19:16.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9390000000000002e-06 2024-09-18T00:19:16.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.011010+0000", 2024-09-18T00:19:16.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038855033999999997 2024-09-18T00:19:16.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.011080+0000", 2024-09-18T00:19:16.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0457000000000007e-05 2024-09-18T00:19:16.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.011145+0000", 2024-09-18T00:19:16.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4919000000000006e-05 2024-09-18T00:19:16.385 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.385 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.385 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.386 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.386 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.386 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:16.386 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is set 2024-09-18T00:19:16.387 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:19:16.388 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:16.386+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:16.488 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:16.481+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:16.501 INFO:teuthology.orchestra.run.smithi097.stdout:4825 write 0x52f3e6d thru 0x53010a6 (0xd23a bytes) 2024-09-18T00:19:16.533 INFO:teuthology.orchestra.run.smithi097.stdout:4826 read 0x5e9f13a thru 0x5ea7dfe (0x8cc5 bytes) 2024-09-18T00:19:16.533 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:16.533 INFO:teuthology.orchestra.run.smithi097.stdout:4828 read 0x9ee90b9 thru 0x9eef662 (0x65aa bytes) 2024-09-18T00:19:16.533 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:16.533 INFO:teuthology.orchestra.run.smithi097.stdout:4830 read 0x1291d5e thru 0x1296aca (0x4d6d bytes) 2024-09-18T00:19:16.535 INFO:teuthology.orchestra.run.smithi097.stdout:4832 read 0x280275b thru 0x280f307 (0xcbad bytes) 2024-09-18T00:19:16.535 INFO:teuthology.orchestra.run.smithi097.stdout:4833 punch from 0x5ae189b to 0x5ae2118, (0x87d bytes) 2024-09-18T00:19:16.535 INFO:teuthology.orchestra.run.smithi097.stdout:4835 write 0x4a141a8 thru 0x4a15645 (0x149e bytes) 2024-09-18T00:19:16.546 INFO:teuthology.orchestra.run.smithi097.stdout:4836 writesame 0x7812d69 thru 0x781df20 data_size 0x79(0xb1b8 bytes) 2024-09-18T00:19:16.577 INFO:teuthology.orchestra.run.smithi097.stdout:4837 punch from 0x1057993 to 0x105eedb, (0x7548 bytes) 2024-09-18T00:19:16.577 INFO:teuthology.orchestra.run.smithi097.stdout:4838 read 0x60949dd thru 0x6094bb7 (0x1db bytes) 2024-09-18T00:19:16.578 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:16.578 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:16.578 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:16.578 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:16.586 INFO:teuthology.orchestra.run.smithi097.stdout:4840 trunc from 0x9faf091 to 0x58bc897 2024-09-18T00:19:16.589 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:16.586+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:16.601 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:19:16.616 INFO:teuthology.orchestra.run.smithi097.stdout:4841 trunc from 0x58bc897 to 0x48a150f 2024-09-18T00:19:16.638 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:16.658 INFO:teuthology.orchestra.run.smithi097.stdout:4843 write 0xceb3270 thru 0xceb63df (0x3170 bytes) 2024-09-18T00:19:16.670 INFO:teuthology.orchestra.run.smithi097.stdout:4844 punch from 0x2c9805e to 0x2ca0e1b, (0x8dbd bytes) 2024-09-18T00:19:16.674 INFO:teuthology.orchestra.run.smithi097.stdout:4845 trunc from 0xceb63e0 to 0xabed4bb 2024-09-18T00:19:16.690 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:16.685+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:16.692 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:16.692 INFO:teuthology.orchestra.run.smithi097.stdout:4848 read 0x873ec24 thru 0x874aff5 (0xc3d2 bytes) 2024-09-18T00:19:16.692 INFO:teuthology.orchestra.run.smithi097.stdout:4850 read 0x26e4c76 thru 0x26e6b93 (0x1f1e bytes) 2024-09-18T00:19:16.692 INFO:teuthology.orchestra.run.smithi097.stdout:4853 write 0x844b497 thru 0x845701d (0xbb87 bytes) 2024-09-18T00:19:16.709 INFO:teuthology.orchestra.run.smithi097.stdout:4854 trunc from 0xabed4bb to 0xa96bfe2 2024-09-18T00:19:16.717 INFO:teuthology.orchestra.run.smithi097.stdout:4855 punch from 0x57ec60e to 0x57edd45, (0x1737 bytes) 2024-09-18T00:19:16.717 INFO:teuthology.orchestra.run.smithi097.stdout:4856 read 0x2f7fac3 thru 0x2f7fe85 (0x3c3 bytes) 2024-09-18T00:19:16.717 INFO:teuthology.orchestra.run.smithi097.stdout:4858 write 0x5081084 thru 0x508b928 (0xa8a5 bytes) 2024-09-18T00:19:16.736 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:16.745 INFO:teuthology.orchestra.run.smithi097.stdout:4860 write 0xbaa9eba thru 0xbab4223 (0xa36a bytes) 2024-09-18T00:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:16.763 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:16.766 INFO:teuthology.orchestra.run.smithi097.stdout:4864 clone 21 order 19 su 4096 sc 5 2024-09-18T00:19:16.778 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:19:16.791 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:16.789+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:16.893 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:16.889+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:16.939 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:16.939 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:16.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:16.940 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:16.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.940 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:5722 3.4 e41/23 3:20381d86:::rbd_data.112a3a39fc45.000000000000001d:head v 41'107, mlcod=40'105)", 2024-09-18T00:19:16.940 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:16.940 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.602932482, 2024-09-18T00:19:16.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01185172, 2024-09-18T00:19:16.940 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.940 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.940 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:16.940 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:16.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:16.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335497+0000", 2024-09-18T00:19:16.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.505e-06 2024-09-18T00:19:16.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335945+0000", 2024-09-18T00:19:16.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000447687 2024-09-18T00:19:16.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335961+0000", 2024-09-18T00:19:16.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6430999999999999e-05 2024-09-18T00:19:16.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335969+0000", 2024-09-18T00:19:16.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:19:16.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336030+0000", 2024-09-18T00:19:16.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1255999999999998e-05 2024-09-18T00:19:16.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336061+0000", 2024-09-18T00:19:16.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0579e-05 2024-09-18T00:19:16.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:16.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347296+0000", 2024-09-18T00:19:16.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011235663 2024-09-18T00:19:16.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347347+0000", 2024-09-18T00:19:16.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0914000000000001e-05 2024-09-18T00:19:16.944 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.944 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.944 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.945 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11457 3.0 3:0214e7b3:::rbd_data.112ab1fa102a.0000000000000150:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:16.945 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:16.945 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.678566543999999, 2024-09-18T00:19:16.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028499165, 2024-09-18T00:19:16.945 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.945 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.945 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:16.945 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.945 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.945 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.945 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11457 2024-09-18T00:19:16.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.946 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:16.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:16.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259865+0000", 2024-09-18T00:19:16.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7230000000000002e-06 2024-09-18T00:19:16.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259883+0000", 2024-09-18T00:19:16.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8067000000000002e-05 2024-09-18T00:19:16.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259924+0000", 2024-09-18T00:19:16.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0550999999999999e-05 2024-09-18T00:19:16.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259943+0000", 2024-09-18T00:19:16.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:19:16.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260054+0000", 2024-09-18T00:19:16.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011075 2024-09-18T00:19:16.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260138+0000", 2024-09-18T00:19:16.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4549000000000007e-05 2024-09-18T00:19:16.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:16.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.287926+0000", 2024-09-18T00:19:16.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027787642000000001 2024-09-18T00:19:16.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:16.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288090+0000", 2024-09-18T00:19:16.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016402499999999999 2024-09-18T00:19:16.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288361+0000", 2024-09-18T00:19:16.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027097800000000001 2024-09-18T00:19:16.949 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.949 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.949 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.950 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18661 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:16.950 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:16.950 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.003798678999999, 2024-09-18T00:19:16.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020008037999999999, 2024-09-18T00:19:16.950 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.950 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.950 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:16.950 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.950 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.950 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.950 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18661 2024-09-18T00:19:16.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.951 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:16.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:16.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934633+0000", 2024-09-18T00:19:16.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8e-06 2024-09-18T00:19:16.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935511+0000", 2024-09-18T00:19:16.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087785700000000001 2024-09-18T00:19:16.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935561+0000", 2024-09-18T00:19:16.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9472000000000003e-05 2024-09-18T00:19:16.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935586+0000", 2024-09-18T00:19:16.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5616e-05 2024-09-18T00:19:16.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935838+0000", 2024-09-18T00:19:16.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025202699999999999 2024-09-18T00:19:16.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.936023+0000", 2024-09-18T00:19:16.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018480300000000001 2024-09-18T00:19:16.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:16.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953545+0000", 2024-09-18T00:19:16.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017522157 2024-09-18T00:19:16.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:16.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953723+0000", 2024-09-18T00:19:16.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017753800000000001 2024-09-18T00:19:16.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954637+0000", 2024-09-18T00:19:16.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091476800000000005 2024-09-18T00:19:16.955 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.955 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.955 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.955 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18662 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:16.955 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:16.955 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.002793742000001, 2024-09-18T00:19:16.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019388843999999999, 2024-09-18T00:19:16.955 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.955 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.956 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:16.956 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.956 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.956 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.956 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18662 2024-09-18T00:19:16.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.956 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.956 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.956 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:16.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:16.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935636+0000", 2024-09-18T00:19:16.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.663e-06 2024-09-18T00:19:16.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935646+0000", 2024-09-18T00:19:16.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000006e-06 2024-09-18T00:19:16.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935672+0000", 2024-09-18T00:19:16.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5868999999999999e-05 2024-09-18T00:19:16.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935692+0000", 2024-09-18T00:19:16.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0604000000000001e-05 2024-09-18T00:19:16.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950738+0000", 2024-09-18T00:19:16.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015045868 2024-09-18T00:19:16.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950992+0000", 2024-09-18T00:19:16.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025368400000000001 2024-09-18T00:19:16.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:16.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954674+0000", 2024-09-18T00:19:16.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003682154 2024-09-18T00:19:16.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:16.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954806+0000", 2024-09-18T00:19:16.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013252000000000001 2024-09-18T00:19:16.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.955023+0000", 2024-09-18T00:19:16.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021681200000000001 2024-09-18T00:19:16.960 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.960 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.960 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.960 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19382 3.4 3:22f09d3a:::rbd_data.112a7fe29ab4.000000000000015a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:16.960 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:16.960 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.316383691999999, 2024-09-18T00:19:16.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075119302999999998, 2024-09-18T00:19:16.960 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.961 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.961 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:16.961 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.961 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.961 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.961 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19382 2024-09-18T00:19:16.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.961 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:16.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:16.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622045+0000", 2024-09-18T00:19:16.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0020000000000001e-06 2024-09-18T00:19:16.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.962 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622053+0000", 2024-09-18T00:19:16.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2679999999999997e-06 2024-09-18T00:19:16.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.962 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622062+0000", 2024-09-18T00:19:16.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3749999999999992e-06 2024-09-18T00:19:16.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622068+0000", 2024-09-18T00:19:16.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5729999999999998e-06 2024-09-18T00:19:16.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622118+0000", 2024-09-18T00:19:16.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0696999999999999e-05 2024-09-18T00:19:16.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622222+0000", 2024-09-18T00:19:16.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103286 2024-09-18T00:19:16.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:16.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697061+0000", 2024-09-18T00:19:16.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074839141999999997 2024-09-18T00:19:16.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:16.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697102+0000", 2024-09-18T00:19:16.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1513e-05 2024-09-18T00:19:16.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697164+0000", 2024-09-18T00:19:16.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1446999999999996e-05 2024-09-18T00:19:16.965 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.965 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.965 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.965 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19896 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:16.965 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:19:16.965 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.996999553999999, 2024-09-18T00:19:16.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097478370000000005, 2024-09-18T00:19:16.965 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.965 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.966 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.966 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.966 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.966 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.966 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19896 2024-09-18T00:19:16.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.966 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:19:16.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:19:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941429+0000", 2024-09-18T00:19:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6000000000000002e-07 2024-09-18T00:19:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941434+0000", 2024-09-18T00:19:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4750000000000004e-06 2024-09-18T00:19:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941452+0000", 2024-09-18T00:19:16.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8603000000000001e-05 2024-09-18T00:19:16.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941461+0000", 2024-09-18T00:19:16.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5199999999999997e-06 2024-09-18T00:19:16.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.947391+0000", 2024-09-18T00:19:16.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0059302399999999998 2024-09-18T00:19:16.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.947479+0000", 2024-09-18T00:19:16.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8002000000000002e-05 2024-09-18T00:19:16.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.951176+0000", 2024-09-18T00:19:16.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036971370000000001 2024-09-18T00:19:16.969 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.969 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.969 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.969 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19910 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:16.969 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.941878+0000", 2024-09-18T00:19:16.970 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.996549824999999, 2024-09-18T00:19:16.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0092861479999999993, 2024-09-18T00:19:16.970 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.970 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.970 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.970 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.970 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.970 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.970 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19910 2024-09-18T00:19:16.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.970 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941878+0000", 2024-09-18T00:19:16.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941878+0000", 2024-09-18T00:19:16.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941879+0000", 2024-09-18T00:19:16.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6199999999999998e-07 2024-09-18T00:19:16.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941881+0000", 2024-09-18T00:19:16.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6390000000000001e-06 2024-09-18T00:19:16.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941890+0000", 2024-09-18T00:19:16.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4999999999999999e-06 2024-09-18T00:19:16.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.941896+0000", 2024-09-18T00:19:16.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1299999999999998e-06 2024-09-18T00:19:16.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950898+0000", 2024-09-18T00:19:16.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0090014499999999994 2024-09-18T00:19:16.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.950984+0000", 2024-09-18T00:19:16.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6933000000000003e-05 2024-09-18T00:19:16.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.951164+0000", 2024-09-18T00:19:16.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000179934 2024-09-18T00:19:16.973 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.973 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.973 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.974 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20242 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:16.974 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155613+0000", 2024-09-18T00:19:16.974 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.782815034999999, 2024-09-18T00:19:16.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00933551, 2024-09-18T00:19:16.974 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.974 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.974 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.974 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.974 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.974 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.974 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20242 2024-09-18T00:19:16.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.974 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155613+0000", 2024-09-18T00:19:16.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155613+0000", 2024-09-18T00:19:16.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.975 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.975 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155613+0000", 2024-09-18T00:19:16.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1e-07 2024-09-18T00:19:16.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155615+0000", 2024-09-18T00:19:16.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1110000000000002e-06 2024-09-18T00:19:16.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155621+0000", 2024-09-18T00:19:16.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8760000000000002e-06 2024-09-18T00:19:16.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155628+0000", 2024-09-18T00:19:16.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7259999999999996e-06 2024-09-18T00:19:16.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164725+0000", 2024-09-18T00:19:16.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0090965009999999999 2024-09-18T00:19:16.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164832+0000", 2024-09-18T00:19:16.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106956 2024-09-18T00:19:16.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164949+0000", 2024-09-18T00:19:16.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011703 2024-09-18T00:19:16.978 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.978 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.978 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.978 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20243 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:16.978 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155635+0000", 2024-09-18T00:19:16.978 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.782792992000001, 2024-09-18T00:19:16.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.009569358, 2024-09-18T00:19:16.978 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.978 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.978 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.978 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.979 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.979 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.979 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20243 2024-09-18T00:19:16.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.979 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155635+0000", 2024-09-18T00:19:16.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155635+0000", 2024-09-18T00:19:16.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155635+0000", 2024-09-18T00:19:16.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8000000000000001e-07 2024-09-18T00:19:16.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155639+0000", 2024-09-18T00:19:16.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0809999999999998e-06 2024-09-18T00:19:16.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.980 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155645+0000", 2024-09-18T00:19:16.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0469999999999999e-06 2024-09-18T00:19:16.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155649+0000", 2024-09-18T00:19:16.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7419999999999997e-06 2024-09-18T00:19:16.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.164990+0000", 2024-09-18T00:19:16.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093401370000000001 2024-09-18T00:19:16.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165066+0000", 2024-09-18T00:19:16.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6861999999999997e-05 2024-09-18T00:19:16.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165204+0000", 2024-09-18T00:19:16.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138109 2024-09-18T00:19:16.982 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.982 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.982 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.982 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20244 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:16.982 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155679+0000", 2024-09-18T00:19:16.982 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.782749035999998, 2024-09-18T00:19:16.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097580089999999998, 2024-09-18T00:19:16.982 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.983 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.983 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.983 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.983 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.983 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.983 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20244 2024-09-18T00:19:16.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.983 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155679+0000", 2024-09-18T00:19:16.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155679+0000", 2024-09-18T00:19:16.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155680+0000", 2024-09-18T00:19:16.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0900000000000002e-07 2024-09-18T00:19:16.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155682+0000", 2024-09-18T00:19:16.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6589999999999999e-06 2024-09-18T00:19:16.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155690+0000", 2024-09-18T00:19:16.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3139999999999998e-06 2024-09-18T00:19:16.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155695+0000", 2024-09-18T00:19:16.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0950000000000003e-06 2024-09-18T00:19:16.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165243+0000", 2024-09-18T00:19:16.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0095483519999999995 2024-09-18T00:19:16.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165321+0000", 2024-09-18T00:19:16.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8219000000000004e-05 2024-09-18T00:19:16.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165437+0000", 2024-09-18T00:19:16.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011586099999999999 2024-09-18T00:19:16.986 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.986 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.986 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20245 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.782725363000001, 2024-09-18T00:19:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099981159999999996, 2024-09-18T00:19:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20245 2024-09-18T00:19:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:16.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:16.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:16.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.65e-07 2024-09-18T00:19:16.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155711+0000", 2024-09-18T00:19:16.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:19:16.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155720+0000", 2024-09-18T00:19:16.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6580000000000008e-06 2024-09-18T00:19:16.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155725+0000", 2024-09-18T00:19:16.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3430000000000003e-06 2024-09-18T00:19:16.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165479+0000", 2024-09-18T00:19:16.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097538239999999995 2024-09-18T00:19:16.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165556+0000", 2024-09-18T00:19:16.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6724000000000006e-05 2024-09-18T00:19:16.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165701+0000", 2024-09-18T00:19:16.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145129 2024-09-18T00:19:16.990 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.991 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.991 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.991 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20246 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:16.991 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:16.991 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.782695499999999, 2024-09-18T00:19:16.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01018784, 2024-09-18T00:19:16.991 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.991 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.991 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.991 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.991 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.991 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.991 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20246 2024-09-18T00:19:16.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.992 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:16.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:16.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:16.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2599999999999998e-07 2024-09-18T00:19:16.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155735+0000", 2024-09-18T00:19:16.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2699999999999999e-06 2024-09-18T00:19:16.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155744+0000", 2024-09-18T00:19:16.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8629999999999997e-06 2024-09-18T00:19:16.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155749+0000", 2024-09-18T00:19:16.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0359999999999997e-06 2024-09-18T00:19:16.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165748+0000", 2024-09-18T00:19:16.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099991449999999992 2024-09-18T00:19:16.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165819+0000", 2024-09-18T00:19:16.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0964999999999996e-05 2024-09-18T00:19:16.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.995 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.995 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165920+0000", 2024-09-18T00:19:16.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101135 2024-09-18T00:19:16.995 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.995 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:16.995 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:16.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.995 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20247 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:16.996 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:16.996 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.782672158, 2024-09-18T00:19:16.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010443987, 2024-09-18T00:19:16.996 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:16.996 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:16.996 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:16.996 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:16.996 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:16.996 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:16.996 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20247 2024-09-18T00:19:16.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.996 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:16.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:16.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:16.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:16.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:16.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:16.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:16.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:16.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1899999999999998e-07 2024-09-18T00:19:16.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:16.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155760+0000", 2024-09-18T00:19:16.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6459999999999999e-06 2024-09-18T00:19:16.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:16.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155767+0000", 2024-09-18T00:19:16.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0350000000000004e-06 2024-09-18T00:19:16.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:16.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155774+0000", 2024-09-18T00:19:16.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:19:16.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:16.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165960+0000", 2024-09-18T00:19:16.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186051999999999 2024-09-18T00:19:16.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:16.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166048+0000", 2024-09-18T00:19:16.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8848e-05 2024-09-18T00:19:16.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:16.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:16.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:16.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166200+0000", 2024-09-18T00:19:16.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015147000000000001 2024-09-18T00:19:17.000 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.000 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.000 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.000 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20248 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:17.000 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:17.000 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.782646994, 2024-09-18T00:19:17.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010617721, 2024-09-18T00:19:17.000 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.000 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.000 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:17.000 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.000 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.000 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.001 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20248 2024-09-18T00:19:17.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.001 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:17.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:17.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155782+0000", 2024-09-18T00:19:17.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.84e-07 2024-09-18T00:19:17.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155784+0000", 2024-09-18T00:19:17.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2800000000000002e-06 2024-09-18T00:19:17.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155790+0000", 2024-09-18T00:19:17.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5679999999999996e-06 2024-09-18T00:19:17.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155795+0000", 2024-09-18T00:19:17.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6929999999999998e-06 2024-09-18T00:19:17.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166245+0000", 2024-09-18T00:19:17.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010449847999999999 2024-09-18T00:19:17.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166307+0000", 2024-09-18T00:19:17.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2287999999999997e-05 2024-09-18T00:19:17.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166399+0000", 2024-09-18T00:19:17.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.166e-05 2024-09-18T00:19:17.004 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.004 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.004 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.004 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20249 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:17.004 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:17.004 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.782626617999998, 2024-09-18T00:19:17.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010811983000000001, 2024-09-18T00:19:17.004 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.004 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.004 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:17.004 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.005 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20249 2024-09-18T00:19:17.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.005 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:17.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:17.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155802+0000", 2024-09-18T00:19:17.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:19:17.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155804+0000", 2024-09-18T00:19:17.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:19:17.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155810+0000", 2024-09-18T00:19:17.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0580000000000003e-06 2024-09-18T00:19:17.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155817+0000", 2024-09-18T00:19:17.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3980000000000004e-06 2024-09-18T00:19:17.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166451+0000", 2024-09-18T00:19:17.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010634825000000001 2024-09-18T00:19:17.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166520+0000", 2024-09-18T00:19:17.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8317000000000003e-05 2024-09-18T00:19:17.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166613+0000", 2024-09-18T00:19:17.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3801000000000005e-05 2024-09-18T00:19:17.008 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.008 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.008 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.009 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:21913 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:17.009 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:17.009 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.644781471999998, 2024-09-18T00:19:17.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090508916999999994, 2024-09-18T00:19:17.009 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.009 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.009 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.009 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.009 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.009 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.009 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 21913 2024-09-18T00:19:17.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.009 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:17.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:17.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293648+0000", 2024-09-18T00:19:17.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6400000000000001e-07 2024-09-18T00:19:17.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293653+0000", 2024-09-18T00:19:17.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3820000000000003e-06 2024-09-18T00:19:17.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293665+0000", 2024-09-18T00:19:17.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1778e-05 2024-09-18T00:19:17.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293673+0000", 2024-09-18T00:19:17.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4130000000000006e-06 2024-09-18T00:19:17.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293719+0000", 2024-09-18T00:19:17.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5534999999999997e-05 2024-09-18T00:19:17.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293773+0000", 2024-09-18T00:19:17.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4389999999999999e-05 2024-09-18T00:19:17.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384053+0000", 2024-09-18T00:19:17.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090279617000000006 2024-09-18T00:19:17.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384088+0000", 2024-09-18T00:19:17.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5466000000000002e-05 2024-09-18T00:19:17.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384156+0000", 2024-09-18T00:19:17.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7471999999999994e-05 2024-09-18T00:19:17.014 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.014 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.014 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.014 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22274 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:17.014 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:17.014 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.833665251999999, 2024-09-18T00:19:17.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.069820742000000005, 2024-09-18T00:19:17.014 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.014 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.014 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.014 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.015 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.015 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.015 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22274 2024-09-18T00:19:17.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.015 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:17.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:17.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104766+0000", 2024-09-18T00:19:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3859999999999999e-06 2024-09-18T00:19:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104783+0000", 2024-09-18T00:19:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6697e-05 2024-09-18T00:19:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104816+0000", 2024-09-18T00:19:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2786000000000001e-05 2024-09-18T00:19:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104833+0000", 2024-09-18T00:19:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7125999999999998e-05 2024-09-18T00:19:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104965+0000", 2024-09-18T00:19:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013215899999999999 2024-09-18T00:19:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.105103+0000", 2024-09-18T00:19:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138193 2024-09-18T00:19:17.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174496+0000", 2024-09-18T00:19:17.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.06939244 2024-09-18T00:19:17.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174530+0000", 2024-09-18T00:19:17.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4601999999999998e-05 2024-09-18T00:19:17.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174584+0000", 2024-09-18T00:19:17.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3353000000000002e-05 2024-09-18T00:19:17.019 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.019 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.019 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.019 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22485 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write 16521958~30847] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:17.019 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:17.019 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.984876241999999, 2024-09-18T00:19:17.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012574648000000001, 2024-09-18T00:19:17.019 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.019 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.019 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.019 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.019 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22485 2024-09-18T00:19:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953554+0000", 2024-09-18T00:19:17.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1150000000000001e-06 2024-09-18T00:19:17.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962036+0000", 2024-09-18T00:19:17.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0084817599999999996 2024-09-18T00:19:17.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962062+0000", 2024-09-18T00:19:17.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6499000000000002e-05 2024-09-18T00:19:17.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962074+0000", 2024-09-18T00:19:17.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2095e-05 2024-09-18T00:19:17.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962165+0000", 2024-09-18T00:19:17.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0703000000000002e-05 2024-09-18T00:19:17.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962276+0000", 2024-09-18T00:19:17.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111406 2024-09-18T00:19:17.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965925+0000", 2024-09-18T00:19:17.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036488919999999999 2024-09-18T00:19:17.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965992+0000", 2024-09-18T00:19:17.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6806999999999999e-05 2024-09-18T00:19:17.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.966127+0000", 2024-09-18T00:19:17.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013437099999999999 2024-09-18T00:19:17.024 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.024 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.024 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.024 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22681 3.4 3:22e76021:::rbd_data.112a948d5956.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:17.024 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:17.024 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.650876937, 2024-09-18T00:19:17.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076591189000000004, 2024-09-18T00:19:17.024 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.024 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.024 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.025 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.025 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.025 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.025 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22681 2024-09-18T00:19:17.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.025 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:17.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:17.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287554+0000", 2024-09-18T00:19:17.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.514e-06 2024-09-18T00:19:17.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287567+0000", 2024-09-18T00:19:17.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-18T00:19:17.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287597+0000", 2024-09-18T00:19:17.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0406e-05 2024-09-18T00:19:17.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287626+0000", 2024-09-18T00:19:17.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9306e-05 2024-09-18T00:19:17.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287801+0000", 2024-09-18T00:19:17.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017411400000000001 2024-09-18T00:19:17.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.288053+0000", 2024-09-18T00:19:17.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025223500000000001 2024-09-18T00:19:17.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364016+0000", 2024-09-18T00:19:17.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075963301999999996 2024-09-18T00:19:17.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364064+0000", 2024-09-18T00:19:17.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7886000000000002e-05 2024-09-18T00:19:17.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364142+0000", 2024-09-18T00:19:17.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8345999999999998e-05 2024-09-18T00:19:17.029 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.029 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.029 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.029 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:24700 3.4 3:23a8f5d4:::rbd_data.112a990280ef.0000000000000016:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:17.029 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:19:17.029 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 10.000410958, 2024-09-18T00:19:17.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091975649000000007, 2024-09-18T00:19:17.030 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.030 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.030 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.030 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.030 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.030 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.030 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 24700 2024-09-18T00:19:17.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.030 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:19:17.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:19:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938019+0000", 2024-09-18T00:19:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:19:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938030+0000", 2024-09-18T00:19:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0278000000000001e-05 2024-09-18T00:19:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938053+0000", 2024-09-18T00:19:17.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2946999999999999e-05 2024-09-18T00:19:17.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938062+0000", 2024-09-18T00:19:17.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8509999999999994e-06 2024-09-18T00:19:17.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938140+0000", 2024-09-18T00:19:17.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7664999999999996e-05 2024-09-18T00:19:17.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938314+0000", 2024-09-18T00:19:17.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017398500000000001 2024-09-18T00:19:17.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029896+0000", 2024-09-18T00:19:17.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091582078999999997 2024-09-18T00:19:17.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029930+0000", 2024-09-18T00:19:17.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3723999999999998e-05 2024-09-18T00:19:17.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029993+0000", 2024-09-18T00:19:17.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2901e-05 2024-09-18T00:19:17.034 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.034 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.034 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.034 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.034 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.034 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:17.035 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:19:17.036 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:17.029+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:17.136 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:17.129+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:17.174 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:19:17.204 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:17.205 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:17.205 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:17.205 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:17.218 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:19:17.237 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:17.233+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:17.306 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:17.306 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:17.306 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:17.307 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:17.338 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:17.333+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:17.395 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:17.395 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:17.395 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:17.396 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:17.396 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:17.411 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:19:17.453 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:17.449+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:17.498 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:19:17.539 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:17.537+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:17.578 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:17.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:19:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.775451763999996, 2024-09-18T00:19:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:19:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.579 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:19:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:19:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:19:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:19:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:19:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:19:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:19:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:19:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:19:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:19:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:19:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:19:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:19:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:19:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:19:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:19:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:19:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:19:17.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:19:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:19:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:19:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:19:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:19:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:19:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.759243890999997, 2024-09-18T00:19:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:19:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:19:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:19:17.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:19:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:19:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:19:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:19:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:19:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:19:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:19:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:19:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:19:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:19:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:19:17.588 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.732202571000002, 2024-09-18T00:19:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:19:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.589 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:19:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:19:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:19:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:19:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:19:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:19:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:19:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:19:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:19:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:19:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:19:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:19:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:19:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:19:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:19:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:19:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:19:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:19:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:19:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:19:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:19:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:19:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:19:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.594 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:19:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.710926018999999, 2024-09-18T00:19:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:19:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:17.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:19:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:19:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:19:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:19:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:19:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:19:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:19:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:19:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:19:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:19:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:19:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:19:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:19:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:19:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.598 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11460 3.19 3:9e578b6c:::rbd_data.112ab1fa102a.0000000000000153:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.308554567000002, 2024-09-18T00:19:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041989011999999999, 2024-09-18T00:19:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11460 2024-09-18T00:19:17.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267067+0000", 2024-09-18T00:19:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:19:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267088+0000", 2024-09-18T00:19:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0908999999999999e-05 2024-09-18T00:19:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267133+0000", 2024-09-18T00:19:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5232000000000003e-05 2024-09-18T00:19:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267157+0000", 2024-09-18T00:19:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3833e-05 2024-09-18T00:19:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267300+0000", 2024-09-18T00:19:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142282 2024-09-18T00:19:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267447+0000", 2024-09-18T00:19:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147851 2024-09-18T00:19:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308889+0000", 2024-09-18T00:19:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041441779999999998 2024-09-18T00:19:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308956+0000", 2024-09-18T00:19:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7136000000000002e-05 2024-09-18T00:19:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.309053+0000", 2024-09-18T00:19:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6552999999999995e-05 2024-09-18T00:19:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.603 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:16947 3.18 3:18b58c7b:::rbd_data.112a6efb7564.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:19:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.958776866000001, 2024-09-18T00:19:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.101692484, 2024-09-18T00:19:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.604 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 16947 2024-09-18T00:19:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616843+0000", 2024-09-18T00:19:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0780000000000001e-06 2024-09-18T00:19:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617785+0000", 2024-09-18T00:19:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094204100000000002 2024-09-18T00:19:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617801+0000", 2024-09-18T00:19:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6436999999999999e-05 2024-09-18T00:19:17.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617809+0000", 2024-09-18T00:19:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9049999999999997e-06 2024-09-18T00:19:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617904+0000", 2024-09-18T00:19:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5204999999999994e-05 2024-09-18T00:19:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.607 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.618071+0000", 2024-09-18T00:19:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016640000000000001 2024-09-18T00:19:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.624534+0000", 2024-09-18T00:19:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064634499999999999 2024-09-18T00:19:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718299+0000", 2024-09-18T00:19:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093764739 2024-09-18T00:19:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718375+0000", 2024-09-18T00:19:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6532999999999994e-05 2024-09-18T00:19:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718466+0000", 2024-09-18T00:19:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0581000000000006e-05 2024-09-18T00:19:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718534+0000", 2024-09-18T00:19:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8114999999999994e-05 2024-09-18T00:19:17.609 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:16947 3.18 e59/23 ondisk, result = 0)", 2024-09-18T00:19:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.933560172, 2024-09-18T00:19:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076488069000000006, 2024-09-18T00:19:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642060+0000", 2024-09-18T00:19:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6640000000000001e-06 2024-09-18T00:19:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642071+0000", 2024-09-18T00:19:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1204e-05 2024-09-18T00:19:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642091+0000", 2024-09-18T00:19:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0313999999999999e-05 2024-09-18T00:19:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642103+0000", 2024-09-18T00:19:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1214e-05 2024-09-18T00:19:17.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718350+0000", 2024-09-18T00:19:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076247681999999997 2024-09-18T00:19:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718366+0000", 2024-09-18T00:19:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5420000000000001e-05 2024-09-18T00:19:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718546+0000", 2024-09-18T00:19:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000180571 2024-09-18T00:19:17.613 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18686 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.626882802000001, 2024-09-18T00:19:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041132573999999998, 2024-09-18T00:19:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.614 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18686 2024-09-18T00:19:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948737+0000", 2024-09-18T00:19:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3769999999999999e-06 2024-09-18T00:19:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949968+0000", 2024-09-18T00:19:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001230493 2024-09-18T00:19:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949993+0000", 2024-09-18T00:19:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5282e-05 2024-09-18T00:19:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950011+0000", 2024-09-18T00:19:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8199999999999999e-05 2024-09-18T00:19:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950180+0000", 2024-09-18T00:19:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016867600000000001 2024-09-18T00:19:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950337+0000", 2024-09-18T00:19:17.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156902 2024-09-18T00:19:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950818+0000", 2024-09-18T00:19:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048192599999999998 2024-09-18T00:19:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978491+0000", 2024-09-18T00:19:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027672568000000002 2024-09-18T00:19:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989759+0000", 2024-09-18T00:19:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011268144000000001 2024-09-18T00:19:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989824+0000", 2024-09-18T00:19:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4556999999999999e-05 2024-09-18T00:19:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989868+0000", 2024-09-18T00:19:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4449000000000001e-05 2024-09-18T00:19:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.619 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18687 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.625555037000002, 2024-09-18T00:19:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040034156000000001, 2024-09-18T00:19:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.620 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18687 2024-09-18T00:19:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:17.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950065+0000", 2024-09-18T00:19:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5820000000000001e-06 2024-09-18T00:19:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950072+0000", 2024-09-18T00:19:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1029999999999996e-06 2024-09-18T00:19:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950092+0000", 2024-09-18T00:19:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0288e-05 2024-09-18T00:19:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950107+0000", 2024-09-18T00:19:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4290999999999999e-05 2024-09-18T00:19:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975820+0000", 2024-09-18T00:19:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025713573999999999 2024-09-18T00:19:17.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975980+0000", 2024-09-18T00:19:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016002799999999999 2024-09-18T00:19:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.976299+0000", 2024-09-18T00:19:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031853300000000002 2024-09-18T00:19:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:17.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989936+0000", 2024-09-18T00:19:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013637046999999999 2024-09-18T00:19:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990025+0000", 2024-09-18T00:19:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9040000000000001e-05 2024-09-18T00:19:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990064+0000", 2024-09-18T00:19:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9394000000000003e-05 2024-09-18T00:19:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990098+0000", 2024-09-18T00:19:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3275999999999998e-05 2024-09-18T00:19:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18688 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.625453685, 2024-09-18T00:19:17.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040041259000000003, 2024-09-18T00:19:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18688 2024-09-18T00:19:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950166+0000", 2024-09-18T00:19:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:19:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951295+0000", 2024-09-18T00:19:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011293480000000001 2024-09-18T00:19:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951328+0000", 2024-09-18T00:19:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2524999999999998e-05 2024-09-18T00:19:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951346+0000", 2024-09-18T00:19:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8170999999999999e-05 2024-09-18T00:19:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978546+0000", 2024-09-18T00:19:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027200418000000001 2024-09-18T00:19:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978666+0000", 2024-09-18T00:19:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119913 2024-09-18T00:19:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.979098+0000", 2024-09-18T00:19:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00043169399999999999 2024-09-18T00:19:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990000+0000", 2024-09-18T00:19:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010901719000000001 2024-09-18T00:19:17.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990109+0000", 2024-09-18T00:19:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010964299999999999 2024-09-18T00:19:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990152+0000", 2024-09-18T00:19:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2877000000000003e-05 2024-09-18T00:19:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990206+0000", 2024-09-18T00:19:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3717000000000001e-05 2024-09-18T00:19:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.631 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18689 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.624241987000001, 2024-09-18T00:19:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051022338, 2024-09-18T00:19:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.632 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18689 2024-09-18T00:19:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951378+0000", 2024-09-18T00:19:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:19:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951385+0000", 2024-09-18T00:19:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-18T00:19:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951402+0000", 2024-09-18T00:19:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6538999999999999e-05 2024-09-18T00:19:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951565+0000", 2024-09-18T00:19:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016307999999999999 2024-09-18T00:19:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988232+0000", 2024-09-18T00:19:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036667656999999999 2024-09-18T00:19:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988318+0000", 2024-09-18T00:19:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5348999999999999e-05 2024-09-18T00:19:17.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988574+0000", 2024-09-18T00:19:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025670800000000002 2024-09-18T00:19:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002254+0000", 2024-09-18T00:19:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013679274 2024-09-18T00:19:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002326+0000", 2024-09-18T00:19:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2720999999999997e-05 2024-09-18T00:19:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002364+0000", 2024-09-18T00:19:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7314000000000002e-05 2024-09-18T00:19:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002399+0000", 2024-09-18T00:19:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5162e-05 2024-09-18T00:19:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.637 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18690 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.624024969000001, 2024-09-18T00:19:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050872134999999999, 2024-09-18T00:19:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18690 2024-09-18T00:19:17.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951595+0000", 2024-09-18T00:19:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4479999999999999e-06 2024-09-18T00:19:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952167+0000", 2024-09-18T00:19:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00057188400000000004 2024-09-18T00:19:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952192+0000", 2024-09-18T00:19:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4936999999999999e-05 2024-09-18T00:19:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952206+0000", 2024-09-18T00:19:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4565e-05 2024-09-18T00:19:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990303+0000", 2024-09-18T00:19:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038096954000000002 2024-09-18T00:19:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990403+0000", 2024-09-18T00:19:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9486000000000003e-05 2024-09-18T00:19:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990773+0000", 2024-09-18T00:19:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037037799999999998 2024-09-18T00:19:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002316+0000", 2024-09-18T00:19:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011542581 2024-09-18T00:19:17.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002407+0000", 2024-09-18T00:19:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0968e-05 2024-09-18T00:19:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002442+0000", 2024-09-18T00:19:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4962000000000002e-05 2024-09-18T00:19:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002466+0000", 2024-09-18T00:19:17.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3972e-05 2024-09-18T00:19:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18691 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.623392092, 2024-09-18T00:19:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051495530999999997, 2024-09-18T00:19:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.645 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18691 2024-09-18T00:19:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952228+0000", 2024-09-18T00:19:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:19:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952234+0000", 2024-09-18T00:19:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0310000000000004e-06 2024-09-18T00:19:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952248+0000", 2024-09-18T00:19:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4577e-05 2024-09-18T00:19:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952260+0000", 2024-09-18T00:19:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1659e-05 2024-09-18T00:19:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.000989+0000", 2024-09-18T00:19:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048729445000000003 2024-09-18T00:19:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001076+0000", 2024-09-18T00:19:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6323999999999995e-05 2024-09-18T00:19:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001295+0000", 2024-09-18T00:19:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021944299999999999 2024-09-18T00:19:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003502+0000", 2024-09-18T00:19:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022065209999999999 2024-09-18T00:19:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003581+0000", 2024-09-18T00:19:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9404000000000001e-05 2024-09-18T00:19:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003652+0000", 2024-09-18T00:19:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0506999999999994e-05 2024-09-18T00:19:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003722+0000", 2024-09-18T00:19:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0300000000000001e-05 2024-09-18T00:19:17.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18692 3.7 3:e69124d0:::rbd_data.112a6efb7564.0000000000000001:f [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.623344030999998, 2024-09-18T00:19:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050418114999999999, 2024-09-18T00:19:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.651 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18692 2024-09-18T00:19:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952275+0000", 2024-09-18T00:19:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4e-07 2024-09-18T00:19:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952281+0000", 2024-09-18T00:19:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2000000000000002e-06 2024-09-18T00:19:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952295+0000", 2024-09-18T00:19:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4731e-05 2024-09-18T00:19:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952307+0000", 2024-09-18T00:19:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1877e-05 2024-09-18T00:19:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002506+0000", 2024-09-18T00:19:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050199230999999997 2024-09-18T00:19:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002574+0000", 2024-09-18T00:19:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8076000000000003e-05 2024-09-18T00:19:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002693+0000", 2024-09-18T00:19:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011806 2024-09-18T00:19:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.655 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19383 3.5 3:a1859bb0:::rbd_data.112a7fe29ab4.000000000000015b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.953332434, 2024-09-18T00:19:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056760192000000001, 2024-09-18T00:19:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19383 2024-09-18T00:19:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622287+0000", 2024-09-18T00:19:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-18T00:19:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622292+0000", 2024-09-18T00:19:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6110000000000004e-06 2024-09-18T00:19:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622304+0000", 2024-09-18T00:19:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2106000000000001e-05 2024-09-18T00:19:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622309+0000", 2024-09-18T00:19:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.818e-06 2024-09-18T00:19:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622360+0000", 2024-09-18T00:19:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0430999999999999e-05 2024-09-18T00:19:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622438+0000", 2024-09-18T00:19:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8362999999999995e-05 2024-09-18T00:19:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678811+0000", 2024-09-18T00:19:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056372894 2024-09-18T00:19:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678906+0000", 2024-09-18T00:19:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5078e-05 2024-09-18T00:19:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.679046+0000", 2024-09-18T00:19:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139987 2024-09-18T00:19:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22019 3.c 3:37b77e94:::rbd_data.112a948d5956.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:19:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.076357604999998, 2024-09-18T00:19:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.039176065000000003, 2024-09-18T00:19:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.661 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22019 2024-09-18T00:19:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:19:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:19:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499262+0000", 2024-09-18T00:19:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:19:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499268+0000", 2024-09-18T00:19:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8780000000000003e-06 2024-09-18T00:19:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499283+0000", 2024-09-18T00:19:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5184000000000001e-05 2024-09-18T00:19:17.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499291+0000", 2024-09-18T00:19:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5959999999999998e-06 2024-09-18T00:19:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499364+0000", 2024-09-18T00:19:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2855000000000006e-05 2024-09-18T00:19:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499427+0000", 2024-09-18T00:19:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3645999999999993e-05 2024-09-18T00:19:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538252+0000", 2024-09-18T00:19:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038825207 2024-09-18T00:19:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538328+0000", 2024-09-18T00:19:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5938000000000004e-05 2024-09-18T00:19:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538437+0000", 2024-09-18T00:19:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108527 2024-09-18T00:19:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22468 3.5 3:a250f1ef:::rbd_data.112a948d5956.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.804251027999999, 2024-09-18T00:19:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.055599787999999997, 2024-09-18T00:19:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.666 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22468 2024-09-18T00:19:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771370+0000", 2024-09-18T00:19:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9629999999999998e-06 2024-09-18T00:19:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771385+0000", 2024-09-18T00:19:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5053e-05 2024-09-18T00:19:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771429+0000", 2024-09-18T00:19:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3192999999999998e-05 2024-09-18T00:19:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771453+0000", 2024-09-18T00:19:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3995999999999999e-05 2024-09-18T00:19:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771623+0000", 2024-09-18T00:19:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000170846 2024-09-18T00:19:17.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771838+0000", 2024-09-18T00:19:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021416600000000001 2024-09-18T00:19:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826866+0000", 2024-09-18T00:19:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.05502866 2024-09-18T00:19:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826906+0000", 2024-09-18T00:19:17.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9805999999999998e-05 2024-09-18T00:19:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826967+0000", 2024-09-18T00:19:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1105000000000005e-05 2024-09-18T00:19:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22469 3.2 3:474595b6:::rbd_data.112a948d5956.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.771512+0000", 2024-09-18T00:19:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.804106888, 2024-09-18T00:19:17.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034768343, 2024-09-18T00:19:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22469 2024-09-18T00:19:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771512+0000", 2024-09-18T00:19:17.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771512+0000", 2024-09-18T00:19:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771513+0000", 2024-09-18T00:19:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8840000000000001e-06 2024-09-18T00:19:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771523+0000", 2024-09-18T00:19:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7429999999999997e-06 2024-09-18T00:19:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771548+0000", 2024-09-18T00:19:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4426e-05 2024-09-18T00:19:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771564+0000", 2024-09-18T00:19:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6820000000000002e-05 2024-09-18T00:19:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771695+0000", 2024-09-18T00:19:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000130321 2024-09-18T00:19:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771897+0000", 2024-09-18T00:19:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00020252200000000001 2024-09-18T00:19:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.806120+0000", 2024-09-18T00:19:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034222475000000002 2024-09-18T00:19:17.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.806178+0000", 2024-09-18T00:19:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8696999999999997e-05 2024-09-18T00:19:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.806280+0000", 2024-09-18T00:19:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101455 2024-09-18T00:19:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22470 3.2 3:474595b6:::rbd_data.112a948d5956.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 18.748060043999999, 2024-09-18T00:19:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088321282000000001, 2024-09-18T00:19:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22470 2024-09-18T00:19:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.677 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827560+0000", 2024-09-18T00:19:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.762e-06 2024-09-18T00:19:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827569+0000", 2024-09-18T00:19:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8740000000000001e-06 2024-09-18T00:19:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827585+0000", 2024-09-18T00:19:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5962000000000001e-05 2024-09-18T00:19:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827595+0000", 2024-09-18T00:19:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5910000000000002e-06 2024-09-18T00:19:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827811+0000", 2024-09-18T00:19:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000216858 2024-09-18T00:19:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:19:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827818+0000", 2024-09-18T00:19:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.99e-06 2024-09-18T00:19:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829799+0000", 2024-09-18T00:19:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019803849999999999 2024-09-18T00:19:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829884+0000", 2024-09-18T00:19:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4852999999999995e-05 2024-09-18T00:19:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915616+0000", 2024-09-18T00:19:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085732808999999993 2024-09-18T00:19:17.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915711+0000", 2024-09-18T00:19:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4578000000000001e-05 2024-09-18T00:19:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915880+0000", 2024-09-18T00:19:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016861999999999999 2024-09-18T00:19:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.682 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:30171 3.18 3:18b382f1:::rbd_data.112a990280ef.0000000000000005:15 [read 385024~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:12.453913+0000", 2024-09-18T00:19:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.1217049860000001, 2024-09-18T00:19:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034893857, 2024-09-18T00:19:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 30171 2024-09-18T00:19:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:17.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.453913+0000", 2024-09-18T00:19:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.453913+0000", 2024-09-18T00:19:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.453914+0000", 2024-09-18T00:19:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1799999999999999e-07 2024-09-18T00:19:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.453915+0000", 2024-09-18T00:19:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3659999999999999e-06 2024-09-18T00:19:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.453919+0000", 2024-09-18T00:19:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.403e-06 2024-09-18T00:19:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.453923+0000", 2024-09-18T00:19:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3179999999999998e-06 2024-09-18T00:19:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.484449+0000", 2024-09-18T00:19:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030526635 2024-09-18T00:19:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.484531+0000", 2024-09-18T00:19:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2010999999999998e-05 2024-09-18T00:19:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.488807+0000", 2024-09-18T00:19:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0042759060000000003 2024-09-18T00:19:17.686 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:17.687 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:17.687 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:17.688 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:19:17.689 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:17.689 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:17.689 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:17.690 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:17.690 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:17.690 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:17.690 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:17.691 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:17.681+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:17.691 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:17.691 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:17.708 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:17.708 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:17.728 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:17.728 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:17.747 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:17.747 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:17.751 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:17.751 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:17.790 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:17.785+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:17.818 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:17.818 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:17.832 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:19:17.891 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:17.885+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:17.991 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:17.989+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:17.997 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:17.997 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:18.011 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:19:18.098 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:18.097+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:18.177 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:18.177 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:18.191 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:19:18.235 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:18.233+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:18.295 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:18.293+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:18.366 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:18.366 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:18.380 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:19:18.389 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:19:18.422 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:18.421+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:18.432 INFO:teuthology.orchestra.run.smithi097.stdout:leaving image image_client.0-clone20 intact 2024-09-18T00:19:18.496 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:18.489+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:18.548 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:18.548 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:18.562 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:19:18.606 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:18.605+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:18.684 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:18.685 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:18.697 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:18.693+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:18.698 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:19:18.798 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:18.793+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:18.872 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:18.872 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:18.886 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:19:18.934 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:18.933+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:18.999 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:18.997+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:19.059 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:19.059 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:19.075 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:19:19.122 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:19.121+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:19.201 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:19.197+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:19.242 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:19.243 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:19.257 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:19:19.302 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:19.301+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:19.345 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is set 2024-09-18T00:19:19.376 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:19.376 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:19.391 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:19:19.417 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:19.413+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:19.479 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-18T00:19:19.505 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:19.501+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:19.511 INFO:teuthology.orchestra.run.smithi097.stdout:4865 write 0x901159e thru 0x901a97d (0x93e0 bytes) 2024-09-18T00:19:19.537 INFO:teuthology.orchestra.run.smithi097.stdout:4866 write 0xac24654 thru 0xac2efbe (0xa96b bytes) 2024-09-18T00:19:19.555 INFO:teuthology.orchestra.run.smithi097.stdout:4868 punch from 0x92efb74 to 0x92f721f, (0x76ab bytes) 2024-09-18T00:19:19.555 INFO:teuthology.orchestra.run.smithi097.stdout:4870 write 0x98ff4e4 thru 0x990553b (0x6058 bytes) 2024-09-18T00:19:19.569 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:19.569 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:19.571 INFO:teuthology.orchestra.run.smithi097.stdout:4873 writesame 0x9008d2f thru 0x90125e1 data_size 0x1f(0x98b3 bytes) 2024-09-18T00:19:19.577 INFO:teuthology.orchestra.run.smithi097.stdout:4875 punch from 0xb79d751 to 0xb7a2e5f, (0x570e bytes) 2024-09-18T00:19:19.577 INFO:teuthology.orchestra.run.smithi097.stdout:4876 write 0xb93802a thru 0xb943e4e (0xbe25 bytes) 2024-09-18T00:19:19.590 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:19:19.606 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:19.601+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:19.638 INFO:teuthology.orchestra.run.smithi097.stdout:4877 trunc from 0xbab4224 to 0xa9c1f05 2024-09-18T00:19:19.665 INFO:teuthology.orchestra.run.smithi097.stdout:4878 read 0xa7006e5 thru 0xa704f7b (0x4897 bytes) 2024-09-18T00:19:19.666 INFO:teuthology.orchestra.run.smithi097.stdout:4879 read 0x3007694 thru 0x3015b35 (0xe4a2 bytes) 2024-09-18T00:19:19.666 INFO:teuthology.orchestra.run.smithi097.stdout:4880 write 0x32e814a thru 0x32f1fce (0x9e85 bytes) 2024-09-18T00:19:19.679 INFO:teuthology.orchestra.run.smithi097.stdout:4881 punch from 0x6015ed to 0x6029ea, (0x13fd bytes) 2024-09-18T00:19:19.686 INFO:teuthology.orchestra.run.smithi097.stdout:4882 trunc from 0xa9c1f05 to 0x752872d 2024-09-18T00:19:19.709 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:19.705+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:19.716 INFO:teuthology.orchestra.run.smithi097.stdout:4883 read 0x2e26980 thru 0x2e337f2 (0xce73 bytes) 2024-09-18T00:19:19.716 INFO:teuthology.orchestra.run.smithi097.stdout:4884 write 0x7134747 thru 0x713fefd (0xb7b7 bytes) 2024-09-18T00:19:19.731 INFO:teuthology.orchestra.run.smithi097.stdout:4887 trunc from 0x752872d to 0x9aaccd7 2024-09-18T00:19:19.746 INFO:teuthology.orchestra.run.smithi097.stdout:4889 write 0xa82a4d6 thru 0xa83495e (0xa489 bytes) 2024-09-18T00:19:19.752 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:19:19.753 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:19.753 INFO:teuthology.orchestra.run.smithi097.stdout:4892 read 0x44648e thru 0x44ff0a (0x9a7d bytes) 2024-09-18T00:19:19.756 INFO:teuthology.orchestra.run.smithi097.stdout:4893 read 0x60c5866 thru 0x60ce967 (0x9102 bytes) 2024-09-18T00:19:19.756 INFO:teuthology.orchestra.run.smithi097.stdout:4895 read 0x6dfbdcd thru 0x6e05911 (0x9b45 bytes) 2024-09-18T00:19:19.756 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:19.756 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:19.756 INFO:teuthology.orchestra.run.smithi097.stdout:4898 read 0x8712cdc thru 0x871f10f (0xc434 bytes) 2024-09-18T00:19:19.757 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:19.757 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:19.767 INFO:teuthology.orchestra.run.smithi097.stdout:4900 writesame 0xc14113c thru 0xc149050 data_size 0x1(0x7f15 bytes) 2024-09-18T00:19:19.767 INFO:teuthology.orchestra.run.smithi097.stdout:4901 write 0xa603746 thru 0xa6134ea (0xfda5 bytes) 2024-09-18T00:19:19.773 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:19:19.777 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:19.777 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:19.777 INFO:teuthology.orchestra.run.smithi097.stdout:4906 read 0x4e6cf3a thru 0x4e7b21e (0xe2e5 bytes) 2024-09-18T00:19:19.793 INFO:teuthology.orchestra.run.smithi097.stdout:4907 trunc from 0xc149051 to 0x3d449f8 2024-09-18T00:19:19.810 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:19.805+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:19.819 INFO:teuthology.orchestra.run.smithi097.stdout:4908 read 0xc9c4f5 thru 0xca59a6 (0x94b2 bytes) 2024-09-18T00:19:19.819 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:19.819 INFO:teuthology.orchestra.run.smithi097.stdout:4911 read 0x27e9331 thru 0x27eb453 (0x2123 bytes) 2024-09-18T00:19:19.819 INFO:teuthology.orchestra.run.smithi097.stdout:4912 read 0x22c2b9c thru 0x22c37ce (0xc33 bytes) 2024-09-18T00:19:19.839 INFO:teuthology.orchestra.run.smithi097.stdout:4913 writesame 0xd6625af thru 0xd669e64 data_size 0x1(0x78b6 bytes) 2024-09-18T00:19:19.839 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:19.839 INFO:teuthology.orchestra.run.smithi097.stdout:4917 write 0xd2a3156 thru 0xd2afc54 (0xcaff bytes) 2024-09-18T00:19:19.851 INFO:teuthology.orchestra.run.smithi097.stdout:4919 write 0xaab33fb thru 0xaabe5ef (0xb1f5 bytes) 2024-09-18T00:19:19.861 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:19.862 INFO:teuthology.orchestra.run.smithi097.stdout:4923 read 0x12a458a thru 0x12b17ed (0xd264 bytes) 2024-09-18T00:19:19.865 INFO:teuthology.orchestra.run.smithi097.stdout:4924 read 0xc8cc061 thru 0xc8ceeff (0x2e9f bytes) 2024-09-18T00:19:19.865 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:19.872 INFO:teuthology.orchestra.run.smithi097.stdout:4928 writesame 0xd6b056c thru 0xd6bab33 data_size 0x14(0xa5c8 bytes) 2024-09-18T00:19:19.872 INFO:teuthology.orchestra.run.smithi097.stdout:4929 read 0xa0717b0 thru 0xa07ca20 (0xb271 bytes) 2024-09-18T00:19:19.872 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:19.872 INFO:teuthology.orchestra.run.smithi097.stdout:4931 read 0x103c289 thru 0x1045039 (0x8db1 bytes) 2024-09-18T00:19:19.874 INFO:teuthology.orchestra.run.smithi097.stdout:4932 read 0x9d6ef1f thru 0x9d77fb6 (0x9098 bytes) 2024-09-18T00:19:19.874 INFO:teuthology.orchestra.run.smithi097.stdout:4933 punch from 0x70cf78 to 0x7113eb, (0x4473 bytes) 2024-09-18T00:19:19.874 INFO:teuthology.orchestra.run.smithi097.stdout:4934 read 0xe355c thru 0xefef6 (0xc99b bytes) 2024-09-18T00:19:19.884 INFO:teuthology.orchestra.run.smithi097.stdout:4936 writesame 0xdb9128b thru 0xdb95940 data_size 0x6(0x46b6 bytes) 2024-09-18T00:19:19.902 INFO:teuthology.orchestra.run.smithi097.stdout:4937 trunc from 0xdb95941 to 0x1cccced 2024-09-18T00:19:19.910 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:19.905+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:19.916 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:19.916 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:19.926 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:19.934 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:19:19.935 INFO:teuthology.orchestra.run.smithi097.stdout:4939 read 0x3e57e3 thru 0x3f157d (0xbd9b bytes) 2024-09-18T00:19:19.944 INFO:teuthology.orchestra.run.smithi097.stdout:4940 write 0x26f67b5 thru 0x2701ea5 (0xb6f1 bytes) 2024-09-18T00:19:19.951 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:19.954 INFO:teuthology.orchestra.run.smithi097.stdout:4941 read 0xb0f1af thru 0xb11fae (0x2e00 bytes) 2024-09-18T00:19:19.954 INFO:teuthology.orchestra.run.smithi097.stdout:4942 read 0x19107a1 thru 0x191d6a5 (0xcf05 bytes) 2024-09-18T00:19:19.954 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:19.954 INFO:teuthology.orchestra.run.smithi097.stdout:4945 write 0x1fabd81 thru 0x1fb1b65 (0x5de5 bytes) 2024-09-18T00:19:19.973 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:19.973 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:19.977 INFO:teuthology.orchestra.run.smithi097.stdout:4946 write 0x569138d thru 0x569b734 (0xa3a8 bytes) 2024-09-18T00:19:19.996 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:19.996 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:20.006 INFO:teuthology.orchestra.run.smithi097.stdout:4947 write 0xb55616f thru 0xb563565 (0xd3f7 bytes) 2024-09-18T00:19:20.011 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:20.005+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:20.019 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:20.019 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:20.025 INFO:teuthology.orchestra.run.smithi097.stdout:4948 trunc from 0xb563566 to 0xe2b6f74 2024-09-18T00:19:20.033 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:20.033 INFO:teuthology.orchestra.run.smithi097.stdout:4950 write 0x942ed85 thru 0x9437190 (0x840c bytes) 2024-09-18T00:19:20.042 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:20.042 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:20.044 INFO:teuthology.orchestra.run.smithi097.stdout:4952 writesame 0xa8f0b6c thru 0xa8f51c6 data_size 0x1f(0x465b bytes) 2024-09-18T00:19:20.063 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:20.064 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:20.066 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:20.067 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:20.067 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:20.067 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:20.067 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:20.067 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:20.067 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:20.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:20.088 INFO:teuthology.orchestra.run.smithi097.stdout:4955 trunc from 0xe2b6f74 to 0x699e249 2024-09-18T00:19:20.112 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:20.109+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:20.116 INFO:teuthology.orchestra.run.smithi097.stdout:4957 trunc from 0x699e249 to 0xced0b99 2024-09-18T00:19:20.123 INFO:teuthology.orchestra.run.smithi097.stdout:4959 read 0x1ffd1c thru 0x205300 (0x55e5 bytes) 2024-09-18T00:19:20.125 INFO:teuthology.orchestra.run.smithi097.stdout:4961 writesame 0x28cea7a thru 0x28da662 data_size 0xf(0xbbe9 bytes) 2024-09-18T00:19:20.135 INFO:teuthology.orchestra.run.smithi097.stdout:4963 write 0xcd98695 thru 0xcda1dbb (0x9727 bytes) 2024-09-18T00:19:20.146 INFO:teuthology.orchestra.run.smithi097.stdout:4964 read 0x563380f thru 0x5636fb2 (0x37a4 bytes) 2024-09-18T00:19:20.147 INFO:teuthology.orchestra.run.smithi097.stdout:4965 read 0x9be5867 thru 0x9bf384c (0xdfe6 bytes) 2024-09-18T00:19:20.147 INFO:teuthology.orchestra.run.smithi097.stdout:4966 write 0xbf26672 thru 0xbf28850 (0x21df bytes) 2024-09-18T00:19:20.154 INFO:teuthology.orchestra.run.smithi097.stdout:4967 write 0x9cd9a5e thru 0x9ce2c39 (0x91dc bytes) 2024-09-18T00:19:20.168 INFO:teuthology.orchestra.run.smithi097.stdout:4968 read 0x5880000 thru 0x5888ab5 (0x8ab6 bytes) 2024-09-18T00:19:20.168 INFO:teuthology.orchestra.run.smithi097.stdout:4969 read 0x6f0f1e0 thru 0x6f174c5 (0x82e6 bytes) 2024-09-18T00:19:20.168 INFO:teuthology.orchestra.run.smithi097.stdout:4970 writesame 0x46b2eb8 thru 0x46bacd3 data_size 0xe(0x7e1c bytes) 2024-09-18T00:19:20.179 INFO:teuthology.orchestra.run.smithi097.stdout:4971 read 0x8b80bcf thru 0x8b87636 (0x6a68 bytes) 2024-09-18T00:19:20.179 INFO:teuthology.orchestra.run.smithi097.stdout:4972 write 0x51b85bc thru 0x51c7323 (0xed68 bytes) 2024-09-18T00:19:20.192 INFO:teuthology.orchestra.run.smithi097.stdout:4973 punch from 0x64833f to 0x64cb50, (0x4811 bytes) 2024-09-18T00:19:20.192 INFO:teuthology.orchestra.run.smithi097.stdout:4974 read 0x8f665da thru 0x8f6cda8 (0x67cf bytes) 2024-09-18T00:19:20.192 INFO:teuthology.orchestra.run.smithi097.stdout:4976 writesame 0x236e24e thru 0x2373425 data_size 0x61(0x51d8 bytes) 2024-09-18T00:19:20.208 INFO:teuthology.orchestra.run.smithi097.stdout:4977 trunc from 0xced0b99 to 0x9e7f9c4 2024-09-18T00:19:20.213 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:20.209+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:20.219 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:20.219 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:20.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:20.219 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:20.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.220 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2230 3.17 3:ef492888:::rbd_data.112aa7ba4578.000000000000014f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:20.220 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:20.220 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.952220664000002, 2024-09-18T00:19:20.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075065283999999996, 2024-09-18T00:19:20.220 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.220 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.220 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:20.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.220 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.220 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2230 2024-09-18T00:19:20.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.220 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:20.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:20.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266014+0000", 2024-09-18T00:19:20.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:19:20.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266029+0000", 2024-09-18T00:19:20.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4616e-05 2024-09-18T00:19:20.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266053+0000", 2024-09-18T00:19:20.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4369999999999999e-05 2024-09-18T00:19:20.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266067+0000", 2024-09-18T00:19:20.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4243999999999999e-05 2024-09-18T00:19:20.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266157+0000", 2024-09-18T00:19:20.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9909999999999998e-05 2024-09-18T00:19:20.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266327+0000", 2024-09-18T00:19:20.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016981200000000001 2024-09-18T00:19:20.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:19:20.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269180+0000", 2024-09-18T00:19:20.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028526559999999999 2024-09-18T00:19:20.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:20.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340955+0000", 2024-09-18T00:19:20.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.071774904 2024-09-18T00:19:20.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:20.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340974+0000", 2024-09-18T00:19:20.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8824000000000001e-05 2024-09-18T00:19:20.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:20.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341028+0000", 2024-09-18T00:19:20.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4456999999999997e-05 2024-09-18T00:19:20.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341076+0000", 2024-09-18T00:19:20.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8284000000000001e-05 2024-09-18T00:19:20.226 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.226 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.226 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.226 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2230 3.17 e36/23 ondisk, result = 0)", 2024-09-18T00:19:20.227 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:20.227 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.934877083000003, 2024-09-18T00:19:20.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057609645000000001, 2024-09-18T00:19:20.227 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.227 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.227 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:20.227 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:20.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:20.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283356+0000", 2024-09-18T00:19:20.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.44e-06 2024-09-18T00:19:20.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283362+0000", 2024-09-18T00:19:20.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2539999999999998e-06 2024-09-18T00:19:20.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283382+0000", 2024-09-18T00:19:20.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9896000000000001e-05 2024-09-18T00:19:20.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283398+0000", 2024-09-18T00:19:20.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5489999999999999e-05 2024-09-18T00:19:20.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340936+0000", 2024-09-18T00:19:20.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057538649999999997 2024-09-18T00:19:20.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340947+0000", 2024-09-18T00:19:20.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0981e-05 2024-09-18T00:19:20.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340964+0000", 2024-09-18T00:19:20.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6934000000000001e-05 2024-09-18T00:19:20.230 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.230 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.230 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.230 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2419 3.17 3:ed94b133:::rbd_data.112aa7ba4578.00000000000001b5:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:19:20.231 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:20.231 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.480755610000003, 2024-09-18T00:19:20.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.064761812000000002, 2024-09-18T00:19:20.231 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.231 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.231 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:20.231 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.231 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.231 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.231 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2419 2024-09-18T00:19:20.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.231 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:20.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:20.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737479+0000", 2024-09-18T00:19:20.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6089999999999999e-06 2024-09-18T00:19:20.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737487+0000", 2024-09-18T00:19:20.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7900000000000005e-06 2024-09-18T00:19:20.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737507+0000", 2024-09-18T00:19:20.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9936e-05 2024-09-18T00:19:20.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737518+0000", 2024-09-18T00:19:20.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1046999999999999e-05 2024-09-18T00:19:20.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737599+0000", 2024-09-18T00:19:20.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0697000000000003e-05 2024-09-18T00:19:20.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737697+0000", 2024-09-18T00:19:20.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8058000000000002e-05 2024-09-18T00:19:20.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:19:20.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.740445+0000", 2024-09-18T00:19:20.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027481279999999999 2024-09-18T00:19:20.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:20.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802023+0000", 2024-09-18T00:19:20.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.061577321999999997 2024-09-18T00:19:20.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:20.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802060+0000", 2024-09-18T00:19:20.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7586000000000002e-05 2024-09-18T00:19:20.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:20.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802158+0000", 2024-09-18T00:19:20.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7583000000000004e-05 2024-09-18T00:19:20.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802238+0000", 2024-09-18T00:19:20.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0055999999999994e-05 2024-09-18T00:19:20.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.237 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.238 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2419 3.17 e37/23 ondisk, result = 0)", 2024-09-18T00:19:20.238 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:20.238 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.468530663999999, 2024-09-18T00:19:20.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052342948, 2024-09-18T00:19:20.238 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.238 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.238 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:20.238 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:20.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:20.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749702+0000", 2024-09-18T00:19:20.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:19:20.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749710+0000", 2024-09-18T00:19:20.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7279999999999995e-06 2024-09-18T00:19:20.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749736+0000", 2024-09-18T00:19:20.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5797999999999999e-05 2024-09-18T00:19:20.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749751+0000", 2024-09-18T00:19:20.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5238999999999999e-05 2024-09-18T00:19:20.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801745+0000", 2024-09-18T00:19:20.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051994022000000001 2024-09-18T00:19:20.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801770+0000", 2024-09-18T00:19:20.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5103e-05 2024-09-18T00:19:20.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802044+0000", 2024-09-18T00:19:20.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027395799999999998 2024-09-18T00:19:20.242 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.242 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.242 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.242 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6503 3.1b 3:d9387a1f:::rbd_data.112a3a39fc45.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:20.242 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:20.243 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.653857694000003, 2024-09-18T00:19:20.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050510547000000003, 2024-09-18T00:19:20.243 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.243 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.243 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:20.243 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.243 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.243 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.243 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6503 2024-09-18T00:19:20.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.243 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:20.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:20.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564376+0000", 2024-09-18T00:19:20.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-06 2024-09-18T00:19:20.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564386+0000", 2024-09-18T00:19:20.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0312999999999999e-05 2024-09-18T00:19:20.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564408+0000", 2024-09-18T00:19:20.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2001000000000002e-05 2024-09-18T00:19:20.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564421+0000", 2024-09-18T00:19:20.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2271e-05 2024-09-18T00:19:20.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564491+0000", 2024-09-18T00:19:20.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0630000000000006e-05 2024-09-18T00:19:20.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564606+0000", 2024-09-18T00:19:20.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011449 2024-09-18T00:19:20.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:20.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.566743+0000", 2024-09-18T00:19:20.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021373830000000001 2024-09-18T00:19:20.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:20.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614738+0000", 2024-09-18T00:19:20.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047995359000000001 2024-09-18T00:19:20.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:20.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614759+0000", 2024-09-18T00:19:20.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1010999999999999e-05 2024-09-18T00:19:20.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:20.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614826+0000", 2024-09-18T00:19:20.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6654000000000003e-05 2024-09-18T00:19:20.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614884+0000", 2024-09-18T00:19:20.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8332999999999999e-05 2024-09-18T00:19:20.248 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.248 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.248 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.248 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6503 3.1b e42/23 ondisk, result = 0)", 2024-09-18T00:19:20.248 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:19:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.644598225999999, 2024-09-18T00:19:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041115407, 2024-09-18T00:19:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:19:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573633+0000", 2024-09-18T00:19:20.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573635+0000", 2024-09-18T00:19:20.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5689999999999999e-06 2024-09-18T00:19:20.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573643+0000", 2024-09-18T00:19:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9699999999999999e-06 2024-09-18T00:19:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573668+0000", 2024-09-18T00:19:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5536000000000001e-05 2024-09-18T00:19:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.573683+0000", 2024-09-18T00:19:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4025e-05 2024-09-18T00:19:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614717+0000", 2024-09-18T00:19:20.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041034517 2024-09-18T00:19:20.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614730+0000", 2024-09-18T00:19:20.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2945999999999999e-05 2024-09-18T00:19:20.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614749+0000", 2024-09-18T00:19:20.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8844000000000001e-05 2024-09-18T00:19:20.252 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.253 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.253 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.253 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6518 3.1 3:8061bf9f:::rbd_data.112a3a39fc45.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:20.254 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:20.254 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.56741942, 2024-09-18T00:19:20.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062303123000000002, 2024-09-18T00:19:20.254 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.254 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.254 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:20.254 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.255 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.255 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.255 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6518 2024-09-18T00:19:20.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.255 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:20.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:20.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650814+0000", 2024-09-18T00:19:20.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.629e-06 2024-09-18T00:19:20.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650823+0000", 2024-09-18T00:19:20.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1800000000000002e-06 2024-09-18T00:19:20.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650843+0000", 2024-09-18T00:19:20.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9836000000000001e-05 2024-09-18T00:19:20.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650854+0000", 2024-09-18T00:19:20.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1564e-05 2024-09-18T00:19:20.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650917+0000", 2024-09-18T00:19:20.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3076000000000004e-05 2024-09-18T00:19:20.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.651029+0000", 2024-09-18T00:19:20.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111154 2024-09-18T00:19:20.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:20.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.655531+0000", 2024-09-18T00:19:20.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045027629999999999 2024-09-18T00:19:20.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:20.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712924+0000", 2024-09-18T00:19:20.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057393006000000003 2024-09-18T00:19:20.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:20.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712988+0000", 2024-09-18T00:19:20.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000005e-05 2024-09-18T00:19:20.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:20.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713056+0000", 2024-09-18T00:19:20.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7904999999999994e-05 2024-09-18T00:19:20.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713115+0000", 2024-09-18T00:19:20.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9339999999999998e-05 2024-09-18T00:19:20.266 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.266 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.266 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.266 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6518 3.1 e42/23 ondisk, result = 0)", 2024-09-18T00:19:20.267 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:20.267 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.554994336999997, 2024-09-18T00:19:20.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049887304, 2024-09-18T00:19:20.267 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.267 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.267 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:20.268 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:20.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:20.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663239+0000", 2024-09-18T00:19:20.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:19:20.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663246+0000", 2024-09-18T00:19:20.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4189999999999996e-06 2024-09-18T00:19:20.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663265+0000", 2024-09-18T00:19:20.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8323000000000001e-05 2024-09-18T00:19:20.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663277+0000", 2024-09-18T00:19:20.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2065999999999999e-05 2024-09-18T00:19:20.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712965+0000", 2024-09-18T00:19:20.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049688317000000003 2024-09-18T00:19:20.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712977+0000", 2024-09-18T00:19:20.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.206e-05 2024-09-18T00:19:20.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713125+0000", 2024-09-18T00:19:20.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014749899999999999 2024-09-18T00:19:20.275 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.275 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.276 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.276 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:8831 3.12 3:49cac8cf:::rbd_data.112a9f4259df.0000000000000130:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T00:19:20.276 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:20.277 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.937134088000001, 2024-09-18T00:19:20.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104125047, 2024-09-18T00:19:20.277 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.277 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.277 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:20.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.278 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 8831 2024-09-18T00:19:20.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.279 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:20.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:20.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281099+0000", 2024-09-18T00:19:20.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.029e-06 2024-09-18T00:19:20.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281103+0000", 2024-09-18T00:19:20.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7759999999999997e-06 2024-09-18T00:19:20.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281114+0000", 2024-09-18T00:19:20.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1090000000000001e-05 2024-09-18T00:19:20.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281120+0000", 2024-09-18T00:19:20.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.604e-06 2024-09-18T00:19:20.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281164+0000", 2024-09-18T00:19:20.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3696999999999999e-05 2024-09-18T00:19:20.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281270+0000", 2024-09-18T00:19:20.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106041 2024-09-18T00:19:20.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:20.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.285596+0000", 2024-09-18T00:19:20.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043266659999999998 2024-09-18T00:19:20.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:20.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385028+0000", 2024-09-18T00:19:20.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.099431808999999996 2024-09-18T00:19:20.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:20.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385094+0000", 2024-09-18T00:19:20.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5740000000000004e-05 2024-09-18T00:19:20.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:20.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385166+0000", 2024-09-18T00:19:20.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1637000000000005e-05 2024-09-18T00:19:20.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385223+0000", 2024-09-18T00:19:20.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6957999999999999e-05 2024-09-18T00:19:20.290 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.290 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.290 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.291 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:8831 3.12 e48/32 ondisk, result = 0)", 2024-09-18T00:19:20.291 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:20.291 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.920619977999998, 2024-09-18T00:19:20.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087621558000000002, 2024-09-18T00:19:20.292 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.292 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.292 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:20.292 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:20.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:20.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297613+0000", 2024-09-18T00:19:20.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3400000000000001e-06 2024-09-18T00:19:20.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297620+0000", 2024-09-18T00:19:20.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5869999999999998e-06 2024-09-18T00:19:20.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297639+0000", 2024-09-18T00:19:20.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9888999999999999e-05 2024-09-18T00:19:20.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297661+0000", 2024-09-18T00:19:20.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1772000000000001e-05 2024-09-18T00:19:20.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385073+0000", 2024-09-18T00:19:20.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087411303999999995 2024-09-18T00:19:20.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385086+0000", 2024-09-18T00:19:20.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3040000000000001e-05 2024-09-18T00:19:20.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385233+0000", 2024-09-18T00:19:20.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014762600000000001 2024-09-18T00:19:20.300 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.300 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.301 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.301 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22269 3.17 3:e9a1eb35:::rbd_data.112a948d5956.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:20.301 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:20.301 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.309088152000001, 2024-09-18T00:19:20.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.063462693000000001, 2024-09-18T00:19:20.301 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.301 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.301 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:20.301 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.301 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.301 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.302 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22269 2024-09-18T00:19:20.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.302 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:20.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:20.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909146+0000", 2024-09-18T00:19:20.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3700000000000002e-06 2024-09-18T00:19:20.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909160+0000", 2024-09-18T00:19:20.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3825000000000001e-05 2024-09-18T00:19:20.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909178+0000", 2024-09-18T00:19:20.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8567999999999999e-05 2024-09-18T00:19:20.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909189+0000", 2024-09-18T00:19:20.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0881e-05 2024-09-18T00:19:20.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909270+0000", 2024-09-18T00:19:20.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0846999999999994e-05 2024-09-18T00:19:20.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909383+0000", 2024-09-18T00:19:20.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112704 2024-09-18T00:19:20.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:20.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972310+0000", 2024-09-18T00:19:20.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062927081999999995 2024-09-18T00:19:20.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:20.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972433+0000", 2024-09-18T00:19:20.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-18T00:19:20.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972606+0000", 2024-09-18T00:19:20.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017317799999999999 2024-09-18T00:19:20.306 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.306 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.306 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.306 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22379 3.17 3:e9a1eb35:::rbd_data.112a948d5956.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:20.306 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:20.306 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 21.815012854999999, 2024-09-18T00:19:20.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.12560279099999999, 2024-09-18T00:19:20.306 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.306 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.306 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:20.306 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.306 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.307 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.307 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22379 2024-09-18T00:19:20.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.307 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:20.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:20.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403221+0000", 2024-09-18T00:19:20.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3269999999999999e-06 2024-09-18T00:19:20.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403232+0000", 2024-09-18T00:19:20.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1392e-05 2024-09-18T00:19:20.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403264+0000", 2024-09-18T00:19:20.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1146999999999998e-05 2024-09-18T00:19:20.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403279+0000", 2024-09-18T00:19:20.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5109000000000001e-05 2024-09-18T00:19:20.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403424+0000", 2024-09-18T00:19:20.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014516999999999999 2024-09-18T00:19:20.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403567+0000", 2024-09-18T00:19:20.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142824 2024-09-18T00:19:20.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:20.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528723+0000", 2024-09-18T00:19:20.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.12515647299999999 2024-09-18T00:19:20.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:20.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528764+0000", 2024-09-18T00:19:20.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0624000000000003e-05 2024-09-18T00:19:20.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528822+0000", 2024-09-18T00:19:20.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7725e-05 2024-09-18T00:19:20.311 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.311 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.311 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.311 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:27243 3.1b 3:df1bd230:::rbd_data.112aa34d8ec3.0000000000000029:16 [read 16384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:20.311 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:11.971339+0000", 2024-09-18T00:19:20.311 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.2468924159999997, 2024-09-18T00:19:20.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.039816139, 2024-09-18T00:19:20.311 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.311 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.312 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:20.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.312 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.312 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27243 2024-09-18T00:19:20.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.312 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971339+0000", 2024-09-18T00:19:20.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971339+0000", 2024-09-18T00:19:20.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971340+0000", 2024-09-18T00:19:20.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2600000000000002e-07 2024-09-18T00:19:20.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971343+0000", 2024-09-18T00:19:20.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.994e-06 2024-09-18T00:19:20.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971351+0000", 2024-09-18T00:19:20.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8769999999999993e-06 2024-09-18T00:19:20.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971357+0000", 2024-09-18T00:19:20.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1040000000000003e-06 2024-09-18T00:19:20.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.006500+0000", 2024-09-18T00:19:20.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.035143488000000001 2024-09-18T00:19:20.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.006572+0000", 2024-09-18T00:19:20.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2268999999999995e-05 2024-09-18T00:19:20.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.011155+0000", 2024-09-18T00:19:20.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045828809999999996 2024-09-18T00:19:20.315 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.315 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.315 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.316 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:27293 3.1b 3:df1bd230:::rbd_data.112aa34d8ec3.0000000000000029:16 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:20.316 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:11.971960+0000", 2024-09-18T00:19:20.316 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.2462711560000006, 2024-09-18T00:19:20.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.03820109, 2024-09-18T00:19:20.316 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.316 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.316 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:20.316 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.316 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.316 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.316 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27293 2024-09-18T00:19:20.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.316 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971960+0000", 2024-09-18T00:19:20.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971960+0000", 2024-09-18T00:19:20.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971961+0000", 2024-09-18T00:19:20.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8400000000000004e-07 2024-09-18T00:19:20.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971972+0000", 2024-09-18T00:19:20.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.079e-05 2024-09-18T00:19:20.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971980+0000", 2024-09-18T00:19:20.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5059999999999995e-06 2024-09-18T00:19:20.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971987+0000", 2024-09-18T00:19:20.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2609999999999999e-06 2024-09-18T00:19:20.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.009999+0000", 2024-09-18T00:19:20.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038012137000000001 2024-09-18T00:19:20.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010078+0000", 2024-09-18T00:19:20.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9580999999999996e-05 2024-09-18T00:19:20.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010162+0000", 2024-09-18T00:19:20.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3131000000000001e-05 2024-09-18T00:19:20.320 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.320 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.320 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.320 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:27295 3.1b 3:df1bd230:::rbd_data.112aa34d8ec3.0000000000000029:16 [read 75776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:20.320 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:11.971996+0000", 2024-09-18T00:19:20.320 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.2462358089999999, 2024-09-18T00:19:20.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038329568000000001, 2024-09-18T00:19:20.320 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.320 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.320 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:20.320 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.321 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.321 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.321 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27295 2024-09-18T00:19:20.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.321 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971996+0000", 2024-09-18T00:19:20.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971996+0000", 2024-09-18T00:19:20.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971997+0000", 2024-09-18T00:19:20.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2399999999999997e-07 2024-09-18T00:19:20.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.971999+0000", 2024-09-18T00:19:20.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7760000000000002e-06 2024-09-18T00:19:20.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972007+0000", 2024-09-18T00:19:20.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7449999999999995e-06 2024-09-18T00:19:20.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972013+0000", 2024-09-18T00:19:20.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7869999999999998e-06 2024-09-18T00:19:20.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010186+0000", 2024-09-18T00:19:20.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038172999999999999 2024-09-18T00:19:20.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010259+0000", 2024-09-18T00:19:20.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2725999999999995e-05 2024-09-18T00:19:20.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010325+0000", 2024-09-18T00:19:20.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6810000000000006e-05 2024-09-18T00:19:20.324 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.324 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.324 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.324 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:27297 3.1b 3:df1bd230:::rbd_data.112aa34d8ec3.0000000000000029:16 [read 104448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:20.324 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:11.972021+0000", 2024-09-18T00:19:20.325 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.2462104449999991, 2024-09-18T00:19:20.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038465829, 2024-09-18T00:19:20.325 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.325 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.325 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:20.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.325 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27297 2024-09-18T00:19:20.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.325 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972021+0000", 2024-09-18T00:19:20.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972021+0000", 2024-09-18T00:19:20.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972022+0000", 2024-09-18T00:19:20.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1399999999999997e-07 2024-09-18T00:19:20.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972025+0000", 2024-09-18T00:19:20.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.717e-06 2024-09-18T00:19:20.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972032+0000", 2024-09-18T00:19:20.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6480000000000003e-06 2024-09-18T00:19:20.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972038+0000", 2024-09-18T00:19:20.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9549999999999997e-06 2024-09-18T00:19:20.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010349+0000", 2024-09-18T00:19:20.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038311031000000002 2024-09-18T00:19:20.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010421+0000", 2024-09-18T00:19:20.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1365999999999994e-05 2024-09-18T00:19:20.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010487+0000", 2024-09-18T00:19:20.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6497999999999999e-05 2024-09-18T00:19:20.329 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.329 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.329 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.329 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:27298 3.1b 3:df1bd230:::rbd_data.112aa34d8ec3.0000000000000029:16 [read 153600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:20.329 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:11.972047+0000", 2024-09-18T00:19:20.329 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.2461850800000001, 2024-09-18T00:19:20.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038602364, 2024-09-18T00:19:20.329 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.329 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.329 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:20.329 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.329 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.330 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.330 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27298 2024-09-18T00:19:20.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.330 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972047+0000", 2024-09-18T00:19:20.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972047+0000", 2024-09-18T00:19:20.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972047+0000", 2024-09-18T00:19:20.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2e-07 2024-09-18T00:19:20.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972050+0000", 2024-09-18T00:19:20.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0699999999999998e-06 2024-09-18T00:19:20.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972058+0000", 2024-09-18T00:19:20.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7379999999999994e-06 2024-09-18T00:19:20.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972071+0000", 2024-09-18T00:19:20.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3196999999999999e-05 2024-09-18T00:19:20.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010510+0000", 2024-09-18T00:19:20.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038439373999999998 2024-09-18T00:19:20.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010581+0000", 2024-09-18T00:19:20.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0773999999999997e-05 2024-09-18T00:19:20.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010649+0000", 2024-09-18T00:19:20.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7691e-05 2024-09-18T00:19:20.333 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.333 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.333 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.333 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:27300 3.1b 3:df1bd230:::rbd_data.112aa34d8ec3.0000000000000029:16 [read 182272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:20.333 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:11.972080+0000", 2024-09-18T00:19:20.333 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.2461513879999995, 2024-09-18T00:19:20.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038744935000000001, 2024-09-18T00:19:20.334 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.334 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.334 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:20.334 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.334 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.334 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.334 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27300 2024-09-18T00:19:20.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.334 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972080+0000", 2024-09-18T00:19:20.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972080+0000", 2024-09-18T00:19:20.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972081+0000", 2024-09-18T00:19:20.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5700000000000002e-07 2024-09-18T00:19:20.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972089+0000", 2024-09-18T00:19:20.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4570000000000005e-06 2024-09-18T00:19:20.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972098+0000", 2024-09-18T00:19:20.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7320000000000004e-06 2024-09-18T00:19:20.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972104+0000", 2024-09-18T00:19:20.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.207e-06 2024-09-18T00:19:20.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010672+0000", 2024-09-18T00:19:20.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038568189000000003 2024-09-18T00:19:20.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010743+0000", 2024-09-18T00:19:20.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0535000000000001e-05 2024-09-18T00:19:20.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010825+0000", 2024-09-18T00:19:20.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2257999999999997e-05 2024-09-18T00:19:20.338 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.338 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.338 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.338 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:27302 3.1b 3:df1bd230:::rbd_data.112aa34d8ec3.0000000000000029:16 [read 210944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:20.338 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:11.972113+0000", 2024-09-18T00:19:20.338 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.2461184079999992, 2024-09-18T00:19:20.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038872974999999997, 2024-09-18T00:19:20.338 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.338 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.338 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:20.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.339 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27302 2024-09-18T00:19:20.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.339 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972113+0000", 2024-09-18T00:19:20.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972113+0000", 2024-09-18T00:19:20.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972114+0000", 2024-09-18T00:19:20.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-07 2024-09-18T00:19:20.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972116+0000", 2024-09-18T00:19:20.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.633e-06 2024-09-18T00:19:20.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972124+0000", 2024-09-18T00:19:20.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.374e-06 2024-09-18T00:19:20.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972130+0000", 2024-09-18T00:19:20.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7799999999999997e-06 2024-09-18T00:19:20.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010848+0000", 2024-09-18T00:19:20.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038718283999999999 2024-09-18T00:19:20.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010917+0000", 2024-09-18T00:19:20.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9622000000000005e-05 2024-09-18T00:19:20.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.010986+0000", 2024-09-18T00:19:20.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8707000000000005e-05 2024-09-18T00:19:20.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.342 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.342 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:27304 3.1b 3:df1bd230:::rbd_data.112aa34d8ec3.0000000000000029:16 [read 239616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-18T00:19:20.343 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:11.972138+0000", 2024-09-18T00:19:20.343 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.2460938329999998, 2024-09-18T00:19:20.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.039007344999999999, 2024-09-18T00:19:20.343 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.343 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.343 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:20.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.343 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.343 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 27304 2024-09-18T00:19:20.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.343 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972138+0000", 2024-09-18T00:19:20.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972138+0000", 2024-09-18T00:19:20.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972138+0000", 2024-09-18T00:19:20.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2900000000000004e-07 2024-09-18T00:19:20.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972141+0000", 2024-09-18T00:19:20.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.74e-06 2024-09-18T00:19:20.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972149+0000", 2024-09-18T00:19:20.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7270000000000007e-06 2024-09-18T00:19:20.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:11.972155+0000", 2024-09-18T00:19:20.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9390000000000002e-06 2024-09-18T00:19:20.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.011010+0000", 2024-09-18T00:19:20.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038855033999999997 2024-09-18T00:19:20.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.011080+0000", 2024-09-18T00:19:20.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0457000000000007e-05 2024-09-18T00:19:20.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:12.011145+0000", 2024-09-18T00:19:20.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4919000000000006e-05 2024-09-18T00:19:20.347 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.347 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.347 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.347 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.347 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.347 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:20.348 INFO:teuthology.orchestra.run.smithi097.stdout:4978 writesame 0x3dee00c thru 0x3df3556 data_size 0xb(0x554b bytes) 2024-09-18T00:19:20.349 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:19:20.350 INFO:teuthology.orchestra.run.smithi097.stdout:4979 write 0x47cf5c8 thru 0x47d683e (0x7277 bytes) 2024-09-18T00:19:20.350 INFO:teuthology.orchestra.run.smithi097.stdout:4980 write 0xc9b8fd2 thru 0xc9bbed5 (0x2f04 bytes) 2024-09-18T00:19:20.350 INFO:teuthology.orchestra.run.smithi097.stdout:4981 read 0xb8470d2 thru 0xb85111d (0xa04c bytes) 2024-09-18T00:19:20.350 INFO:teuthology.orchestra.run.smithi097.stdout:4982 writesame 0x8ca2d37 thru 0x8caabbd data_size 0x3d(0x7e87 bytes) 2024-09-18T00:19:20.350 INFO:teuthology.orchestra.run.smithi097.stdout:4983 read 0xc1bcc40 thru 0xc1bcdd7 (0x198 bytes) 2024-09-18T00:19:20.350 INFO:teuthology.orchestra.run.smithi097.stdout:4984 punch from 0x9c760bb to 0x9c7b3b5, (0x52fa bytes) 2024-09-18T00:19:20.350 INFO:teuthology.orchestra.run.smithi097.stdout:4985 trunc from 0xc9bbed6 to 0x2d38914 2024-09-18T00:19:20.350 INFO:teuthology.orchestra.run.smithi097.stdout:4986 write 0x487902f thru 0x4885c4e (0xcc20 bytes) 2024-09-18T00:19:20.351 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:20.341+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:20.357 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:20.368 INFO:teuthology.orchestra.run.smithi097.stdout:4988 writesame 0x616f030 thru 0x61725ce data_size 0x35(0x359f bytes) 2024-09-18T00:19:20.368 INFO:teuthology.orchestra.run.smithi097.stdout:4989 punch from 0x559a4ac to 0x55a49f0, (0xa544 bytes) 2024-09-18T00:19:20.368 INFO:teuthology.orchestra.run.smithi097.stdout:4991 punch from 0x56e5f40 to 0x56f2784, (0xc844 bytes) 2024-09-18T00:19:20.368 INFO:teuthology.orchestra.run.smithi097.stdout:4992 writesame 0x3a3a6e0 thru 0x3a42ae3 data_size 0x22(0x8404 bytes) 2024-09-18T00:19:20.378 INFO:teuthology.orchestra.run.smithi097.stdout:4994 read 0x47d7a95 thru 0x47db598 (0x3b04 bytes) 2024-09-18T00:19:20.379 INFO:teuthology.orchestra.run.smithi097.stdout:4995 punch from 0x55aad4b to 0x55b12e2, (0x6597 bytes) 2024-09-18T00:19:20.379 INFO:teuthology.orchestra.run.smithi097.stdout:4996 read 0x55606f6 thru 0x5565bcc (0x54d7 bytes) 2024-09-18T00:19:20.380 INFO:teuthology.orchestra.run.smithi097.stdout:4998 read 0x171825b thru 0x17203e1 (0x8187 bytes) 2024-09-18T00:19:20.390 INFO:teuthology.orchestra.run.smithi097.stdout:4999 trunc from 0x61725cf to 0xce11eb9 2024-09-18T00:19:20.398 INFO:teuthology.orchestra.run.smithi097.stdout:5000 read 0x654f080 thru 0x6558bf9 (0x9b7a bytes) 2024-09-18T00:19:20.398 INFO:teuthology.orchestra.run.smithi097.stdout:5001 flatten 2024-09-18T00:19:20.449 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:20.445+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:20.526 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:20.526 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:20.527 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:20.527 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:20.549 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:19:20.550 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:20.549+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:20.651 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:20.645+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:20.752 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:20.749+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:20.779 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:20.779 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:20.779 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:20.779 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:20.779 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:20.792 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:19:20.846 INFO:teuthology.orchestra.run.smithi097.stdout:5002 trunc from 0xce11eb9 to 0x719ffc1 2024-09-18T00:19:20.852 INFO:teuthology.orchestra.run.smithi097.stdout:5004 write 0x89275f thru 0x8939c6 (0x1268 bytes) 2024-09-18T00:19:20.853 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:20.849+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:20.864 INFO:teuthology.orchestra.run.smithi097.stdout:5005 trunc from 0x719ffc1 to 0xd9f2a04 2024-09-18T00:19:20.871 INFO:teuthology.orchestra.run.smithi097.stdout:5006 write 0x3a4449f thru 0x3a50a2b (0xc58d bytes) 2024-09-18T00:19:20.873 INFO:teuthology.orchestra.run.smithi097.stdout:5007 read 0x84b84d thru 0x851b4b (0x62ff bytes) 2024-09-18T00:19:20.874 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:20.874 INFO:teuthology.orchestra.run.smithi097.stdout:5010 writesame 0x87eaf16 thru 0x87f0ff4 data_size 0x1(0x60df bytes) 2024-09-18T00:19:20.887 INFO:teuthology.orchestra.run.smithi097.stdout:5011 write 0x8c29291 thru 0x8c2d228 (0x3f98 bytes) 2024-09-18T00:19:20.896 INFO:teuthology.orchestra.run.smithi097.stdout:5012 punch from 0x3bff061 to 0x3c0838b, (0x932a bytes) 2024-09-18T00:19:20.896 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:20.896 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:20.896 INFO:teuthology.orchestra.run.smithi097.stdout:5015 write 0x73a9ce4 thru 0x73aa1b1 (0x4ce bytes) 2024-09-18T00:19:20.900 INFO:teuthology.orchestra.run.smithi097.stdout:5016 trunc from 0xd9f2a04 to 0xd4ec3a1 2024-09-18T00:19:20.905 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:20.905 INFO:teuthology.orchestra.run.smithi097.stdout:5022 write 0x3170f63 thru 0x317ebc0 (0xdc5e bytes) 2024-09-18T00:19:20.913 INFO:teuthology.orchestra.run.smithi097.stdout:5024 punch from 0x2361464 to 0x236b36d, (0x9f09 bytes) 2024-09-18T00:19:20.913 INFO:teuthology.orchestra.run.smithi097.stdout:5025 punch from 0x2857e7f to 0x285f28f, (0x7410 bytes) 2024-09-18T00:19:20.913 INFO:teuthology.orchestra.run.smithi097.stdout:5026 read 0x39631c1 thru 0x3970848 (0xd688 bytes) 2024-09-18T00:19:20.920 INFO:teuthology.orchestra.run.smithi097.stdout:5027 write 0xdccb4f4 thru 0xdcd1262 (0x5d6f bytes) 2024-09-18T00:19:20.928 INFO:teuthology.orchestra.run.smithi097.stdout:5028 write 0xc10d03a thru 0xc1154fc (0x84c3 bytes) 2024-09-18T00:19:20.936 INFO:teuthology.orchestra.run.smithi097.stdout:5031 read 0x1e71d64 thru 0x1e8187e (0xfb1b bytes) 2024-09-18T00:19:20.937 INFO:teuthology.orchestra.run.smithi097.stdout:5033 read 0x4c40d24 thru 0x4c477a0 (0x6a7d bytes) 2024-09-18T00:19:20.942 INFO:teuthology.orchestra.run.smithi097.stdout:5034 trunc from 0xdcd1263 to 0x9a1cb0b 2024-09-18T00:19:20.954 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:20.953+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:20.956 INFO:teuthology.orchestra.run.smithi097.stdout:5035 read 0xc02ce0 thru 0xc0cded (0xa10e bytes) 2024-09-18T00:19:20.956 INFO:teuthology.orchestra.run.smithi097.stdout:5039 read 0x2137bfb thru 0x214240b (0xa811 bytes) 2024-09-18T00:19:20.964 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:20.964 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:20.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:20.964 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:20.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.965 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:5722 3.4 e41/23 3:20381d86:::rbd_data.112a3a39fc45.000000000000001d:head v 41'107, mlcod=40'105)", 2024-09-18T00:19:20.965 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:20.965 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.627730841000002, 2024-09-18T00:19:20.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01185172, 2024-09-18T00:19:20.965 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.965 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.965 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:20.965 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:20.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:20.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335497+0000", 2024-09-18T00:19:20.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.505e-06 2024-09-18T00:19:20.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335945+0000", 2024-09-18T00:19:20.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000447687 2024-09-18T00:19:20.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335961+0000", 2024-09-18T00:19:20.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6430999999999999e-05 2024-09-18T00:19:20.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335969+0000", 2024-09-18T00:19:20.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:19:20.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336030+0000", 2024-09-18T00:19:20.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1255999999999998e-05 2024-09-18T00:19:20.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336061+0000", 2024-09-18T00:19:20.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0579e-05 2024-09-18T00:19:20.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347296+0000", 2024-09-18T00:19:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011235663 2024-09-18T00:19:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347347+0000", 2024-09-18T00:19:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0914000000000001e-05 2024-09-18T00:19:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11457 3.0 3:0214e7b3:::rbd_data.112ab1fa102a.0000000000000150:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.703364903000001, 2024-09-18T00:19:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028499165, 2024-09-18T00:19:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11457 2024-09-18T00:19:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259865+0000", 2024-09-18T00:19:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7230000000000002e-06 2024-09-18T00:19:20.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259883+0000", 2024-09-18T00:19:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8067000000000002e-05 2024-09-18T00:19:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259924+0000", 2024-09-18T00:19:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0550999999999999e-05 2024-09-18T00:19:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259943+0000", 2024-09-18T00:19:20.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:19:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260054+0000", 2024-09-18T00:19:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011075 2024-09-18T00:19:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260138+0000", 2024-09-18T00:19:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4549000000000007e-05 2024-09-18T00:19:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.287926+0000", 2024-09-18T00:19:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027787642000000001 2024-09-18T00:19:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288090+0000", 2024-09-18T00:19:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016402499999999999 2024-09-18T00:19:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288361+0000", 2024-09-18T00:19:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027097800000000001 2024-09-18T00:19:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.974 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18661 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.028597038000001, 2024-09-18T00:19:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020008037999999999, 2024-09-18T00:19:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.975 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18661 2024-09-18T00:19:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934633+0000", 2024-09-18T00:19:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8e-06 2024-09-18T00:19:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935511+0000", 2024-09-18T00:19:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087785700000000001 2024-09-18T00:19:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935561+0000", 2024-09-18T00:19:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9472000000000003e-05 2024-09-18T00:19:20.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935586+0000", 2024-09-18T00:19:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5616e-05 2024-09-18T00:19:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935838+0000", 2024-09-18T00:19:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025202699999999999 2024-09-18T00:19:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.936023+0000", 2024-09-18T00:19:20.978 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018480300000000001 2024-09-18T00:19:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953545+0000", 2024-09-18T00:19:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017522157 2024-09-18T00:19:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953723+0000", 2024-09-18T00:19:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017753800000000001 2024-09-18T00:19:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.979 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954637+0000", 2024-09-18T00:19:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091476800000000005 2024-09-18T00:19:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18662 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.027592101, 2024-09-18T00:19:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019388843999999999, 2024-09-18T00:19:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.980 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18662 2024-09-18T00:19:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.981 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935636+0000", 2024-09-18T00:19:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.663e-06 2024-09-18T00:19:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935646+0000", 2024-09-18T00:19:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000006e-06 2024-09-18T00:19:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.982 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935672+0000", 2024-09-18T00:19:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5868999999999999e-05 2024-09-18T00:19:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935692+0000", 2024-09-18T00:19:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0604000000000001e-05 2024-09-18T00:19:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950738+0000", 2024-09-18T00:19:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015045868 2024-09-18T00:19:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.983 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950992+0000", 2024-09-18T00:19:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025368400000000001 2024-09-18T00:19:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954674+0000", 2024-09-18T00:19:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003682154 2024-09-18T00:19:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954806+0000", 2024-09-18T00:19:20.984 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013252000000000001 2024-09-18T00:19:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.955023+0000", 2024-09-18T00:19:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021681200000000001 2024-09-18T00:19:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19382 3.4 3:22f09d3a:::rbd_data.112a7fe29ab4.000000000000015a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.341182051000001, 2024-09-18T00:19:20.985 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075119302999999998, 2024-09-18T00:19:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19382 2024-09-18T00:19:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.986 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622045+0000", 2024-09-18T00:19:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0020000000000001e-06 2024-09-18T00:19:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622053+0000", 2024-09-18T00:19:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2679999999999997e-06 2024-09-18T00:19:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.987 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622062+0000", 2024-09-18T00:19:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3749999999999992e-06 2024-09-18T00:19:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622068+0000", 2024-09-18T00:19:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5729999999999998e-06 2024-09-18T00:19:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622118+0000", 2024-09-18T00:19:20.988 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0696999999999999e-05 2024-09-18T00:19:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622222+0000", 2024-09-18T00:19:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103286 2024-09-18T00:19:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697061+0000", 2024-09-18T00:19:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074839141999999997 2024-09-18T00:19:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:20.989 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697102+0000", 2024-09-18T00:19:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1513e-05 2024-09-18T00:19:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697164+0000", 2024-09-18T00:19:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1446999999999996e-05 2024-09-18T00:19:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20245 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:20.990 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.807523721999999, 2024-09-18T00:19:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099981159999999996, 2024-09-18T00:19:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20245 2024-09-18T00:19:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.991 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.65e-07 2024-09-18T00:19:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155711+0000", 2024-09-18T00:19:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:19:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155720+0000", 2024-09-18T00:19:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6580000000000008e-06 2024-09-18T00:19:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155725+0000", 2024-09-18T00:19:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3430000000000003e-06 2024-09-18T00:19:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165479+0000", 2024-09-18T00:19:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097538239999999995 2024-09-18T00:19:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165556+0000", 2024-09-18T00:19:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6724000000000006e-05 2024-09-18T00:19:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165701+0000", 2024-09-18T00:19:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145129 2024-09-18T00:19:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.994 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20246 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.807493859000001, 2024-09-18T00:19:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01018784, 2024-09-18T00:19:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20246 2024-09-18T00:19:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.995 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:20.996 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2599999999999998e-07 2024-09-18T00:19:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155735+0000", 2024-09-18T00:19:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2699999999999999e-06 2024-09-18T00:19:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155744+0000", 2024-09-18T00:19:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8629999999999997e-06 2024-09-18T00:19:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.997 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155749+0000", 2024-09-18T00:19:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0359999999999997e-06 2024-09-18T00:19:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165748+0000", 2024-09-18T00:19:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099991449999999992 2024-09-18T00:19:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165819+0000", 2024-09-18T00:19:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0964999999999996e-05 2024-09-18T00:19:20.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165920+0000", 2024-09-18T00:19:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101135 2024-09-18T00:19:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20247 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.807470516999999, 2024-09-18T00:19:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010443987, 2024-09-18T00:19:20.999 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20247 2024-09-18T00:19:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.000 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1899999999999998e-07 2024-09-18T00:19:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155760+0000", 2024-09-18T00:19:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6459999999999999e-06 2024-09-18T00:19:21.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155767+0000", 2024-09-18T00:19:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0350000000000004e-06 2024-09-18T00:19:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155774+0000", 2024-09-18T00:19:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:19:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165960+0000", 2024-09-18T00:19:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186051999999999 2024-09-18T00:19:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166048+0000", 2024-09-18T00:19:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8848e-05 2024-09-18T00:19:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166200+0000", 2024-09-18T00:19:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015147000000000001 2024-09-18T00:19:21.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20248 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.807445352999999, 2024-09-18T00:19:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010617721, 2024-09-18T00:19:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.004 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20248 2024-09-18T00:19:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155782+0000", 2024-09-18T00:19:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.84e-07 2024-09-18T00:19:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155784+0000", 2024-09-18T00:19:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2800000000000002e-06 2024-09-18T00:19:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155790+0000", 2024-09-18T00:19:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5679999999999996e-06 2024-09-18T00:19:21.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155795+0000", 2024-09-18T00:19:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6929999999999998e-06 2024-09-18T00:19:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166245+0000", 2024-09-18T00:19:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010449847999999999 2024-09-18T00:19:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166307+0000", 2024-09-18T00:19:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2287999999999997e-05 2024-09-18T00:19:21.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166399+0000", 2024-09-18T00:19:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.166e-05 2024-09-18T00:19:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20249 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.807424977, 2024-09-18T00:19:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010811983000000001, 2024-09-18T00:19:21.008 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20249 2024-09-18T00:19:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155802+0000", 2024-09-18T00:19:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:19:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155804+0000", 2024-09-18T00:19:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:19:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155810+0000", 2024-09-18T00:19:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0580000000000003e-06 2024-09-18T00:19:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155817+0000", 2024-09-18T00:19:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3980000000000004e-06 2024-09-18T00:19:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166451+0000", 2024-09-18T00:19:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010634825000000001 2024-09-18T00:19:21.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166520+0000", 2024-09-18T00:19:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8317000000000003e-05 2024-09-18T00:19:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166613+0000", 2024-09-18T00:19:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3801000000000005e-05 2024-09-18T00:19:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:21913 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.669579831, 2024-09-18T00:19:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090508916999999994, 2024-09-18T00:19:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 21913 2024-09-18T00:19:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293648+0000", 2024-09-18T00:19:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6400000000000001e-07 2024-09-18T00:19:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293653+0000", 2024-09-18T00:19:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3820000000000003e-06 2024-09-18T00:19:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293665+0000", 2024-09-18T00:19:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1778e-05 2024-09-18T00:19:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293673+0000", 2024-09-18T00:19:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4130000000000006e-06 2024-09-18T00:19:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293719+0000", 2024-09-18T00:19:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5534999999999997e-05 2024-09-18T00:19:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293773+0000", 2024-09-18T00:19:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4389999999999999e-05 2024-09-18T00:19:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384053+0000", 2024-09-18T00:19:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090279617000000006 2024-09-18T00:19:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384088+0000", 2024-09-18T00:19:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5466000000000002e-05 2024-09-18T00:19:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384156+0000", 2024-09-18T00:19:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7471999999999994e-05 2024-09-18T00:19:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22274 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.858463611000001, 2024-09-18T00:19:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.069820742000000005, 2024-09-18T00:19:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22274 2024-09-18T00:19:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104766+0000", 2024-09-18T00:19:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3859999999999999e-06 2024-09-18T00:19:21.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104783+0000", 2024-09-18T00:19:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6697e-05 2024-09-18T00:19:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104816+0000", 2024-09-18T00:19:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2786000000000001e-05 2024-09-18T00:19:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104833+0000", 2024-09-18T00:19:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7125999999999998e-05 2024-09-18T00:19:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104965+0000", 2024-09-18T00:19:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013215899999999999 2024-09-18T00:19:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.105103+0000", 2024-09-18T00:19:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138193 2024-09-18T00:19:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174496+0000", 2024-09-18T00:19:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.06939244 2024-09-18T00:19:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174530+0000", 2024-09-18T00:19:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4601999999999998e-05 2024-09-18T00:19:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174584+0000", 2024-09-18T00:19:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3353000000000002e-05 2024-09-18T00:19:21.022 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22485 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write 16521958~30847] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.009674601, 2024-09-18T00:19:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012574648000000001, 2024-09-18T00:19:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22485 2024-09-18T00:19:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953554+0000", 2024-09-18T00:19:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1150000000000001e-06 2024-09-18T00:19:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962036+0000", 2024-09-18T00:19:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0084817599999999996 2024-09-18T00:19:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962062+0000", 2024-09-18T00:19:21.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6499000000000002e-05 2024-09-18T00:19:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962074+0000", 2024-09-18T00:19:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2095e-05 2024-09-18T00:19:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962165+0000", 2024-09-18T00:19:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0703000000000002e-05 2024-09-18T00:19:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962276+0000", 2024-09-18T00:19:21.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111406 2024-09-18T00:19:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965925+0000", 2024-09-18T00:19:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036488919999999999 2024-09-18T00:19:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965992+0000", 2024-09-18T00:19:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6806999999999999e-05 2024-09-18T00:19:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.966127+0000", 2024-09-18T00:19:21.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013437099999999999 2024-09-18T00:19:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22681 3.4 3:22e76021:::rbd_data.112a948d5956.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 21.675675296000001, 2024-09-18T00:19:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076591189000000004, 2024-09-18T00:19:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.028 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22681 2024-09-18T00:19:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287554+0000", 2024-09-18T00:19:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.514e-06 2024-09-18T00:19:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287567+0000", 2024-09-18T00:19:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-18T00:19:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287597+0000", 2024-09-18T00:19:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0406e-05 2024-09-18T00:19:21.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287626+0000", 2024-09-18T00:19:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9306e-05 2024-09-18T00:19:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287801+0000", 2024-09-18T00:19:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017411400000000001 2024-09-18T00:19:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.288053+0000", 2024-09-18T00:19:21.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025223500000000001 2024-09-18T00:19:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364016+0000", 2024-09-18T00:19:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075963301999999996 2024-09-18T00:19:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364064+0000", 2024-09-18T00:19:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7886000000000002e-05 2024-09-18T00:19:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364142+0000", 2024-09-18T00:19:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8345999999999998e-05 2024-09-18T00:19:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:24700 3.4 3:23a8f5d4:::rbd_data.112a990280ef.0000000000000016:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:19:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 14.025209317, 2024-09-18T00:19:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091975649000000007, 2024-09-18T00:19:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.033 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.034 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.034 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.034 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 24700 2024-09-18T00:19:21.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.034 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:19:21.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:19:21.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938019+0000", 2024-09-18T00:19:21.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:19:21.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938030+0000", 2024-09-18T00:19:21.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0278000000000001e-05 2024-09-18T00:19:21.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938053+0000", 2024-09-18T00:19:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2946999999999999e-05 2024-09-18T00:19:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938062+0000", 2024-09-18T00:19:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8509999999999994e-06 2024-09-18T00:19:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938140+0000", 2024-09-18T00:19:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7664999999999996e-05 2024-09-18T00:19:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938314+0000", 2024-09-18T00:19:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017398500000000001 2024-09-18T00:19:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029896+0000", 2024-09-18T00:19:21.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091582078999999997 2024-09-18T00:19:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029930+0000", 2024-09-18T00:19:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3723999999999998e-05 2024-09-18T00:19:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029993+0000", 2024-09-18T00:19:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2901e-05 2024-09-18T00:19:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37798 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410235+0000", 2024-09-18T00:19:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.55299151599999996, 2024-09-18T00:19:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099276060000000003, 2024-09-18T00:19:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37798 2024-09-18T00:19:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.039 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410235+0000", 2024-09-18T00:19:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410235+0000", 2024-09-18T00:19:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410235+0000", 2024-09-18T00:19:21.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4699999999999998e-07 2024-09-18T00:19:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410236+0000", 2024-09-18T00:19:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1069999999999999e-06 2024-09-18T00:19:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410240+0000", 2024-09-18T00:19:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3119999999999998e-06 2024-09-18T00:19:21.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410242+0000", 2024-09-18T00:19:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3860000000000001e-06 2024-09-18T00:19:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.419934+0000", 2024-09-18T00:19:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0096915720000000007 2024-09-18T00:19:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420021+0000", 2024-09-18T00:19:21.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6941999999999995e-05 2024-09-18T00:19:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420163+0000", 2024-09-18T00:19:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014203999999999999 2024-09-18T00:19:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37799 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:19:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.55298116600000002, 2024-09-18T00:19:21.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186016000000001, 2024-09-18T00:19:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37799 2024-09-18T00:19:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:19:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:19:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410246+0000", 2024-09-18T00:19:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.35e-07 2024-09-18T00:19:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410247+0000", 2024-09-18T00:19:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0389999999999999e-06 2024-09-18T00:19:21.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410250+0000", 2024-09-18T00:19:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3380000000000001e-06 2024-09-18T00:19:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410252+0000", 2024-09-18T00:19:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5150000000000001e-06 2024-09-18T00:19:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420198+0000", 2024-09-18T00:19:21.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099454360000000002 2024-09-18T00:19:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420284+0000", 2024-09-18T00:19:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5638999999999998e-05 2024-09-18T00:19:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420431+0000", 2024-09-18T00:19:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014781399999999999 2024-09-18T00:19:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37800 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:19:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.55297088999999999, 2024-09-18T00:19:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010452559, 2024-09-18T00:19:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37800 2024-09-18T00:19:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.048 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:19:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:19:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:19:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2499999999999999e-07 2024-09-18T00:19:21.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410257+0000", 2024-09-18T00:19:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0750000000000001e-06 2024-09-18T00:19:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410260+0000", 2024-09-18T00:19:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.428e-06 2024-09-18T00:19:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410263+0000", 2024-09-18T00:19:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5339999999999998e-06 2024-09-18T00:19:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420467+0000", 2024-09-18T00:19:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010204615 2024-09-18T00:19:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420557+0000", 2024-09-18T00:19:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9183000000000003e-05 2024-09-18T00:19:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420708+0000", 2024-09-18T00:19:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015149900000000001 2024-09-18T00:19:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37801 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:19:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.55296055300000002, 2024-09-18T00:19:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010623357, 2024-09-18T00:19:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.052 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37801 2024-09-18T00:19:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:19:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:19:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:19:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.23e-07 2024-09-18T00:19:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410271+0000", 2024-09-18T00:19:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8040000000000001e-06 2024-09-18T00:19:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410274+0000", 2024-09-18T00:19:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5310000000000002e-06 2024-09-18T00:19:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410277+0000", 2024-09-18T00:19:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.605e-06 2024-09-18T00:19:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420743+0000", 2024-09-18T00:19:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010465487000000001 2024-09-18T00:19:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420811+0000", 2024-09-18T00:19:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8845999999999998e-05 2024-09-18T00:19:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420889+0000", 2024-09-18T00:19:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7861000000000006e-05 2024-09-18T00:19:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.056 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37802 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:19:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.552945927, 2024-09-18T00:19:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010745463, 2024-09-18T00:19:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37802 2024-09-18T00:19:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:19:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:19:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:19:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6199999999999999e-07 2024-09-18T00:19:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410282+0000", 2024-09-18T00:19:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.085e-06 2024-09-18T00:19:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410286+0000", 2024-09-18T00:19:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6610000000000002e-06 2024-09-18T00:19:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410288+0000", 2024-09-18T00:19:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6410000000000002e-06 2024-09-18T00:19:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.059 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420910+0000", 2024-09-18T00:19:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010621479 2024-09-18T00:19:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420952+0000", 2024-09-18T00:19:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2497999999999999e-05 2024-09-18T00:19:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.421026+0000", 2024-09-18T00:19:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3837000000000005e-05 2024-09-18T00:19:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.060 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.061 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.061 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.061 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:21.062 INFO:teuthology.orchestra.run.smithi097.stdout:5040 write 0xb67c961 thru 0xb6840f4 (0x7794 bytes) 2024-09-18T00:19:21.062 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:19:21.064 INFO:teuthology.orchestra.run.smithi097.stdout:5041 read 0x98aa13d thru 0x98acf4d (0x2e11 bytes) 2024-09-18T00:19:21.065 INFO:teuthology.orchestra.run.smithi097.stdout:5042 write 0x11ec717 thru 0x11efa44 (0x332e bytes) 2024-09-18T00:19:21.065 INFO:teuthology.orchestra.run.smithi097.stdout:5043 punch from 0x2bac719 to 0x2bbabcd, (0xe4b4 bytes) 2024-09-18T00:19:21.065 INFO:teuthology.orchestra.run.smithi097.stdout:5044 read 0x65f9221 thru 0x6607e7a (0xec5a bytes) 2024-09-18T00:19:21.065 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:21.065 INFO:teuthology.orchestra.run.smithi097.stdout:5046 read 0x6566fc3 thru 0x656f353 (0x8391 bytes) 2024-09-18T00:19:21.065 INFO:teuthology.orchestra.run.smithi097.stdout:5047 writesame 0x9bb8b05 thru 0x9bc28e4 data_size 0x30(0x9de0 bytes) 2024-09-18T00:19:21.065 INFO:teuthology.orchestra.run.smithi097.stdout:5048 writesame 0x46901a9 thru 0x469bbc8 data_size 0x10(0xba20 bytes) 2024-09-18T00:19:21.065 INFO:teuthology.orchestra.run.smithi097.stdout:5049 write 0x5d24f26 thru 0x5d30272 (0xb34d bytes) 2024-09-18T00:19:21.065 INFO:teuthology.orchestra.run.smithi097.stdout:5051 punch from 0x3e73603 to 0x3e7599c, (0x2399 bytes) 2024-09-18T00:19:21.065 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:21.065 INFO:teuthology.orchestra.run.smithi097.stdout:5053 writesame 0x602caf3 thru 0x602d866 data_size 0xc(0xd74 bytes) 2024-09-18T00:19:21.065 INFO:teuthology.orchestra.run.smithi097.stdout:5054 read 0x48f73e7 thru 0x49053ad (0xdfc7 bytes) 2024-09-18T00:19:21.066 INFO:teuthology.orchestra.run.smithi097.stdout:5055 read 0x2a2ef88 thru 0x2a3726d (0x82e6 bytes) 2024-09-18T00:19:21.066 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:21.066 INFO:teuthology.orchestra.run.smithi097.stdout:5058 write 0x36840f9 thru 0x3687c44 (0x3b4c bytes) 2024-09-18T00:19:21.066 INFO:teuthology.orchestra.run.smithi097.stdout:5059 punch from 0x8ed327e to 0x8edb4c9, (0x824b bytes) 2024-09-18T00:19:21.066 INFO:teuthology.orchestra.run.smithi097.stdout:5060 write 0x3f3cd04 thru 0x3f47774 (0xaa71 bytes) 2024-09-18T00:19:21.066 INFO:teuthology.orchestra.run.smithi097.stdout:5061 write 0x4378a9 thru 0x43e07f (0x67d7 bytes) 2024-09-18T00:19:21.066 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:21.066 INFO:teuthology.orchestra.run.smithi097.stdout:5063 writesame 0xb31f0a3 thru 0xb32a9b2 data_size 0x70(0xb910 bytes) 2024-09-18T00:19:21.066 INFO:teuthology.orchestra.run.smithi097.stdout:5064 read 0x689d668 thru 0x68a95cc (0xbf65 bytes) 2024-09-18T00:19:21.066 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:21.066 INFO:teuthology.orchestra.run.smithi097.stdout:5066 trunc from 0xb6840f5 to 0x9cd62c2 2024-09-18T00:19:21.067 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:21.057+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:21.094 INFO:teuthology.orchestra.run.smithi097.stdout:5067 write 0xc9e49c1 thru 0xc9ea623 (0x5c63 bytes) 2024-09-18T00:19:21.104 INFO:teuthology.orchestra.run.smithi097.stdout:5069 read 0x8d06a46 thru 0x8d0b5d4 (0x4b8f bytes) 2024-09-18T00:19:21.105 INFO:teuthology.orchestra.run.smithi097.stdout:5071 clone 22 order 24 su 8388608 sc 4 2024-09-18T00:19:21.163 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:21.157+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:21.200 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:21.200 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:21.200 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:21.212 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:19:21.297 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:21.293+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:21.364 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:21.361+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:21.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:19:21.379 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:21.380 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:21.380 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:21.380 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:21.380 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:21.395 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:19:21.465 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:21.461+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:21.562 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:19:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.759724156000004, 2024-09-18T00:19:21.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:19:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:19:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:19:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:19:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:19:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:19:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:19:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:19:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:19:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:19:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:19:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:19:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:19:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:19:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:19:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:19:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:19:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:19:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:19:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:19:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:19:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:19:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:19:21.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:19:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:19:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.743516283000005, 2024-09-18T00:19:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:19:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:19:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:19:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:19:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:19:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:19:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:19:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:19:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:19:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:19:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:19:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:19:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:19:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:19:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.716474963000003, 2024-09-18T00:19:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:19:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:19:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.572 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:19:21.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:19:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:19:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:19:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:19:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:19:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:19:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:19:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:19:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:19:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:19:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:19:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:19:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:19:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:19:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:19:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:19:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:19:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:19:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:19:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:19:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:19:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:19:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:21.577 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.695198411, 2024-09-18T00:19:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:19:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:19:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:19:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:19:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:19:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:19:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:19:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:19:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:19:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:19:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:19:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:19:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:19:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:19:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:19:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11460 3.19 3:9e578b6c:::rbd_data.112ab1fa102a.0000000000000153:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.292826959000003, 2024-09-18T00:19:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041989011999999999, 2024-09-18T00:19:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.581 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11460 2024-09-18T00:19:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267067+0000", 2024-09-18T00:19:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:19:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267088+0000", 2024-09-18T00:19:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0908999999999999e-05 2024-09-18T00:19:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267133+0000", 2024-09-18T00:19:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5232000000000003e-05 2024-09-18T00:19:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267157+0000", 2024-09-18T00:19:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3833e-05 2024-09-18T00:19:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267300+0000", 2024-09-18T00:19:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142282 2024-09-18T00:19:21.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267447+0000", 2024-09-18T00:19:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147851 2024-09-18T00:19:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308889+0000", 2024-09-18T00:19:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041441779999999998 2024-09-18T00:19:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308956+0000", 2024-09-18T00:19:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7136000000000002e-05 2024-09-18T00:19:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.309053+0000", 2024-09-18T00:19:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6552999999999995e-05 2024-09-18T00:19:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:16947 3.18 3:18b58c7b:::rbd_data.112a6efb7564.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:19:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.943049258000002, 2024-09-18T00:19:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.101692484, 2024-09-18T00:19:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 16947 2024-09-18T00:19:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616843+0000", 2024-09-18T00:19:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0780000000000001e-06 2024-09-18T00:19:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617785+0000", 2024-09-18T00:19:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094204100000000002 2024-09-18T00:19:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617801+0000", 2024-09-18T00:19:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6436999999999999e-05 2024-09-18T00:19:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617809+0000", 2024-09-18T00:19:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9049999999999997e-06 2024-09-18T00:19:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617904+0000", 2024-09-18T00:19:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5204999999999994e-05 2024-09-18T00:19:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.618071+0000", 2024-09-18T00:19:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016640000000000001 2024-09-18T00:19:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.624534+0000", 2024-09-18T00:19:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064634499999999999 2024-09-18T00:19:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718299+0000", 2024-09-18T00:19:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093764739 2024-09-18T00:19:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718375+0000", 2024-09-18T00:19:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6532999999999994e-05 2024-09-18T00:19:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718466+0000", 2024-09-18T00:19:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0581000000000006e-05 2024-09-18T00:19:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718534+0000", 2024-09-18T00:19:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8114999999999994e-05 2024-09-18T00:19:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:16947 3.18 e59/23 ondisk, result = 0)", 2024-09-18T00:19:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.917832564000001, 2024-09-18T00:19:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076488069000000006, 2024-09-18T00:19:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642060+0000", 2024-09-18T00:19:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6640000000000001e-06 2024-09-18T00:19:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642071+0000", 2024-09-18T00:19:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1204e-05 2024-09-18T00:19:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642091+0000", 2024-09-18T00:19:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0313999999999999e-05 2024-09-18T00:19:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642103+0000", 2024-09-18T00:19:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1214e-05 2024-09-18T00:19:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718350+0000", 2024-09-18T00:19:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076247681999999997 2024-09-18T00:19:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718366+0000", 2024-09-18T00:19:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5420000000000001e-05 2024-09-18T00:19:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718546+0000", 2024-09-18T00:19:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000180571 2024-09-18T00:19:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18686 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.611155193999998, 2024-09-18T00:19:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041132573999999998, 2024-09-18T00:19:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18686 2024-09-18T00:19:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948737+0000", 2024-09-18T00:19:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3769999999999999e-06 2024-09-18T00:19:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949968+0000", 2024-09-18T00:19:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001230493 2024-09-18T00:19:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949993+0000", 2024-09-18T00:19:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5282e-05 2024-09-18T00:19:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950011+0000", 2024-09-18T00:19:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8199999999999999e-05 2024-09-18T00:19:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950180+0000", 2024-09-18T00:19:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016867600000000001 2024-09-18T00:19:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950337+0000", 2024-09-18T00:19:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156902 2024-09-18T00:19:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950818+0000", 2024-09-18T00:19:21.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048192599999999998 2024-09-18T00:19:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978491+0000", 2024-09-18T00:19:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027672568000000002 2024-09-18T00:19:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989759+0000", 2024-09-18T00:19:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011268144000000001 2024-09-18T00:19:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989824+0000", 2024-09-18T00:19:21.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4556999999999999e-05 2024-09-18T00:19:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989868+0000", 2024-09-18T00:19:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4449000000000001e-05 2024-09-18T00:19:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18687 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.609827428999999, 2024-09-18T00:19:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040034156000000001, 2024-09-18T00:19:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18687 2024-09-18T00:19:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950065+0000", 2024-09-18T00:19:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5820000000000001e-06 2024-09-18T00:19:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950072+0000", 2024-09-18T00:19:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1029999999999996e-06 2024-09-18T00:19:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950092+0000", 2024-09-18T00:19:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0288e-05 2024-09-18T00:19:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950107+0000", 2024-09-18T00:19:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4290999999999999e-05 2024-09-18T00:19:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975820+0000", 2024-09-18T00:19:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025713573999999999 2024-09-18T00:19:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975980+0000", 2024-09-18T00:19:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016002799999999999 2024-09-18T00:19:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.976299+0000", 2024-09-18T00:19:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031853300000000002 2024-09-18T00:19:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989936+0000", 2024-09-18T00:19:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013637046999999999 2024-09-18T00:19:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990025+0000", 2024-09-18T00:19:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9040000000000001e-05 2024-09-18T00:19:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990064+0000", 2024-09-18T00:19:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9394000000000003e-05 2024-09-18T00:19:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990098+0000", 2024-09-18T00:19:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3275999999999998e-05 2024-09-18T00:19:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18688 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.609726077000001, 2024-09-18T00:19:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040041259000000003, 2024-09-18T00:19:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18688 2024-09-18T00:19:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.607 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950166+0000", 2024-09-18T00:19:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:19:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951295+0000", 2024-09-18T00:19:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011293480000000001 2024-09-18T00:19:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951328+0000", 2024-09-18T00:19:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2524999999999998e-05 2024-09-18T00:19:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951346+0000", 2024-09-18T00:19:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8170999999999999e-05 2024-09-18T00:19:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978546+0000", 2024-09-18T00:19:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027200418000000001 2024-09-18T00:19:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978666+0000", 2024-09-18T00:19:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119913 2024-09-18T00:19:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.979098+0000", 2024-09-18T00:19:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00043169399999999999 2024-09-18T00:19:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:21.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990000+0000", 2024-09-18T00:19:21.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010901719000000001 2024-09-18T00:19:21.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990109+0000", 2024-09-18T00:19:21.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010964299999999999 2024-09-18T00:19:21.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.611 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990152+0000", 2024-09-18T00:19:21.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2877000000000003e-05 2024-09-18T00:19:21.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.612 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.612 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990206+0000", 2024-09-18T00:19:21.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3717000000000001e-05 2024-09-18T00:19:21.612 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.612 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.612 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.612 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.612 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18689 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:21.612 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:21.612 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.608514378999999, 2024-09-18T00:19:21.612 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051022338, 2024-09-18T00:19:21.612 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.613 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.613 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.613 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.613 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.613 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.613 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18689 2024-09-18T00:19:21.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.613 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.613 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:21.613 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.613 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.613 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:21.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951378+0000", 2024-09-18T00:19:21.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:19:21.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.614 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951385+0000", 2024-09-18T00:19:21.614 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-18T00:19:21.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.614 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.614 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951402+0000", 2024-09-18T00:19:21.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6538999999999999e-05 2024-09-18T00:19:21.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951565+0000", 2024-09-18T00:19:21.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016307999999999999 2024-09-18T00:19:21.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.615 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.615 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988232+0000", 2024-09-18T00:19:21.615 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036667656999999999 2024-09-18T00:19:21.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988318+0000", 2024-09-18T00:19:21.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5348999999999999e-05 2024-09-18T00:19:21.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:21.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988574+0000", 2024-09-18T00:19:21.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025670800000000002 2024-09-18T00:19:21.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.616 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:21.616 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002254+0000", 2024-09-18T00:19:21.616 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013679274 2024-09-18T00:19:21.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002326+0000", 2024-09-18T00:19:21.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2720999999999997e-05 2024-09-18T00:19:21.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002364+0000", 2024-09-18T00:19:21.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7314000000000002e-05 2024-09-18T00:19:21.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.617 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.617 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002399+0000", 2024-09-18T00:19:21.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5162e-05 2024-09-18T00:19:21.617 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.617 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.618 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.618 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18690 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:21.618 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:21.618 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.608297360999998, 2024-09-18T00:19:21.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050872134999999999, 2024-09-18T00:19:21.618 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.618 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.618 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.618 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.618 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.618 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.618 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18690 2024-09-18T00:19:21.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.618 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:21.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:21.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951595+0000", 2024-09-18T00:19:21.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4479999999999999e-06 2024-09-18T00:19:21.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952167+0000", 2024-09-18T00:19:21.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00057188400000000004 2024-09-18T00:19:21.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952192+0000", 2024-09-18T00:19:21.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4936999999999999e-05 2024-09-18T00:19:21.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952206+0000", 2024-09-18T00:19:21.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4565e-05 2024-09-18T00:19:21.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990303+0000", 2024-09-18T00:19:21.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038096954000000002 2024-09-18T00:19:21.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990403+0000", 2024-09-18T00:19:21.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9486000000000003e-05 2024-09-18T00:19:21.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:21.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990773+0000", 2024-09-18T00:19:21.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037037799999999998 2024-09-18T00:19:21.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:21.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002316+0000", 2024-09-18T00:19:21.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011542581 2024-09-18T00:19:21.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002407+0000", 2024-09-18T00:19:21.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0968e-05 2024-09-18T00:19:21.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002442+0000", 2024-09-18T00:19:21.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4962000000000002e-05 2024-09-18T00:19:21.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.623 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002466+0000", 2024-09-18T00:19:21.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3972e-05 2024-09-18T00:19:21.623 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.623 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.623 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.623 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18691 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:21.623 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:21.623 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.607664484000001, 2024-09-18T00:19:21.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051495530999999997, 2024-09-18T00:19:21.624 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.624 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.624 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.624 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.624 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.624 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.624 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18691 2024-09-18T00:19:21.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.624 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:21.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:21.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952228+0000", 2024-09-18T00:19:21.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:19:21.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952234+0000", 2024-09-18T00:19:21.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0310000000000004e-06 2024-09-18T00:19:21.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952248+0000", 2024-09-18T00:19:21.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4577e-05 2024-09-18T00:19:21.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952260+0000", 2024-09-18T00:19:21.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1659e-05 2024-09-18T00:19:21.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.000989+0000", 2024-09-18T00:19:21.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048729445000000003 2024-09-18T00:19:21.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001076+0000", 2024-09-18T00:19:21.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6323999999999995e-05 2024-09-18T00:19:21.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:21.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001295+0000", 2024-09-18T00:19:21.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021944299999999999 2024-09-18T00:19:21.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003502+0000", 2024-09-18T00:19:21.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022065209999999999 2024-09-18T00:19:21.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:21.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003581+0000", 2024-09-18T00:19:21.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9404000000000001e-05 2024-09-18T00:19:21.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.628 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003652+0000", 2024-09-18T00:19:21.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0506999999999994e-05 2024-09-18T00:19:21.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003722+0000", 2024-09-18T00:19:21.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0300000000000001e-05 2024-09-18T00:19:21.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.629 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.629 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.629 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18692 3.7 3:e69124d0:::rbd_data.112a6efb7564.0000000000000001:f [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:21.629 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:21.629 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.607616423, 2024-09-18T00:19:21.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050418114999999999, 2024-09-18T00:19:21.630 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.630 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.630 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:21.630 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.630 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.630 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.630 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18692 2024-09-18T00:19:21.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.630 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:21.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:21.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952275+0000", 2024-09-18T00:19:21.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4e-07 2024-09-18T00:19:21.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952281+0000", 2024-09-18T00:19:21.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2000000000000002e-06 2024-09-18T00:19:21.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952295+0000", 2024-09-18T00:19:21.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4731e-05 2024-09-18T00:19:21.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952307+0000", 2024-09-18T00:19:21.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1877e-05 2024-09-18T00:19:21.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002506+0000", 2024-09-18T00:19:21.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050199230999999997 2024-09-18T00:19:21.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002574+0000", 2024-09-18T00:19:21.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8076000000000003e-05 2024-09-18T00:19:21.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002693+0000", 2024-09-18T00:19:21.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011806 2024-09-18T00:19:21.633 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.633 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.633 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.634 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19383 3.5 3:a1859bb0:::rbd_data.112a7fe29ab4.000000000000015b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:21.634 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:21.634 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.937604826000001, 2024-09-18T00:19:21.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056760192000000001, 2024-09-18T00:19:21.634 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.634 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.634 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.634 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.634 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.634 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.634 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19383 2024-09-18T00:19:21.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.634 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622287+0000", 2024-09-18T00:19:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-18T00:19:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622292+0000", 2024-09-18T00:19:21.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6110000000000004e-06 2024-09-18T00:19:21.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622304+0000", 2024-09-18T00:19:21.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2106000000000001e-05 2024-09-18T00:19:21.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622309+0000", 2024-09-18T00:19:21.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.818e-06 2024-09-18T00:19:21.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622360+0000", 2024-09-18T00:19:21.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0430999999999999e-05 2024-09-18T00:19:21.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622438+0000", 2024-09-18T00:19:21.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8362999999999995e-05 2024-09-18T00:19:21.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678811+0000", 2024-09-18T00:19:21.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056372894 2024-09-18T00:19:21.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678906+0000", 2024-09-18T00:19:21.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5078e-05 2024-09-18T00:19:21.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.638 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.638 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.679046+0000", 2024-09-18T00:19:21.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139987 2024-09-18T00:19:21.638 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.638 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.638 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.639 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22019 3.c 3:37b77e94:::rbd_data.112a948d5956.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:21.639 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:19:21.639 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.060629996999999, 2024-09-18T00:19:21.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.039176065000000003, 2024-09-18T00:19:21.639 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.639 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.639 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.639 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.639 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.639 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.639 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22019 2024-09-18T00:19:21.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.639 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:19:21.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:19:21.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499262+0000", 2024-09-18T00:19:21.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:19:21.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499268+0000", 2024-09-18T00:19:21.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8780000000000003e-06 2024-09-18T00:19:21.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499283+0000", 2024-09-18T00:19:21.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5184000000000001e-05 2024-09-18T00:19:21.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499291+0000", 2024-09-18T00:19:21.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5959999999999998e-06 2024-09-18T00:19:21.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499364+0000", 2024-09-18T00:19:21.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2855000000000006e-05 2024-09-18T00:19:21.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499427+0000", 2024-09-18T00:19:21.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3645999999999993e-05 2024-09-18T00:19:21.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538252+0000", 2024-09-18T00:19:21.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038825207 2024-09-18T00:19:21.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538328+0000", 2024-09-18T00:19:21.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5938000000000004e-05 2024-09-18T00:19:21.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538437+0000", 2024-09-18T00:19:21.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108527 2024-09-18T00:19:21.643 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.643 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.643 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.644 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22468 3.5 3:a250f1ef:::rbd_data.112a948d5956.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:21.644 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:21.644 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.788523420000001, 2024-09-18T00:19:21.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.055599787999999997, 2024-09-18T00:19:21.644 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.644 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.644 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.644 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.644 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.644 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.644 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22468 2024-09-18T00:19:21.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.645 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:21.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:21.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771370+0000", 2024-09-18T00:19:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9629999999999998e-06 2024-09-18T00:19:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771385+0000", 2024-09-18T00:19:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5053e-05 2024-09-18T00:19:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771429+0000", 2024-09-18T00:19:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3192999999999998e-05 2024-09-18T00:19:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771453+0000", 2024-09-18T00:19:21.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3995999999999999e-05 2024-09-18T00:19:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771623+0000", 2024-09-18T00:19:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000170846 2024-09-18T00:19:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771838+0000", 2024-09-18T00:19:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021416600000000001 2024-09-18T00:19:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826866+0000", 2024-09-18T00:19:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.05502866 2024-09-18T00:19:21.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826906+0000", 2024-09-18T00:19:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9805999999999998e-05 2024-09-18T00:19:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826967+0000", 2024-09-18T00:19:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1105000000000005e-05 2024-09-18T00:19:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22470 3.2 3:474595b6:::rbd_data.112a948d5956.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:21.648 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 22.732332436, 2024-09-18T00:19:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088321282000000001, 2024-09-18T00:19:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22470 2024-09-18T00:19:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827560+0000", 2024-09-18T00:19:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.762e-06 2024-09-18T00:19:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.650 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827569+0000", 2024-09-18T00:19:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8740000000000001e-06 2024-09-18T00:19:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827585+0000", 2024-09-18T00:19:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5962000000000001e-05 2024-09-18T00:19:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827595+0000", 2024-09-18T00:19:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5910000000000002e-06 2024-09-18T00:19:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827811+0000", 2024-09-18T00:19:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000216858 2024-09-18T00:19:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:19:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827818+0000", 2024-09-18T00:19:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.99e-06 2024-09-18T00:19:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829799+0000", 2024-09-18T00:19:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019803849999999999 2024-09-18T00:19:21.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829884+0000", 2024-09-18T00:19:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4852999999999995e-05 2024-09-18T00:19:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915616+0000", 2024-09-18T00:19:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085732808999999993 2024-09-18T00:19:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:21.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915711+0000", 2024-09-18T00:19:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4578000000000001e-05 2024-09-18T00:19:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915880+0000", 2024-09-18T00:19:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016861999999999999 2024-09-18T00:19:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:42788 3.7 3:e5a8432b:::rbd_data.112a3496573c.0000000000000027:17 [read 309248~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:21.654 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.840920636, 2024-09-18T00:19:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.039621847000000002, 2024-09-18T00:19:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 42788 2024-09-18T00:19:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718971+0000", 2024-09-18T00:19:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2500000000000001e-07 2024-09-18T00:19:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718973+0000", 2024-09-18T00:19:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2510000000000001e-06 2024-09-18T00:19:21.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718979+0000", 2024-09-18T00:19:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1700000000000002e-06 2024-09-18T00:19:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718984+0000", 2024-09-18T00:19:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9110000000000001e-06 2024-09-18T00:19:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.749126+0000", 2024-09-18T00:19:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030142164999999999 2024-09-18T00:19:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.749199+0000", 2024-09-18T00:19:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2985999999999995e-05 2024-09-18T00:19:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.758592+0000", 2024-09-18T00:19:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093929389999999995 2024-09-18T00:19:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.658 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:42820 3.7 3:e2363f9c:::rbd_data.112a3496573c.0000000000000028:17 [read 446464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.719971+0000", 2024-09-18T00:19:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.83992014000000004, 2024-09-18T00:19:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034904399000000003, 2024-09-18T00:19:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 42820 2024-09-18T00:19:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.659 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719971+0000", 2024-09-18T00:19:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719971+0000", 2024-09-18T00:19:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719971+0000", 2024-09-18T00:19:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5999999999999999e-07 2024-09-18T00:19:21.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719973+0000", 2024-09-18T00:19:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.069e-06 2024-09-18T00:19:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719980+0000", 2024-09-18T00:19:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5379999999999999e-06 2024-09-18T00:19:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719985+0000", 2024-09-18T00:19:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1080000000000001e-06 2024-09-18T00:19:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.754731+0000", 2024-09-18T00:19:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034745725999999998 2024-09-18T00:19:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.754804+0000", 2024-09-18T00:19:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3396e-05 2024-09-18T00:19:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:21.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.754875+0000", 2024-09-18T00:19:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1101999999999998e-05 2024-09-18T00:19:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:21.663 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:21.663 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:21.664 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:19:21.665 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-18T00:19:21.665 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:21.661+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:21.765 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:21.761+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:21.801 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:21.801 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:21.801 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:21.801 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:21.815 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:19:21.902 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:21.901+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:21.967 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:21.965+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:21.979 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:21.979 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:21.979 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:21.979 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:21.979 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:21.993 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:19:22.009 INFO:teuthology.orchestra.run.smithi097.stdout:5072 trunc from 0xc9ea624 to 0x6465222 2024-09-18T00:19:22.042 INFO:teuthology.orchestra.run.smithi097.stdout:5073 writesame 0x314d88a thru 0x3158f74 data_size 0x79(0xb6eb bytes) 2024-09-18T00:19:22.067 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:19:22.068 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:22.061+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:22.070 INFO:teuthology.orchestra.run.smithi097.stdout:5074 clone 23 order 21 su 8192 sc 4 2024-09-18T00:19:22.165 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:22.165 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:22.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:22.165 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:22.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.166 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2418 3.1a 3:59a8b40d:::rbd_data.112aa7ba4578.00000000000001b0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:19:22.166 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:22.166 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.425905258, 2024-09-18T00:19:22.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10488562999999999, 2024-09-18T00:19:22.166 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.166 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.166 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:22.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.166 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2418 2024-09-18T00:19:22.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.166 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:22.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:22.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735592+0000", 2024-09-18T00:19:22.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5229999999999998e-06 2024-09-18T00:19:22.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735604+0000", 2024-09-18T00:19:22.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2143e-05 2024-09-18T00:19:22.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735629+0000", 2024-09-18T00:19:22.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.497e-05 2024-09-18T00:19:22.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735648+0000", 2024-09-18T00:19:22.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8896999999999999e-05 2024-09-18T00:19:22.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735775+0000", 2024-09-18T00:19:22.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126935 2024-09-18T00:19:22.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.736118+0000", 2024-09-18T00:19:22.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034330800000000001 2024-09-18T00:19:22.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-18T00:19:22.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.741633+0000", 2024-09-18T00:19:22.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055144139999999996 2024-09-18T00:19:22.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:22.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840134+0000", 2024-09-18T00:19:22.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.098501618999999999 2024-09-18T00:19:22.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:22.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840184+0000", 2024-09-18T00:19:22.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9953000000000001e-05 2024-09-18T00:19:22.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:22.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840336+0000", 2024-09-18T00:19:22.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151476 2024-09-18T00:19:22.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840475+0000", 2024-09-18T00:19:22.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013939200000000001 2024-09-18T00:19:22.171 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.171 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.171 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.171 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2418 3.1a e37/23 ondisk, result = 0)", 2024-09-18T00:19:22.172 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:19:22.172 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.407315414999999, 2024-09-18T00:19:22.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.085979978999999998, 2024-09-18T00:19:22.172 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.172 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.172 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:22.172 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:19:22.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:19:22.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754181+0000", 2024-09-18T00:19:22.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8530000000000001e-06 2024-09-18T00:19:22.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754194+0000", 2024-09-18T00:19:22.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.292e-05 2024-09-18T00:19:22.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754230+0000", 2024-09-18T00:19:22.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5602999999999998e-05 2024-09-18T00:19:22.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754249+0000", 2024-09-18T00:19:22.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8850000000000001e-05 2024-09-18T00:19:22.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840070+0000", 2024-09-18T00:19:22.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.085821123999999999 2024-09-18T00:19:22.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840098+0000", 2024-09-18T00:19:22.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8546000000000001e-05 2024-09-18T00:19:22.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840159+0000", 2024-09-18T00:19:22.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1082999999999998e-05 2024-09-18T00:19:22.175 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.175 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.175 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.175 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12545 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:22.175 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:19:22.176 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.766252653000002, 2024-09-18T00:19:22.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045427642999999997, 2024-09-18T00:19:22.176 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.176 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.176 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:22.176 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.176 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.176 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.176 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12545 2024-09-18T00:19:22.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.176 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:19:22.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:19:22.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395243+0000", 2024-09-18T00:19:22.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.206e-06 2024-09-18T00:19:22.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395249+0000", 2024-09-18T00:19:22.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:19:22.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395270+0000", 2024-09-18T00:19:22.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1223e-05 2024-09-18T00:19:22.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395285+0000", 2024-09-18T00:19:22.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4725999999999999e-05 2024-09-18T00:19:22.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440296+0000", 2024-09-18T00:19:22.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045011402999999998 2024-09-18T00:19:22.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440453+0000", 2024-09-18T00:19:22.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015742299999999999 2024-09-18T00:19:22.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440670+0000", 2024-09-18T00:19:22.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000216302 2024-09-18T00:19:22.179 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.179 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.180 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.180 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12546 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:22.180 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:19:22.180 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.766190444999999, 2024-09-18T00:19:22.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045915171999999997, 2024-09-18T00:19:22.180 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.180 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.180 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:22.180 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.180 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.180 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.180 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12546 2024-09-18T00:19:22.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.181 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:19:22.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:19:22.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395306+0000", 2024-09-18T00:19:22.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.206e-06 2024-09-18T00:19:22.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395330+0000", 2024-09-18T00:19:22.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4686e-05 2024-09-18T00:19:22.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395356+0000", 2024-09-18T00:19:22.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5418999999999998e-05 2024-09-18T00:19:22.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395387+0000", 2024-09-18T00:19:22.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0942000000000003e-05 2024-09-18T00:19:22.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440743+0000", 2024-09-18T00:19:22.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045356592000000001 2024-09-18T00:19:22.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440880+0000", 2024-09-18T00:19:22.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013641700000000001 2024-09-18T00:19:22.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441219+0000", 2024-09-18T00:19:22.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033991000000000002 2024-09-18T00:19:22.184 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.184 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.184 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.184 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12547 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 212992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:22.184 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:19:22.184 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.766081112000002, 2024-09-18T00:19:22.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046163691999999999, 2024-09-18T00:19:22.184 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.184 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.184 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:22.185 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.185 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.185 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.185 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12547 2024-09-18T00:19:22.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.185 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:19:22.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:19:22.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395415+0000", 2024-09-18T00:19:22.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.553e-06 2024-09-18T00:19:22.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395426+0000", 2024-09-18T00:19:22.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0682999999999999e-05 2024-09-18T00:19:22.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395449+0000", 2024-09-18T00:19:22.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3082999999999999e-05 2024-09-18T00:19:22.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395469+0000", 2024-09-18T00:19:22.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0173000000000002e-05 2024-09-18T00:19:22.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441289+0000", 2024-09-18T00:19:22.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045820195000000001 2024-09-18T00:19:22.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441426+0000", 2024-09-18T00:19:22.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013712699999999999 2024-09-18T00:19:22.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441577+0000", 2024-09-18T00:19:22.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015087799999999999 2024-09-18T00:19:22.188 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.188 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.188 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.188 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12548 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:22.188 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:19:22.189 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.766007350999999, 2024-09-18T00:19:22.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046293932000000003, 2024-09-18T00:19:22.189 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.189 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.189 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:22.189 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.189 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.189 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.189 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12548 2024-09-18T00:19:22.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.189 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:19:22.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:19:22.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395489+0000", 2024-09-18T00:19:22.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.22e-06 2024-09-18T00:19:22.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395512+0000", 2024-09-18T00:19:22.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3312e-05 2024-09-18T00:19:22.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395621+0000", 2024-09-18T00:19:22.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010859 2024-09-18T00:19:22.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395689+0000", 2024-09-18T00:19:22.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8232000000000007e-05 2024-09-18T00:19:22.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441610+0000", 2024-09-18T00:19:22.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045920751000000003 2024-09-18T00:19:22.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441695+0000", 2024-09-18T00:19:22.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5379999999999999e-05 2024-09-18T00:19:22.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441781+0000", 2024-09-18T00:19:22.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6446999999999994e-05 2024-09-18T00:19:22.192 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.192 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.192 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.193 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12549 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 294912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:22.193 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:19:22.193 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.765746698999997, 2024-09-18T00:19:22.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046410264999999999, 2024-09-18T00:19:22.193 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.193 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.193 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:22.193 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.193 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.193 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.193 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12549 2024-09-18T00:19:22.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.193 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:19:22.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:19:22.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395750+0000", 2024-09-18T00:19:22.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:19:22.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395758+0000", 2024-09-18T00:19:22.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5259999999999993e-06 2024-09-18T00:19:22.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395793+0000", 2024-09-18T00:19:22.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4459000000000003e-05 2024-09-18T00:19:22.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395809+0000", 2024-09-18T00:19:22.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6433e-05 2024-09-18T00:19:22.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441851+0000", 2024-09-18T00:19:22.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046041493000000003 2024-09-18T00:19:22.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442002+0000", 2024-09-18T00:19:22.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151433 2024-09-18T00:19:22.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442158+0000", 2024-09-18T00:19:22.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000156358 2024-09-18T00:19:22.197 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.197 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.197 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.197 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12550 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:22.197 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:19:22.197 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.765635209999999, 2024-09-18T00:19:22.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046498398000000003, 2024-09-18T00:19:22.197 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.197 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.197 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:22.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.198 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12550 2024-09-18T00:19:22.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.198 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:19:22.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:19:22.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395861+0000", 2024-09-18T00:19:22.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.234e-06 2024-09-18T00:19:22.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395872+0000", 2024-09-18T00:19:22.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1409e-05 2024-09-18T00:19:22.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395892+0000", 2024-09-18T00:19:22.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9680000000000001e-05 2024-09-18T00:19:22.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395911+0000", 2024-09-18T00:19:22.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9283000000000002e-05 2024-09-18T00:19:22.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442210+0000", 2024-09-18T00:19:22.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046298372999999997 2024-09-18T00:19:22.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442279+0000", 2024-09-18T00:19:22.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9055999999999997e-05 2024-09-18T00:19:22.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442358+0000", 2024-09-18T00:19:22.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9363000000000006e-05 2024-09-18T00:19:22.201 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.201 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.201 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.201 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12551 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:22.201 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:19:22.201 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.764962758000003, 2024-09-18T00:19:22.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046066902999999999, 2024-09-18T00:19:22.201 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.202 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.202 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:22.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.202 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12551 2024-09-18T00:19:22.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.202 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:19:22.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:19:22.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396534+0000", 2024-09-18T00:19:22.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6869999999999999e-06 2024-09-18T00:19:22.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396543+0000", 2024-09-18T00:19:22.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3430000000000002e-06 2024-09-18T00:19:22.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396563+0000", 2024-09-18T00:19:22.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0063000000000001e-05 2024-09-18T00:19:22.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396579+0000", 2024-09-18T00:19:22.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5610000000000001e-05 2024-09-18T00:19:22.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442434+0000", 2024-09-18T00:19:22.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045855094999999998 2024-09-18T00:19:22.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442515+0000", 2024-09-18T00:19:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0988000000000005e-05 2024-09-18T00:19:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442599+0000", 2024-09-18T00:19:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4116999999999994e-05 2024-09-18T00:19:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.205 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12552 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:22.206 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:19:22.206 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.764887956000003, 2024-09-18T00:19:22.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046210804000000001, 2024-09-18T00:19:22.206 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.206 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.206 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:22.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.206 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12552 2024-09-18T00:19:22.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.206 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:19:22.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:19:22.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396608+0000", 2024-09-18T00:19:22.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4729999999999999e-06 2024-09-18T00:19:22.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396614+0000", 2024-09-18T00:19:22.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1e-06 2024-09-18T00:19:22.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396633+0000", 2024-09-18T00:19:22.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9009e-05 2024-09-18T00:19:22.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396650+0000", 2024-09-18T00:19:22.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7045999999999999e-05 2024-09-18T00:19:22.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442662+0000", 2024-09-18T00:19:22.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046011170999999997 2024-09-18T00:19:22.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442737+0000", 2024-09-18T00:19:22.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5759000000000004e-05 2024-09-18T00:19:22.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442818+0000", 2024-09-18T00:19:22.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0246000000000004e-05 2024-09-18T00:19:22.210 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.210 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.210 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.210 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12553 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:22.210 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:19:22.210 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.764825252000001, 2024-09-18T00:19:22.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046339454000000002, 2024-09-18T00:19:22.211 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.211 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.211 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:22.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.211 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12553 2024-09-18T00:19:22.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.211 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:19:22.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:19:22.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396671+0000", 2024-09-18T00:19:22.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.314e-06 2024-09-18T00:19:22.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396676+0000", 2024-09-18T00:19:22.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6659999999999996e-06 2024-09-18T00:19:22.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396695+0000", 2024-09-18T00:19:22.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8383e-05 2024-09-18T00:19:22.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396709+0000", 2024-09-18T00:19:22.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4309e-05 2024-09-18T00:19:22.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442871+0000", 2024-09-18T00:19:22.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046162295999999998 2024-09-18T00:19:22.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442939+0000", 2024-09-18T00:19:22.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7607999999999994e-05 2024-09-18T00:19:22.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443009+0000", 2024-09-18T00:19:22.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9877999999999997e-05 2024-09-18T00:19:22.214 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.214 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.214 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.215 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12554 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:22.215 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:19:22.215 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.764767329999998, 2024-09-18T00:19:22.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046504050999999998, 2024-09-18T00:19:22.215 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.215 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.215 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:22.215 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.215 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.215 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.215 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12554 2024-09-18T00:19:22.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.215 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:19:22.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:19:22.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396729+0000", 2024-09-18T00:19:22.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2559999999999999e-06 2024-09-18T00:19:22.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396736+0000", 2024-09-18T00:19:22.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6140000000000003e-06 2024-09-18T00:19:22.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396755+0000", 2024-09-18T00:19:22.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8221999999999999e-05 2024-09-18T00:19:22.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396769+0000", 2024-09-18T00:19:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4372999999999999e-05 2024-09-18T00:19:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443078+0000", 2024-09-18T00:19:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046308850999999998 2024-09-18T00:19:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443152+0000", 2024-09-18T00:19:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4083000000000002e-05 2024-09-18T00:19:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443231+0000", 2024-09-18T00:19:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9652000000000002e-05 2024-09-18T00:19:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12632 3.3 3:c21fac02:::rbd_data.112a7b1589a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1490944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:19:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.758209045000001, 2024-09-18T00:19:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045843916999999998, 2024-09-18T00:19:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12632 2024-09-18T00:19:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:19:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:19:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403288+0000", 2024-09-18T00:19:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3630000000000001e-06 2024-09-18T00:19:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403886+0000", 2024-09-18T00:19:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00059840100000000003 2024-09-18T00:19:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403913+0000", 2024-09-18T00:19:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6376e-05 2024-09-18T00:19:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403927+0000", 2024-09-18T00:19:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4188e-05 2024-09-18T00:19:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443268+0000", 2024-09-18T00:19:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.039340591000000001 2024-09-18T00:19:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443488+0000", 2024-09-18T00:19:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000220545 2024-09-18T00:19:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443918+0000", 2024-09-18T00:19:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043021800000000001 2024-09-18T00:19:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.448558+0000", 2024-09-18T00:19:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046391449999999999 2024-09-18T00:19:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449015+0000", 2024-09-18T00:19:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045766600000000002 2024-09-18T00:19:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449085+0000", 2024-09-18T00:19:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0191999999999995e-05 2024-09-18T00:19:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449130+0000", 2024-09-18T00:19:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4231999999999999e-05 2024-09-18T00:19:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:19386 3.1d 3:bcd89fc1:::rbd_data.112a7fe29ab4.00000000000000c8:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:53.699994+0000", 2024-09-18T00:19:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.461500903000001, 2024-09-18T00:19:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050613197999999998, 2024-09-18T00:19:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 19386 2024-09-18T00:19:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.699994+0000", 2024-09-18T00:19:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.699994+0000", 2024-09-18T00:19:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.699996+0000", 2024-09-18T00:19:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2330000000000001e-06 2024-09-18T00:19:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700007+0000", 2024-09-18T00:19:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1199000000000001e-05 2024-09-18T00:19:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700039+0000", 2024-09-18T00:19:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1362999999999998e-05 2024-09-18T00:19:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700054+0000", 2024-09-18T00:19:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4970000000000001e-05 2024-09-18T00:19:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700229+0000", 2024-09-18T00:19:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017526600000000001 2024-09-18T00:19:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700488+0000", 2024-09-18T00:19:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025918099999999998 2024-09-18T00:19:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750393+0000", 2024-09-18T00:19:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049904733 2024-09-18T00:19:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750476+0000", 2024-09-18T00:19:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3413999999999998e-05 2024-09-18T00:19:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750607+0000", 2024-09-18T00:19:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130839 2024-09-18T00:19:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:19387 3.1d 3:bc19744b:::rbd_data.112a7fe29ab4.00000000000000c9:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:53.700113+0000", 2024-09-18T00:19:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.461381493000001, 2024-09-18T00:19:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050693338999999997, 2024-09-18T00:19:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 19387 2024-09-18T00:19:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700113+0000", 2024-09-18T00:19:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700113+0000", 2024-09-18T00:19:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700115+0000", 2024-09-18T00:19:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3599999999999999e-06 2024-09-18T00:19:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700130+0000", 2024-09-18T00:19:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5396999999999998e-05 2024-09-18T00:19:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700147+0000", 2024-09-18T00:19:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7255e-05 2024-09-18T00:19:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700158+0000", 2024-09-18T00:19:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.079e-05 2024-09-18T00:19:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.707161+0000", 2024-09-18T00:19:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070025820000000003 2024-09-18T00:19:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.707407+0000", 2024-09-18T00:19:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024667100000000001 2024-09-18T00:19:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750638+0000", 2024-09-18T00:19:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043230365 2024-09-18T00:19:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750704+0000", 2024-09-18T00:19:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6806999999999999e-05 2024-09-18T00:19:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750807+0000", 2024-09-18T00:19:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102112 2024-09-18T00:19:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:22270 3.14 3:2badd31c:::rbd_data.112a948d5956.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.908651+0000", 2024-09-18T00:19:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.252843786, 2024-09-18T00:19:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.081875197999999996, 2024-09-18T00:19:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 22270 2024-09-18T00:19:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908651+0000", 2024-09-18T00:19:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908651+0000", 2024-09-18T00:19:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908653+0000", 2024-09-18T00:19:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2290000000000002e-06 2024-09-18T00:19:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908665+0000", 2024-09-18T00:19:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.225e-05 2024-09-18T00:19:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908697+0000", 2024-09-18T00:19:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1306000000000001e-05 2024-09-18T00:19:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908712+0000", 2024-09-18T00:19:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5466000000000001e-05 2024-09-18T00:19:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908889+0000", 2024-09-18T00:19:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017653999999999999 2024-09-18T00:19:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.909023+0000", 2024-09-18T00:19:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013473700000000001 2024-09-18T00:19:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.913637+0000", 2024-09-18T00:19:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046132600000000001 2024-09-18T00:19:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990060+0000", 2024-09-18T00:19:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.076422834999999995 2024-09-18T00:19:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990206+0000", 2024-09-18T00:19:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014591499999999999 2024-09-18T00:19:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990391+0000", 2024-09-18T00:19:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018554300000000001 2024-09-18T00:19:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990526+0000", 2024-09-18T00:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000135117 2024-09-18T00:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:22270 3.14 e64/32 ondisk, result = 0)", 2024-09-18T00:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.926819+0000", 2024-09-18T00:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.234675522, 2024-09-18T00:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.063725764000000004, 2024-09-18T00:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926819+0000", 2024-09-18T00:19:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926819+0000", 2024-09-18T00:19:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926821+0000", 2024-09-18T00:19:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5540000000000001e-06 2024-09-18T00:19:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926828+0000", 2024-09-18T00:19:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9260000000000003e-06 2024-09-18T00:19:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926857+0000", 2024-09-18T00:19:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9357e-05 2024-09-18T00:19:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926878+0000", 2024-09-18T00:19:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0571999999999999e-05 2024-09-18T00:19:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990155+0000", 2024-09-18T00:19:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.063277618999999993 2024-09-18T00:19:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990187+0000", 2024-09-18T00:19:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1322999999999998e-05 2024-09-18T00:19:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990545+0000", 2024-09-18T00:19:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00035841300000000003 2024-09-18T00:19:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:22272 3.3 3:c617f123:::rbd_data.112a948d5956.0000000000000009:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.170397058999999, 2024-09-18T00:19:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10752577300000001, 2024-09-18T00:19:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 22272 2024-09-18T00:19:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991102+0000", 2024-09-18T00:19:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9369999999999997e-06 2024-09-18T00:19:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991118+0000", 2024-09-18T00:19:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6396e-05 2024-09-18T00:19:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991150+0000", 2024-09-18T00:19:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2132999999999999e-05 2024-09-18T00:19:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991170+0000", 2024-09-18T00:19:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0085000000000002e-05 2024-09-18T00:19:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991370+0000", 2024-09-18T00:19:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019963700000000001 2024-09-18T00:19:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991518+0000", 2024-09-18T00:19:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000148424 2024-09-18T00:19:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.998850+0000", 2024-09-18T00:19:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073315480000000002 2024-09-18T00:19:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098178+0000", 2024-09-18T00:19:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.099328067000000006 2024-09-18T00:19:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098314+0000", 2024-09-18T00:19:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000135678 2024-09-18T00:19:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098498+0000", 2024-09-18T00:19:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018479600000000001 2024-09-18T00:19:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098623+0000", 2024-09-18T00:19:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000125072 2024-09-18T00:19:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:22272 3.3 e64/23 ondisk, result = 0)", 2024-09-18T00:19:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:58.023226+0000", 2024-09-18T00:19:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.138268763999999, 2024-09-18T00:19:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.075416677000000001, 2024-09-18T00:19:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023226+0000", 2024-09-18T00:19:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023226+0000", 2024-09-18T00:19:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023228+0000", 2024-09-18T00:19:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0600000000000002e-06 2024-09-18T00:19:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023237+0000", 2024-09-18T00:19:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6430000000000001e-06 2024-09-18T00:19:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023271+0000", 2024-09-18T00:19:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4091999999999998e-05 2024-09-18T00:19:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023293+0000", 2024-09-18T00:19:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2103000000000002e-05 2024-09-18T00:19:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098272+0000", 2024-09-18T00:19:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.074979263000000004 2024-09-18T00:19:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098297+0000", 2024-09-18T00:19:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5162999999999999e-05 2024-09-18T00:19:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098643+0000", 2024-09-18T00:19:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034535300000000001 2024-09-18T00:19:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:23689 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-18T00:19:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.309072280999999, 2024-09-18T00:19:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9562625809999998, 2024-09-18T00:19:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 23689 2024-09-18T00:19:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852423+0000", 2024-09-18T00:19:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9999999999999995e-07 2024-09-18T00:19:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852429+0000", 2024-09-18T00:19:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3279999999999996e-06 2024-09-18T00:19:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852442+0000", 2024-09-18T00:19:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3052e-05 2024-09-18T00:19:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852448+0000", 2024-09-18T00:19:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2779999999999999e-06 2024-09-18T00:19:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852523+0000", 2024-09-18T00:19:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4463999999999997e-05 2024-09-18T00:19:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852538+0000", 2024-09-18T00:19:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5466999999999999e-05 2024-09-18T00:19:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793478+0000", 2024-09-18T00:19:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.94094012699999996 2024-09-18T00:19:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793494+0000", 2024-09-18T00:19:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5807999999999998e-05 2024-09-18T00:19:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.804875+0000", 2024-09-18T00:19:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0113811939999999 2024-09-18T00:19:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805359+0000", 2024-09-18T00:19:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000484248 2024-09-18T00:19:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-18T00:19:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805872+0000", 2024-09-18T00:19:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00051218100000000001 2024-09-18T00:19:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.807790+0000", 2024-09-18T00:19:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001918763 2024-09-18T00:19:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808242+0000", 2024-09-18T00:19:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045126200000000002 2024-09-18T00:19:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808428+0000", 2024-09-18T00:19:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018633000000000001 2024-09-18T00:19:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808685+0000", 2024-09-18T00:19:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025707900000000001 2024-09-18T00:19:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:22.262 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:22.263 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:22.264 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:22.264 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:22.264 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:22.265 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:19:22.265 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:22.265 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:22.266 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:22.266 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:22.257+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:22.266 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:22.266 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:22.287 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:22.287 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:22.305 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:22.305 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:22.323 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:22.325 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7] out_osds: [6] dead_osds: [1] live_osds: [0, 2, 3, 4, 6, 7, 5] 2024-09-18T00:19:22.325 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:19:22.325 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 4, 5, 7] 2024-09-18T00:19:22.325 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-18T00:19:22.327 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:22.328 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:22.365 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:22.361+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:22.375 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is unset 2024-09-18T00:19:22.395 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:22.395 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:22.410 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:19:22.461 INFO:teuthology.orchestra.run.smithi097.stdout:truncating image image_client.0-clone22 from 0x6465222 (overlap 0x6465222) to 0x14143a0 2024-09-18T00:19:22.502 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:22.501+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:22.566 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:22.561+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:22.583 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:22.583 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:22.597 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:19:22.638 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-18T00:19:22.645 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-18T00:19:22.641+0000 7f63a6107640 -1 mon.a@0(leader).osd e83 definitely_dead 0 2024-09-18T00:19:22.682 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:22.681+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:22.761 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:22.761 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:22.767 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:22.761+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:22.776 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:19:22.868 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:22.861+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:22.940 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:22.940 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:22.954 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:19:22.969 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:22.965+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:23.070 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:23.069+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:23.075 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:23.075 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:23.089 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:19:23.099 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:23.099 INFO:teuthology.orchestra.run.smithi097.stdout:5077 read 0x18f2c62 thru 0x18f91bf (0x655e bytes) 2024-09-18T00:19:23.110 INFO:teuthology.orchestra.run.smithi097.stdout:5078 trunc from 0x6465222 to 0xc6818dc 2024-09-18T00:19:23.126 INFO:teuthology.orchestra.run.smithi097.stdout:5081 punch from 0xc37aa50 to 0xc38493a, (0x9eea bytes) 2024-09-18T00:19:23.126 INFO:teuthology.orchestra.run.smithi097.stdout:5082 read 0x5ceb523 thru 0x5cf4ea9 (0x9987 bytes) 2024-09-18T00:19:23.127 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:23.127 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:23.127 INFO:teuthology.orchestra.run.smithi097.stdout:5085 read 0x2a7b154 thru 0x2a87469 (0xc316 bytes) 2024-09-18T00:19:23.129 INFO:teuthology.orchestra.run.smithi097.stdout:5086 writesame 0xc6093ee thru 0xc6176d0 data_size 0x13(0xe2e3 bytes) 2024-09-18T00:19:23.139 INFO:teuthology.orchestra.run.smithi097.stdout:5087 read 0xacc8952 thru 0xacd5563 (0xcc12 bytes) 2024-09-18T00:19:23.139 INFO:teuthology.orchestra.run.smithi097.stdout:5088 punch from 0x673a478 to 0x673e6bd, (0x4245 bytes) 2024-09-18T00:19:23.140 INFO:teuthology.orchestra.run.smithi097.stdout:5089 writesame 0x38e409 thru 0x391995 data_size 0x1(0x358d bytes) 2024-09-18T00:19:23.174 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:23.173+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:23.251 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:23.251 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:23.267 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:19:23.310 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:23.309+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:23.373 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:23.369+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:23.387 INFO:teuthology.orchestra.run.smithi097.stderr:marked out osd.1. 2024-09-18T00:19:23.431 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:23.432 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:23.445 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:19:23.474 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:23.469+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:23.575 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:23.573+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:23.613 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:23.613 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:23.627 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:19:23.676 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:23.669+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:23.777 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:23.773+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:23.788 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:23.788 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:23.802 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:19:23.878 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:23.877+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:23.934 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:23.934 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:23.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:19:23.997 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:23.993+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:24.075 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:24.076 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:24.080 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:24.077+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:24.090 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:24.181 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:24.177+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:24.254 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:24.254 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:24.268 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:19:24.282 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:24.281+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:24.329 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:19:24.383 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:24.381+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:24.387 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:24.387 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:24.402 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:19:24.408 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:19:24.475 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:24.476 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:24.476 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:24.483 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:24.481+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:24.493 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:24.493 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:24.509 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:24.510 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:24.532 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:24.532 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:24.553 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:24.553 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:24.566 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:24.566 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:24.566 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:24.566 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:24.566 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:24.573 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:24.573 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:24.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:19:24.584 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:24.581+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:24.593 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:24.594 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:24.597 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:24.597 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:24.685 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:24.681+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:24.707 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:24.708 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:24.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:24.708 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:24.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.708 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:5722 3.4 e41/23 3:20381d86:::rbd_data.112a3a39fc45.000000000000001d:head v 41'107, mlcod=40'105)", 2024-09-18T00:19:24.708 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:24.708 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.370315910000002, 2024-09-18T00:19:24.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01185172, 2024-09-18T00:19:24.708 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.708 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.708 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:24.708 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:24.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:24.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335497+0000", 2024-09-18T00:19:24.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.505e-06 2024-09-18T00:19:24.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335945+0000", 2024-09-18T00:19:24.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000447687 2024-09-18T00:19:24.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335961+0000", 2024-09-18T00:19:24.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6430999999999999e-05 2024-09-18T00:19:24.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335969+0000", 2024-09-18T00:19:24.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:19:24.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336030+0000", 2024-09-18T00:19:24.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1255999999999998e-05 2024-09-18T00:19:24.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.711 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336061+0000", 2024-09-18T00:19:24.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0579e-05 2024-09-18T00:19:24.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.711 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:24.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347296+0000", 2024-09-18T00:19:24.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011235663 2024-09-18T00:19:24.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.712 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.712 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347347+0000", 2024-09-18T00:19:24.712 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0914000000000001e-05 2024-09-18T00:19:24.712 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.712 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.712 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.712 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11457 3.0 3:0214e7b3:::rbd_data.112ab1fa102a.0000000000000150:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:24.713 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:24.713 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.445949972000001, 2024-09-18T00:19:24.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028499165, 2024-09-18T00:19:24.713 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.713 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.713 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:24.713 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.713 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.713 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.713 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11457 2024-09-18T00:19:24.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.713 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:24.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:24.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259865+0000", 2024-09-18T00:19:24.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7230000000000002e-06 2024-09-18T00:19:24.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259883+0000", 2024-09-18T00:19:24.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8067000000000002e-05 2024-09-18T00:19:24.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259924+0000", 2024-09-18T00:19:24.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0550999999999999e-05 2024-09-18T00:19:24.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259943+0000", 2024-09-18T00:19:24.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:19:24.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260054+0000", 2024-09-18T00:19:24.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011075 2024-09-18T00:19:24.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260138+0000", 2024-09-18T00:19:24.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4549000000000007e-05 2024-09-18T00:19:24.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:24.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.287926+0000", 2024-09-18T00:19:24.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027787642000000001 2024-09-18T00:19:24.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:24.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288090+0000", 2024-09-18T00:19:24.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016402499999999999 2024-09-18T00:19:24.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288361+0000", 2024-09-18T00:19:24.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027097800000000001 2024-09-18T00:19:24.718 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.718 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.718 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.718 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18661 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:24.718 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:24.719 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.771182107000001, 2024-09-18T00:19:24.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020008037999999999, 2024-09-18T00:19:24.719 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.719 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.719 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:24.719 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.719 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.719 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.719 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18661 2024-09-18T00:19:24.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.719 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:24.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:24.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934633+0000", 2024-09-18T00:19:24.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8e-06 2024-09-18T00:19:24.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935511+0000", 2024-09-18T00:19:24.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087785700000000001 2024-09-18T00:19:24.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935561+0000", 2024-09-18T00:19:24.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9472000000000003e-05 2024-09-18T00:19:24.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935586+0000", 2024-09-18T00:19:24.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5616e-05 2024-09-18T00:19:24.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935838+0000", 2024-09-18T00:19:24.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025202699999999999 2024-09-18T00:19:24.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.936023+0000", 2024-09-18T00:19:24.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018480300000000001 2024-09-18T00:19:24.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:24.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953545+0000", 2024-09-18T00:19:24.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017522157 2024-09-18T00:19:24.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:24.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953723+0000", 2024-09-18T00:19:24.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017753800000000001 2024-09-18T00:19:24.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954637+0000", 2024-09-18T00:19:24.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091476800000000005 2024-09-18T00:19:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18662 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.77017717, 2024-09-18T00:19:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019388843999999999, 2024-09-18T00:19:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.724 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18662 2024-09-18T00:19:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935636+0000", 2024-09-18T00:19:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.663e-06 2024-09-18T00:19:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935646+0000", 2024-09-18T00:19:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000006e-06 2024-09-18T00:19:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935672+0000", 2024-09-18T00:19:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5868999999999999e-05 2024-09-18T00:19:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935692+0000", 2024-09-18T00:19:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0604000000000001e-05 2024-09-18T00:19:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950738+0000", 2024-09-18T00:19:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015045868 2024-09-18T00:19:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950992+0000", 2024-09-18T00:19:24.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025368400000000001 2024-09-18T00:19:24.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:24.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954674+0000", 2024-09-18T00:19:24.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003682154 2024-09-18T00:19:24.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.728 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:24.728 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954806+0000", 2024-09-18T00:19:24.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013252000000000001 2024-09-18T00:19:24.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.955023+0000", 2024-09-18T00:19:24.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021681200000000001 2024-09-18T00:19:24.729 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.729 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.729 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.729 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19382 3.4 3:22f09d3a:::rbd_data.112a7fe29ab4.000000000000015a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:24.729 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:24.729 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.083767120000001, 2024-09-18T00:19:24.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075119302999999998, 2024-09-18T00:19:24.730 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.730 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.730 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:24.730 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.730 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.730 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.730 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19382 2024-09-18T00:19:24.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.730 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:24.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:24.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622045+0000", 2024-09-18T00:19:24.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0020000000000001e-06 2024-09-18T00:19:24.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622053+0000", 2024-09-18T00:19:24.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2679999999999997e-06 2024-09-18T00:19:24.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622062+0000", 2024-09-18T00:19:24.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3749999999999992e-06 2024-09-18T00:19:24.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622068+0000", 2024-09-18T00:19:24.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5729999999999998e-06 2024-09-18T00:19:24.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622118+0000", 2024-09-18T00:19:24.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0696999999999999e-05 2024-09-18T00:19:24.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622222+0000", 2024-09-18T00:19:24.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103286 2024-09-18T00:19:24.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:24.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697061+0000", 2024-09-18T00:19:24.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074839141999999997 2024-09-18T00:19:24.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:24.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697102+0000", 2024-09-18T00:19:24.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1513e-05 2024-09-18T00:19:24.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697164+0000", 2024-09-18T00:19:24.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1446999999999996e-05 2024-09-18T00:19:24.734 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.734 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.734 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.735 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20245 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:24.735 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:24.735 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 30.550108791, 2024-09-18T00:19:24.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099981159999999996, 2024-09-18T00:19:24.735 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.735 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.735 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:24.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.735 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.735 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20245 2024-09-18T00:19:24.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.735 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:24.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:24.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:24.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.65e-07 2024-09-18T00:19:24.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155711+0000", 2024-09-18T00:19:24.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:19:24.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155720+0000", 2024-09-18T00:19:24.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6580000000000008e-06 2024-09-18T00:19:24.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155725+0000", 2024-09-18T00:19:24.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3430000000000003e-06 2024-09-18T00:19:24.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165479+0000", 2024-09-18T00:19:24.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097538239999999995 2024-09-18T00:19:24.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165556+0000", 2024-09-18T00:19:24.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6724000000000006e-05 2024-09-18T00:19:24.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165701+0000", 2024-09-18T00:19:24.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145129 2024-09-18T00:19:24.739 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.739 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.739 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.739 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20246 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:24.739 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:24.739 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 30.550078928000001, 2024-09-18T00:19:24.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01018784, 2024-09-18T00:19:24.739 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.739 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.739 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:24.740 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.740 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.740 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.740 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20246 2024-09-18T00:19:24.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.740 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:24.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:24.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:24.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2599999999999998e-07 2024-09-18T00:19:24.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155735+0000", 2024-09-18T00:19:24.741 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2699999999999999e-06 2024-09-18T00:19:24.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155744+0000", 2024-09-18T00:19:24.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8629999999999997e-06 2024-09-18T00:19:24.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155749+0000", 2024-09-18T00:19:24.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0359999999999997e-06 2024-09-18T00:19:24.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165748+0000", 2024-09-18T00:19:24.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099991449999999992 2024-09-18T00:19:24.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165819+0000", 2024-09-18T00:19:24.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0964999999999996e-05 2024-09-18T00:19:24.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165920+0000", 2024-09-18T00:19:24.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101135 2024-09-18T00:19:24.743 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.743 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.743 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.744 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20247 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:24.744 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:24.744 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 30.550055585999999, 2024-09-18T00:19:24.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010443987, 2024-09-18T00:19:24.744 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.744 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.744 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:24.744 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.744 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.744 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.744 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20247 2024-09-18T00:19:24.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.744 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:24.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:24.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:24.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1899999999999998e-07 2024-09-18T00:19:24.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155760+0000", 2024-09-18T00:19:24.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6459999999999999e-06 2024-09-18T00:19:24.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155767+0000", 2024-09-18T00:19:24.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0350000000000004e-06 2024-09-18T00:19:24.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155774+0000", 2024-09-18T00:19:24.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:19:24.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165960+0000", 2024-09-18T00:19:24.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186051999999999 2024-09-18T00:19:24.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166048+0000", 2024-09-18T00:19:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8848e-05 2024-09-18T00:19:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166200+0000", 2024-09-18T00:19:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015147000000000001 2024-09-18T00:19:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20248 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 30.550030421999999, 2024-09-18T00:19:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010617721, 2024-09-18T00:19:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20248 2024-09-18T00:19:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:24.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:24.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155782+0000", 2024-09-18T00:19:24.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.84e-07 2024-09-18T00:19:24.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155784+0000", 2024-09-18T00:19:24.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2800000000000002e-06 2024-09-18T00:19:24.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155790+0000", 2024-09-18T00:19:24.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5679999999999996e-06 2024-09-18T00:19:24.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155795+0000", 2024-09-18T00:19:24.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6929999999999998e-06 2024-09-18T00:19:24.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166245+0000", 2024-09-18T00:19:24.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010449847999999999 2024-09-18T00:19:24.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166307+0000", 2024-09-18T00:19:24.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2287999999999997e-05 2024-09-18T00:19:24.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166399+0000", 2024-09-18T00:19:24.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.166e-05 2024-09-18T00:19:24.753 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.753 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.753 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.753 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20249 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:24.753 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:24.754 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 30.550010046000001, 2024-09-18T00:19:24.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010811983000000001, 2024-09-18T00:19:24.754 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.754 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.754 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:24.754 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.754 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.754 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.754 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20249 2024-09-18T00:19:24.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.754 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:24.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:24.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155802+0000", 2024-09-18T00:19:24.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:19:24.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155804+0000", 2024-09-18T00:19:24.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:19:24.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155810+0000", 2024-09-18T00:19:24.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0580000000000003e-06 2024-09-18T00:19:24.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155817+0000", 2024-09-18T00:19:24.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3980000000000004e-06 2024-09-18T00:19:24.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166451+0000", 2024-09-18T00:19:24.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010634825000000001 2024-09-18T00:19:24.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166520+0000", 2024-09-18T00:19:24.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8317000000000003e-05 2024-09-18T00:19:24.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166613+0000", 2024-09-18T00:19:24.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3801000000000005e-05 2024-09-18T00:19:24.758 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.758 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.758 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.758 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:21913 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:24.758 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:24.758 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.412164900000001, 2024-09-18T00:19:24.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090508916999999994, 2024-09-18T00:19:24.758 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.758 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.758 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:24.758 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.759 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.759 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.759 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 21913 2024-09-18T00:19:24.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.759 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:24.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:24.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293648+0000", 2024-09-18T00:19:24.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6400000000000001e-07 2024-09-18T00:19:24.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293653+0000", 2024-09-18T00:19:24.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3820000000000003e-06 2024-09-18T00:19:24.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293665+0000", 2024-09-18T00:19:24.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1778e-05 2024-09-18T00:19:24.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293673+0000", 2024-09-18T00:19:24.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4130000000000006e-06 2024-09-18T00:19:24.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293719+0000", 2024-09-18T00:19:24.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5534999999999997e-05 2024-09-18T00:19:24.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293773+0000", 2024-09-18T00:19:24.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4389999999999999e-05 2024-09-18T00:19:24.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:24.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384053+0000", 2024-09-18T00:19:24.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090279617000000006 2024-09-18T00:19:24.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:24.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384088+0000", 2024-09-18T00:19:24.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5466000000000002e-05 2024-09-18T00:19:24.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384156+0000", 2024-09-18T00:19:24.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7471999999999994e-05 2024-09-18T00:19:24.763 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.763 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.763 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.763 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22274 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:24.763 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:24.764 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.601048680000002, 2024-09-18T00:19:24.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.069820742000000005, 2024-09-18T00:19:24.764 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.764 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.764 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:24.764 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.764 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.764 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.764 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22274 2024-09-18T00:19:24.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.764 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:24.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:24.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104766+0000", 2024-09-18T00:19:24.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3859999999999999e-06 2024-09-18T00:19:24.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104783+0000", 2024-09-18T00:19:24.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6697e-05 2024-09-18T00:19:24.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104816+0000", 2024-09-18T00:19:24.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2786000000000001e-05 2024-09-18T00:19:24.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.766 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.766 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104833+0000", 2024-09-18T00:19:24.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7125999999999998e-05 2024-09-18T00:19:24.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104965+0000", 2024-09-18T00:19:24.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013215899999999999 2024-09-18T00:19:24.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.105103+0000", 2024-09-18T00:19:24.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138193 2024-09-18T00:19:24.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:24.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174496+0000", 2024-09-18T00:19:24.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.06939244 2024-09-18T00:19:24.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:24.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174530+0000", 2024-09-18T00:19:24.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4601999999999998e-05 2024-09-18T00:19:24.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174584+0000", 2024-09-18T00:19:24.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3353000000000002e-05 2024-09-18T00:19:24.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.768 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.769 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22485 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write 16521958~30847] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:24.769 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:24.769 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 25.752259670000001, 2024-09-18T00:19:24.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012574648000000001, 2024-09-18T00:19:24.769 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.769 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.769 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:24.769 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.769 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.769 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.769 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22485 2024-09-18T00:19:24.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.770 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:24.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:24.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953554+0000", 2024-09-18T00:19:24.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1150000000000001e-06 2024-09-18T00:19:24.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962036+0000", 2024-09-18T00:19:24.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0084817599999999996 2024-09-18T00:19:24.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962062+0000", 2024-09-18T00:19:24.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6499000000000002e-05 2024-09-18T00:19:24.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962074+0000", 2024-09-18T00:19:24.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2095e-05 2024-09-18T00:19:24.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962165+0000", 2024-09-18T00:19:24.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0703000000000002e-05 2024-09-18T00:19:24.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962276+0000", 2024-09-18T00:19:24.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111406 2024-09-18T00:19:24.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:24.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965925+0000", 2024-09-18T00:19:24.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036488919999999999 2024-09-18T00:19:24.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:24.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965992+0000", 2024-09-18T00:19:24.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6806999999999999e-05 2024-09-18T00:19:24.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.966127+0000", 2024-09-18T00:19:24.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013437099999999999 2024-09-18T00:19:24.773 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.774 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.774 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.774 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22681 3.4 3:22e76021:::rbd_data.112a948d5956.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:24.774 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:24.774 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 25.418260364999998, 2024-09-18T00:19:24.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076591189000000004, 2024-09-18T00:19:24.774 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.774 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.774 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:24.774 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.774 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.774 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.775 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22681 2024-09-18T00:19:24.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.775 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:24.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:24.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287554+0000", 2024-09-18T00:19:24.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.514e-06 2024-09-18T00:19:24.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287567+0000", 2024-09-18T00:19:24.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-18T00:19:24.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287597+0000", 2024-09-18T00:19:24.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0406e-05 2024-09-18T00:19:24.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287626+0000", 2024-09-18T00:19:24.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9306e-05 2024-09-18T00:19:24.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287801+0000", 2024-09-18T00:19:24.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017411400000000001 2024-09-18T00:19:24.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.288053+0000", 2024-09-18T00:19:24.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025223500000000001 2024-09-18T00:19:24.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:24.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364016+0000", 2024-09-18T00:19:24.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075963301999999996 2024-09-18T00:19:24.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:24.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364064+0000", 2024-09-18T00:19:24.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7886000000000002e-05 2024-09-18T00:19:24.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364142+0000", 2024-09-18T00:19:24.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8345999999999998e-05 2024-09-18T00:19:24.779 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.779 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.779 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.779 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:24700 3.4 3:23a8f5d4:::rbd_data.112a990280ef.0000000000000016:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:24.779 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:19:24.779 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.767794385999998, 2024-09-18T00:19:24.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091975649000000007, 2024-09-18T00:19:24.780 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.780 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.780 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:24.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.780 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 24700 2024-09-18T00:19:24.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.780 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:19:24.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:19:24.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938019+0000", 2024-09-18T00:19:24.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:19:24.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938030+0000", 2024-09-18T00:19:24.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0278000000000001e-05 2024-09-18T00:19:24.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938053+0000", 2024-09-18T00:19:24.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2946999999999999e-05 2024-09-18T00:19:24.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938062+0000", 2024-09-18T00:19:24.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8509999999999994e-06 2024-09-18T00:19:24.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938140+0000", 2024-09-18T00:19:24.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7664999999999996e-05 2024-09-18T00:19:24.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938314+0000", 2024-09-18T00:19:24.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017398500000000001 2024-09-18T00:19:24.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:24.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029896+0000", 2024-09-18T00:19:24.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091582078999999997 2024-09-18T00:19:24.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:24.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029930+0000", 2024-09-18T00:19:24.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3723999999999998e-05 2024-09-18T00:19:24.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029993+0000", 2024-09-18T00:19:24.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2901e-05 2024-09-18T00:19:24.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.784 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.785 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37798 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:24.785 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410235+0000", 2024-09-18T00:19:24.785 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2955765850000001, 2024-09-18T00:19:24.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099276060000000003, 2024-09-18T00:19:24.785 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.785 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.785 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:24.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.785 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.785 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37798 2024-09-18T00:19:24.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.785 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410235+0000", 2024-09-18T00:19:24.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410235+0000", 2024-09-18T00:19:24.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410235+0000", 2024-09-18T00:19:24.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4699999999999998e-07 2024-09-18T00:19:24.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410236+0000", 2024-09-18T00:19:24.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1069999999999999e-06 2024-09-18T00:19:24.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410240+0000", 2024-09-18T00:19:24.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3119999999999998e-06 2024-09-18T00:19:24.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.787 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.787 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410242+0000", 2024-09-18T00:19:24.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3860000000000001e-06 2024-09-18T00:19:24.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.419934+0000", 2024-09-18T00:19:24.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0096915720000000007 2024-09-18T00:19:24.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420021+0000", 2024-09-18T00:19:24.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6941999999999995e-05 2024-09-18T00:19:24.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420163+0000", 2024-09-18T00:19:24.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014203999999999999 2024-09-18T00:19:24.789 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.789 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.789 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.789 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37799 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:24.789 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:19:24.789 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2955662349999999, 2024-09-18T00:19:24.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186016000000001, 2024-09-18T00:19:24.789 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.789 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.789 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:24.789 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.790 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37799 2024-09-18T00:19:24.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.790 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:19:24.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:19:24.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410246+0000", 2024-09-18T00:19:24.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.35e-07 2024-09-18T00:19:24.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410247+0000", 2024-09-18T00:19:24.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0389999999999999e-06 2024-09-18T00:19:24.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410250+0000", 2024-09-18T00:19:24.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3380000000000001e-06 2024-09-18T00:19:24.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410252+0000", 2024-09-18T00:19:24.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5150000000000001e-06 2024-09-18T00:19:24.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420198+0000", 2024-09-18T00:19:24.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099454360000000002 2024-09-18T00:19:24.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420284+0000", 2024-09-18T00:19:24.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5638999999999998e-05 2024-09-18T00:19:24.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420431+0000", 2024-09-18T00:19:24.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014781399999999999 2024-09-18T00:19:24.793 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.793 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.793 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.793 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37800 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:24.794 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:19:24.794 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2955559589999996, 2024-09-18T00:19:24.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010452559, 2024-09-18T00:19:24.794 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.794 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.794 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:24.794 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.794 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.794 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.794 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37800 2024-09-18T00:19:24.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.794 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:19:24.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:19:24.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:19:24.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2499999999999999e-07 2024-09-18T00:19:24.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410257+0000", 2024-09-18T00:19:24.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0750000000000001e-06 2024-09-18T00:19:24.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410260+0000", 2024-09-18T00:19:24.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.428e-06 2024-09-18T00:19:24.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410263+0000", 2024-09-18T00:19:24.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5339999999999998e-06 2024-09-18T00:19:24.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420467+0000", 2024-09-18T00:19:24.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010204615 2024-09-18T00:19:24.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420557+0000", 2024-09-18T00:19:24.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9183000000000003e-05 2024-09-18T00:19:24.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420708+0000", 2024-09-18T00:19:24.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015149900000000001 2024-09-18T00:19:24.797 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.798 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.798 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.798 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37801 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:24.798 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:19:24.798 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2955456219999997, 2024-09-18T00:19:24.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010623357, 2024-09-18T00:19:24.798 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.798 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.798 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:24.798 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.798 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.798 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.799 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37801 2024-09-18T00:19:24.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.799 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:19:24.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:19:24.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:19:24.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.23e-07 2024-09-18T00:19:24.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410271+0000", 2024-09-18T00:19:24.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8040000000000001e-06 2024-09-18T00:19:24.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410274+0000", 2024-09-18T00:19:24.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5310000000000002e-06 2024-09-18T00:19:24.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410277+0000", 2024-09-18T00:19:24.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.605e-06 2024-09-18T00:19:24.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420743+0000", 2024-09-18T00:19:24.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010465487000000001 2024-09-18T00:19:24.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420811+0000", 2024-09-18T00:19:24.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8845999999999998e-05 2024-09-18T00:19:24.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420889+0000", 2024-09-18T00:19:24.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7861000000000006e-05 2024-09-18T00:19:24.802 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.802 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.802 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.802 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37802 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:24.802 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:19:24.803 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.2955309960000001, 2024-09-18T00:19:24.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010745463, 2024-09-18T00:19:24.803 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:24.803 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:24.803 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:24.803 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:24.803 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:24.803 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:24.803 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37802 2024-09-18T00:19:24.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.803 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:24.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:24.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:19:24.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:24.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:19:24.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:24.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:24.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:19:24.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6199999999999999e-07 2024-09-18T00:19:24.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:24.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410282+0000", 2024-09-18T00:19:24.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.085e-06 2024-09-18T00:19:24.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:24.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410286+0000", 2024-09-18T00:19:24.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6610000000000002e-06 2024-09-18T00:19:24.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:24.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410288+0000", 2024-09-18T00:19:24.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6410000000000002e-06 2024-09-18T00:19:24.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:24.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420910+0000", 2024-09-18T00:19:24.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010621479 2024-09-18T00:19:24.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:24.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420952+0000", 2024-09-18T00:19:24.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2497999999999999e-05 2024-09-18T00:19:24.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:24.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:24.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:24.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.421026+0000", 2024-09-18T00:19:24.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3837000000000005e-05 2024-09-18T00:19:24.806 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.807 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.807 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.807 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:24.807 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:24.807 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:24.808 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:19:24.809 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:24.805+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:24.909 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:24.905+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:24.970 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:24.971 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:24.971 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:24.971 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:24.983 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:19:25.025 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:25.021+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:25.111 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:25.109+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:25.142 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:25.142 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:25.143 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:25.143 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:25.143 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:25.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:19:25.212 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:25.205+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:25.277 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:19:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 72.474613372999997, 2024-09-18T00:19:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:19:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.278 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:19:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:19:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:19:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:19:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:19:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:19:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:19:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:19:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:19:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:19:25.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:19:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:19:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:19:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:19:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:19:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:25.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:19:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:19:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:19:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:19:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:19:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:19:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:19:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:19:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:19:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 72.458405499999998, 2024-09-18T00:19:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:19:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.284 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:19:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:19:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:19:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:19:25.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:19:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:19:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:19:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:19:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:19:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:19:25.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:19:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:19:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:19:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:25.288 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.431364180000003, 2024-09-18T00:19:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:19:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:19:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:25.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:19:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:19:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:19:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:19:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:19:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:19:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:19:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:19:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:19:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:19:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:19:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:19:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:19:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:19:25.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:19:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:19:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:19:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:19:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:19:25.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:19:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:19:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:19:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:19:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.410087627999999, 2024-09-18T00:19:25.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:19:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:19:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:19:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:19:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:19:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:19:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:19:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:19:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:19:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:19:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:19:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:19:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:19:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:19:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:19:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11460 3.19 3:9e578b6c:::rbd_data.112ab1fa102a.0000000000000153:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:25.298 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 50.007716176000002, 2024-09-18T00:19:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041989011999999999, 2024-09-18T00:19:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11460 2024-09-18T00:19:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:25.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267067+0000", 2024-09-18T00:19:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:19:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267088+0000", 2024-09-18T00:19:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0908999999999999e-05 2024-09-18T00:19:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267133+0000", 2024-09-18T00:19:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5232000000000003e-05 2024-09-18T00:19:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267157+0000", 2024-09-18T00:19:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3833e-05 2024-09-18T00:19:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267300+0000", 2024-09-18T00:19:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142282 2024-09-18T00:19:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267447+0000", 2024-09-18T00:19:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147851 2024-09-18T00:19:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308889+0000", 2024-09-18T00:19:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041441779999999998 2024-09-18T00:19:25.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308956+0000", 2024-09-18T00:19:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7136000000000002e-05 2024-09-18T00:19:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.309053+0000", 2024-09-18T00:19:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6552999999999995e-05 2024-09-18T00:19:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:16947 3.18 3:18b58c7b:::rbd_data.112a6efb7564.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:19:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.657938475000002, 2024-09-18T00:19:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.101692484, 2024-09-18T00:19:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 16947 2024-09-18T00:19:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.304 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616843+0000", 2024-09-18T00:19:25.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0780000000000001e-06 2024-09-18T00:19:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617785+0000", 2024-09-18T00:19:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094204100000000002 2024-09-18T00:19:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617801+0000", 2024-09-18T00:19:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6436999999999999e-05 2024-09-18T00:19:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617809+0000", 2024-09-18T00:19:25.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9049999999999997e-06 2024-09-18T00:19:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617904+0000", 2024-09-18T00:19:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5204999999999994e-05 2024-09-18T00:19:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.618071+0000", 2024-09-18T00:19:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016640000000000001 2024-09-18T00:19:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:25.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.624534+0000", 2024-09-18T00:19:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064634499999999999 2024-09-18T00:19:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718299+0000", 2024-09-18T00:19:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093764739 2024-09-18T00:19:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718375+0000", 2024-09-18T00:19:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6532999999999994e-05 2024-09-18T00:19:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718466+0000", 2024-09-18T00:19:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0581000000000006e-05 2024-09-18T00:19:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718534+0000", 2024-09-18T00:19:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8114999999999994e-05 2024-09-18T00:19:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.309 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:16947 3.18 e59/23 ondisk, result = 0)", 2024-09-18T00:19:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.632721781000001, 2024-09-18T00:19:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076488069000000006, 2024-09-18T00:19:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642060+0000", 2024-09-18T00:19:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6640000000000001e-06 2024-09-18T00:19:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642071+0000", 2024-09-18T00:19:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1204e-05 2024-09-18T00:19:25.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642091+0000", 2024-09-18T00:19:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0313999999999999e-05 2024-09-18T00:19:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642103+0000", 2024-09-18T00:19:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1214e-05 2024-09-18T00:19:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718350+0000", 2024-09-18T00:19:25.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076247681999999997 2024-09-18T00:19:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718366+0000", 2024-09-18T00:19:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5420000000000001e-05 2024-09-18T00:19:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718546+0000", 2024-09-18T00:19:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000180571 2024-09-18T00:19:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.313 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18686 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.326044410999998, 2024-09-18T00:19:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041132573999999998, 2024-09-18T00:19:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18686 2024-09-18T00:19:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.314 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948737+0000", 2024-09-18T00:19:25.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3769999999999999e-06 2024-09-18T00:19:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949968+0000", 2024-09-18T00:19:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001230493 2024-09-18T00:19:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949993+0000", 2024-09-18T00:19:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5282e-05 2024-09-18T00:19:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950011+0000", 2024-09-18T00:19:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8199999999999999e-05 2024-09-18T00:19:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950180+0000", 2024-09-18T00:19:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016867600000000001 2024-09-18T00:19:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950337+0000", 2024-09-18T00:19:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156902 2024-09-18T00:19:25.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950818+0000", 2024-09-18T00:19:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048192599999999998 2024-09-18T00:19:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978491+0000", 2024-09-18T00:19:25.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027672568000000002 2024-09-18T00:19:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989759+0000", 2024-09-18T00:19:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011268144000000001 2024-09-18T00:19:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989824+0000", 2024-09-18T00:19:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4556999999999999e-05 2024-09-18T00:19:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989868+0000", 2024-09-18T00:19:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4449000000000001e-05 2024-09-18T00:19:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18687 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.324716645999999, 2024-09-18T00:19:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040034156000000001, 2024-09-18T00:19:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.320 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18687 2024-09-18T00:19:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950065+0000", 2024-09-18T00:19:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5820000000000001e-06 2024-09-18T00:19:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950072+0000", 2024-09-18T00:19:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1029999999999996e-06 2024-09-18T00:19:25.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950092+0000", 2024-09-18T00:19:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0288e-05 2024-09-18T00:19:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950107+0000", 2024-09-18T00:19:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4290999999999999e-05 2024-09-18T00:19:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975820+0000", 2024-09-18T00:19:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025713573999999999 2024-09-18T00:19:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975980+0000", 2024-09-18T00:19:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016002799999999999 2024-09-18T00:19:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.976299+0000", 2024-09-18T00:19:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031853300000000002 2024-09-18T00:19:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989936+0000", 2024-09-18T00:19:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013637046999999999 2024-09-18T00:19:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990025+0000", 2024-09-18T00:19:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9040000000000001e-05 2024-09-18T00:19:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990064+0000", 2024-09-18T00:19:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9394000000000003e-05 2024-09-18T00:19:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990098+0000", 2024-09-18T00:19:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3275999999999998e-05 2024-09-18T00:19:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18688 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.324615293999997, 2024-09-18T00:19:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040041259000000003, 2024-09-18T00:19:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.326 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18688 2024-09-18T00:19:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950166+0000", 2024-09-18T00:19:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:19:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951295+0000", 2024-09-18T00:19:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011293480000000001 2024-09-18T00:19:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951328+0000", 2024-09-18T00:19:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2524999999999998e-05 2024-09-18T00:19:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951346+0000", 2024-09-18T00:19:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8170999999999999e-05 2024-09-18T00:19:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978546+0000", 2024-09-18T00:19:25.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027200418000000001 2024-09-18T00:19:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978666+0000", 2024-09-18T00:19:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119913 2024-09-18T00:19:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.979098+0000", 2024-09-18T00:19:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00043169399999999999 2024-09-18T00:19:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:25.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990000+0000", 2024-09-18T00:19:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010901719000000001 2024-09-18T00:19:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990109+0000", 2024-09-18T00:19:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010964299999999999 2024-09-18T00:19:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990152+0000", 2024-09-18T00:19:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2877000000000003e-05 2024-09-18T00:19:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990206+0000", 2024-09-18T00:19:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3717000000000001e-05 2024-09-18T00:19:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18689 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.323403595999999, 2024-09-18T00:19:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051022338, 2024-09-18T00:19:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.332 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18689 2024-09-18T00:19:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951378+0000", 2024-09-18T00:19:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:19:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951385+0000", 2024-09-18T00:19:25.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-18T00:19:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951402+0000", 2024-09-18T00:19:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6538999999999999e-05 2024-09-18T00:19:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951565+0000", 2024-09-18T00:19:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016307999999999999 2024-09-18T00:19:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988232+0000", 2024-09-18T00:19:25.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036667656999999999 2024-09-18T00:19:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988318+0000", 2024-09-18T00:19:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5348999999999999e-05 2024-09-18T00:19:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988574+0000", 2024-09-18T00:19:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025670800000000002 2024-09-18T00:19:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002254+0000", 2024-09-18T00:19:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013679274 2024-09-18T00:19:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002326+0000", 2024-09-18T00:19:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2720999999999997e-05 2024-09-18T00:19:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002364+0000", 2024-09-18T00:19:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7314000000000002e-05 2024-09-18T00:19:25.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002399+0000", 2024-09-18T00:19:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5162e-05 2024-09-18T00:19:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18690 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.323186577999998, 2024-09-18T00:19:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050872134999999999, 2024-09-18T00:19:25.338 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18690 2024-09-18T00:19:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951595+0000", 2024-09-18T00:19:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4479999999999999e-06 2024-09-18T00:19:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952167+0000", 2024-09-18T00:19:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00057188400000000004 2024-09-18T00:19:25.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952192+0000", 2024-09-18T00:19:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4936999999999999e-05 2024-09-18T00:19:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952206+0000", 2024-09-18T00:19:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4565e-05 2024-09-18T00:19:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990303+0000", 2024-09-18T00:19:25.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038096954000000002 2024-09-18T00:19:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990403+0000", 2024-09-18T00:19:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9486000000000003e-05 2024-09-18T00:19:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990773+0000", 2024-09-18T00:19:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037037799999999998 2024-09-18T00:19:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:25.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002316+0000", 2024-09-18T00:19:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011542581 2024-09-18T00:19:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002407+0000", 2024-09-18T00:19:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0968e-05 2024-09-18T00:19:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002442+0000", 2024-09-18T00:19:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4962000000000002e-05 2024-09-18T00:19:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002466+0000", 2024-09-18T00:19:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3972e-05 2024-09-18T00:19:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18691 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.322553700999997, 2024-09-18T00:19:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051495530999999997, 2024-09-18T00:19:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.344 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18691 2024-09-18T00:19:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952228+0000", 2024-09-18T00:19:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:19:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952234+0000", 2024-09-18T00:19:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0310000000000004e-06 2024-09-18T00:19:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952248+0000", 2024-09-18T00:19:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4577e-05 2024-09-18T00:19:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952260+0000", 2024-09-18T00:19:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1659e-05 2024-09-18T00:19:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.000989+0000", 2024-09-18T00:19:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048729445000000003 2024-09-18T00:19:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001076+0000", 2024-09-18T00:19:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6323999999999995e-05 2024-09-18T00:19:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001295+0000", 2024-09-18T00:19:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021944299999999999 2024-09-18T00:19:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003502+0000", 2024-09-18T00:19:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022065209999999999 2024-09-18T00:19:25.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:25.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003581+0000", 2024-09-18T00:19:25.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9404000000000001e-05 2024-09-18T00:19:25.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:25.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003652+0000", 2024-09-18T00:19:25.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0506999999999994e-05 2024-09-18T00:19:25.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003722+0000", 2024-09-18T00:19:25.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0300000000000001e-05 2024-09-18T00:19:25.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.350 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.350 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18692 3.7 3:e69124d0:::rbd_data.112a6efb7564.0000000000000001:f [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:25.350 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:25.350 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 32.322505640000003, 2024-09-18T00:19:25.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050418114999999999, 2024-09-18T00:19:25.350 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.350 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.350 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:25.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.350 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.351 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.351 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18692 2024-09-18T00:19:25.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.351 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:25.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:25.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952275+0000", 2024-09-18T00:19:25.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4e-07 2024-09-18T00:19:25.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952281+0000", 2024-09-18T00:19:25.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2000000000000002e-06 2024-09-18T00:19:25.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952295+0000", 2024-09-18T00:19:25.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4731e-05 2024-09-18T00:19:25.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952307+0000", 2024-09-18T00:19:25.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1877e-05 2024-09-18T00:19:25.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002506+0000", 2024-09-18T00:19:25.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050199230999999997 2024-09-18T00:19:25.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002574+0000", 2024-09-18T00:19:25.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8076000000000003e-05 2024-09-18T00:19:25.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002693+0000", 2024-09-18T00:19:25.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011806 2024-09-18T00:19:25.354 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.354 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.354 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.354 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19383 3.5 3:a1859bb0:::rbd_data.112a7fe29ab4.000000000000015b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:25.354 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:25.355 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.652494043000001, 2024-09-18T00:19:25.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056760192000000001, 2024-09-18T00:19:25.355 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.355 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.355 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.355 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.355 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.355 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.355 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19383 2024-09-18T00:19:25.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.355 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622287+0000", 2024-09-18T00:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-18T00:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622292+0000", 2024-09-18T00:19:25.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6110000000000004e-06 2024-09-18T00:19:25.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622304+0000", 2024-09-18T00:19:25.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2106000000000001e-05 2024-09-18T00:19:25.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622309+0000", 2024-09-18T00:19:25.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.818e-06 2024-09-18T00:19:25.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622360+0000", 2024-09-18T00:19:25.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0430999999999999e-05 2024-09-18T00:19:25.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622438+0000", 2024-09-18T00:19:25.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8362999999999995e-05 2024-09-18T00:19:25.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:25.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678811+0000", 2024-09-18T00:19:25.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056372894 2024-09-18T00:19:25.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:25.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678906+0000", 2024-09-18T00:19:25.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5078e-05 2024-09-18T00:19:25.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.679046+0000", 2024-09-18T00:19:25.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139987 2024-09-18T00:19:25.359 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.359 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.359 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.360 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22019 3.c 3:37b77e94:::rbd_data.112a948d5956.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:25.360 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:19:25.360 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.775519213999999, 2024-09-18T00:19:25.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.039176065000000003, 2024-09-18T00:19:25.360 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.360 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.360 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.360 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.360 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.360 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.360 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22019 2024-09-18T00:19:25.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.360 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:19:25.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:19:25.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499262+0000", 2024-09-18T00:19:25.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:19:25.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499268+0000", 2024-09-18T00:19:25.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8780000000000003e-06 2024-09-18T00:19:25.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499283+0000", 2024-09-18T00:19:25.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5184000000000001e-05 2024-09-18T00:19:25.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499291+0000", 2024-09-18T00:19:25.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5959999999999998e-06 2024-09-18T00:19:25.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499364+0000", 2024-09-18T00:19:25.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2855000000000006e-05 2024-09-18T00:19:25.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499427+0000", 2024-09-18T00:19:25.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3645999999999993e-05 2024-09-18T00:19:25.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:25.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538252+0000", 2024-09-18T00:19:25.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038825207 2024-09-18T00:19:25.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:25.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538328+0000", 2024-09-18T00:19:25.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5938000000000004e-05 2024-09-18T00:19:25.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538437+0000", 2024-09-18T00:19:25.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108527 2024-09-18T00:19:25.364 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.364 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.365 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.365 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22468 3.5 3:a250f1ef:::rbd_data.112a948d5956.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:25.365 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:25.365 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.503412637, 2024-09-18T00:19:25.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.055599787999999997, 2024-09-18T00:19:25.365 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.365 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.365 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.365 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.366 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22468 2024-09-18T00:19:25.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.366 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:25.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:25.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771370+0000", 2024-09-18T00:19:25.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9629999999999998e-06 2024-09-18T00:19:25.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771385+0000", 2024-09-18T00:19:25.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5053e-05 2024-09-18T00:19:25.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771429+0000", 2024-09-18T00:19:25.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3192999999999998e-05 2024-09-18T00:19:25.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771453+0000", 2024-09-18T00:19:25.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3995999999999999e-05 2024-09-18T00:19:25.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771623+0000", 2024-09-18T00:19:25.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000170846 2024-09-18T00:19:25.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771838+0000", 2024-09-18T00:19:25.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021416600000000001 2024-09-18T00:19:25.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:25.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826866+0000", 2024-09-18T00:19:25.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.05502866 2024-09-18T00:19:25.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:25.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826906+0000", 2024-09-18T00:19:25.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9805999999999998e-05 2024-09-18T00:19:25.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826967+0000", 2024-09-18T00:19:25.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1105000000000005e-05 2024-09-18T00:19:25.370 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.370 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.370 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.370 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22470 3.2 3:474595b6:::rbd_data.112a948d5956.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:25.370 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:25.370 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 26.447221653, 2024-09-18T00:19:25.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088321282000000001, 2024-09-18T00:19:25.370 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.370 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.370 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.370 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.371 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.371 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.371 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22470 2024-09-18T00:19:25.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.371 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:25.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:25.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827560+0000", 2024-09-18T00:19:25.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.762e-06 2024-09-18T00:19:25.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827569+0000", 2024-09-18T00:19:25.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8740000000000001e-06 2024-09-18T00:19:25.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.372 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.372 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827585+0000", 2024-09-18T00:19:25.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5962000000000001e-05 2024-09-18T00:19:25.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827595+0000", 2024-09-18T00:19:25.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5910000000000002e-06 2024-09-18T00:19:25.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827811+0000", 2024-09-18T00:19:25.373 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000216858 2024-09-18T00:19:25.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:19:25.373 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827818+0000", 2024-09-18T00:19:25.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.99e-06 2024-09-18T00:19:25.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829799+0000", 2024-09-18T00:19:25.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019803849999999999 2024-09-18T00:19:25.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829884+0000", 2024-09-18T00:19:25.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4852999999999995e-05 2024-09-18T00:19:25.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915616+0000", 2024-09-18T00:19:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085732808999999993 2024-09-18T00:19:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915711+0000", 2024-09-18T00:19:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4578000000000001e-05 2024-09-18T00:19:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915880+0000", 2024-09-18T00:19:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016861999999999999 2024-09-18T00:19:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.375 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.376 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.376 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:42788 3.7 3:e5a8432b:::rbd_data.112a3496573c.0000000000000027:17 [read 309248~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:25.376 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:25.376 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.5558098529999995, 2024-09-18T00:19:25.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.039621847000000002, 2024-09-18T00:19:25.376 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.376 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.376 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:25.376 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.376 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.376 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.377 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 42788 2024-09-18T00:19:25.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.377 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:25.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:25.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718971+0000", 2024-09-18T00:19:25.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2500000000000001e-07 2024-09-18T00:19:25.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718973+0000", 2024-09-18T00:19:25.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2510000000000001e-06 2024-09-18T00:19:25.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718979+0000", 2024-09-18T00:19:25.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1700000000000002e-06 2024-09-18T00:19:25.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718984+0000", 2024-09-18T00:19:25.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9110000000000001e-06 2024-09-18T00:19:25.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.749126+0000", 2024-09-18T00:19:25.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030142164999999999 2024-09-18T00:19:25.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.749199+0000", 2024-09-18T00:19:25.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2985999999999995e-05 2024-09-18T00:19:25.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.380 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.380 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.758592+0000", 2024-09-18T00:19:25.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093929389999999995 2024-09-18T00:19:25.380 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.380 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.380 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.380 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:42820 3.7 3:e2363f9c:::rbd_data.112a3496573c.0000000000000028:17 [read 446464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:25.380 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.719971+0000", 2024-09-18T00:19:25.380 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.5548093569999999, 2024-09-18T00:19:25.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034904399000000003, 2024-09-18T00:19:25.380 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:25.381 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:25.381 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:25.381 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:25.381 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:25.381 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.381 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 42820 2024-09-18T00:19:25.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.381 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:25.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:25.381 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719971+0000", 2024-09-18T00:19:25.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:25.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719971+0000", 2024-09-18T00:19:25.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:25.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:25.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719971+0000", 2024-09-18T00:19:25.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5999999999999999e-07 2024-09-18T00:19:25.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:25.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719973+0000", 2024-09-18T00:19:25.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.069e-06 2024-09-18T00:19:25.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:25.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719980+0000", 2024-09-18T00:19:25.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5379999999999999e-06 2024-09-18T00:19:25.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719985+0000", 2024-09-18T00:19:25.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1080000000000001e-06 2024-09-18T00:19:25.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:25.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.754731+0000", 2024-09-18T00:19:25.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034745725999999998 2024-09-18T00:19:25.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:25.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.754804+0000", 2024-09-18T00:19:25.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3396e-05 2024-09-18T00:19:25.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:25.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:25.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:25.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.754875+0000", 2024-09-18T00:19:25.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1101999999999998e-05 2024-09-18T00:19:25.384 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.384 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.384 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.384 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:25.385 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:25.385 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:25.386 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:19:25.387 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:25.381+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is unset 2024-09-18T00:19:25.424 INFO:teuthology.orchestra.run.smithi097.stdout:5090 writesame 0xeadeeb8 thru 0xeaea61e data_size 0x1(0xb767 bytes) 2024-09-18T00:19:25.425 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:25.425 INFO:teuthology.orchestra.run.smithi097.stdout:5095 writesame 0x2ede1f9 thru 0x2eded61 data_size 0x1(0xb69 bytes) 2024-09-18T00:19:25.489 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:25.485+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:25.520 INFO:teuthology.orchestra.run.smithi097.stdout:5096 trunc from 0xeaea61f to 0xebce5af 2024-09-18T00:19:25.527 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:25.527 INFO:teuthology.orchestra.run.smithi097.stdout:5098 write 0xd65c0a9 thru 0xd66b356 (0xf2ae bytes) 2024-09-18T00:19:25.547 INFO:teuthology.orchestra.run.smithi097.stdout:5099 trunc from 0xebce5af to 0x5f8f39c 2024-09-18T00:19:25.552 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:25.552 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:25.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.552 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4394.0:46533 3.13 e86/32 3:cd7a6742:::rbd_data.112aa373d2ec.000000000000006a:head v 86'671, mlcod=83'669)", 2024-09-18T00:19:25.552 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:25.551102+0000", 2024-09-18T00:19:25.552 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.00089726699999999999, 2024-09-18T00:19:25.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00092036400000000001, 2024-09-18T00:19:25.553 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.553 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.553 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:25.553 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551102+0000", 2024-09-18T00:19:25.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551102+0000", 2024-09-18T00:19:25.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551103+0000", 2024-09-18T00:19:25.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8000000000000004e-07 2024-09-18T00:19:25.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551109+0000", 2024-09-18T00:19:25.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-18T00:19:25.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551123+0000", 2024-09-18T00:19:25.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4054e-05 2024-09-18T00:19:25.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551131+0000", 2024-09-18T00:19:25.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3930000000000002e-06 2024-09-18T00:19:25.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:25.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551186+0000", 2024-09-18T00:19:25.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5407e-05 2024-09-18T00:19:25.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:25.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551207+0000", 2024-09-18T00:19:25.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0754000000000002e-05 2024-09-18T00:19:25.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.556 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:25.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.556 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4394.0:46527 3.19 e86/85 3:9834f227:::rbd_data.112aa373d2ec.0000000000000060:head v 86'264, mlcod=28'1)", 2024-09-18T00:19:25.556 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:25.551755+0000", 2024-09-18T00:19:25.556 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.000244329, 2024-09-18T00:19:25.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032954300000000001, 2024-09-18T00:19:25.556 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.556 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.556 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:25.556 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551755+0000", 2024-09-18T00:19:25.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551755+0000", 2024-09-18T00:19:25.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551756+0000", 2024-09-18T00:19:25.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0020000000000001e-06 2024-09-18T00:19:25.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551767+0000", 2024-09-18T00:19:25.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1674e-05 2024-09-18T00:19:25.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551780+0000", 2024-09-18T00:19:25.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2177999999999999e-05 2024-09-18T00:19:25.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551787+0000", 2024-09-18T00:19:25.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2869999999999999e-06 2024-09-18T00:19:25.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:25.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551845+0000", 2024-09-18T00:19:25.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8118000000000002e-05 2024-09-18T00:19:25.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:25.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551867+0000", 2024-09-18T00:19:25.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1566999999999999e-05 2024-09-18T00:19:25.559 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.560 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:25.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.560 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4394.0:46530 3.19 e86/85)", 2024-09-18T00:19:25.560 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:25.551928+0000", 2024-09-18T00:19:25.560 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.1120999999999999e-05, 2024-09-18T00:19:25.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018819999999999999, 2024-09-18T00:19:25.560 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.560 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.560 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "queued for pg", 2024-09-18T00:19:25.560 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551928+0000", 2024-09-18T00:19:25.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551928+0000", 2024-09-18T00:19:25.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551929+0000", 2024-09-18T00:19:25.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0299999999999997e-07 2024-09-18T00:19:25.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551936+0000", 2024-09-18T00:19:25.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9140000000000002e-06 2024-09-18T00:19:25.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551957+0000", 2024-09-18T00:19:25.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0956999999999999e-05 2024-09-18T00:19:25.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:25.551965+0000", 2024-09-18T00:19:25.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1149999999999994e-06 2024-09-18T00:19:25.563 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.563 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:25.563 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.563 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.563 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-18T00:19:25.563 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 3 2024-09-18T00:19:25.563 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:25.573 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:19:25.618 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:25.617+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:25.688 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:25.685+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:25.741 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:25.741 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:25.741 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:25.742 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:25.742 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:25.758 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:19:25.806 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:25.805+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:25.831 INFO:teuthology.orchestra.run.smithi097.stdout:5100 write 0xcd1bb2d thru 0xcd1e4e6 (0x29ba bytes) 2024-09-18T00:19:25.840 INFO:teuthology.orchestra.run.smithi097.stdout:5101 read 0x846ba35 thru 0x8479452 (0xda1e bytes) 2024-09-18T00:19:25.841 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:25.855 INFO:teuthology.orchestra.run.smithi097.stdout:5103 trunc from 0xcd1e4e7 to 0x5c216fa 2024-09-18T00:19:25.890 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:25.885+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:25.901 INFO:teuthology.orchestra.run.smithi097.stdout:5105 write 0xece6555 thru 0xece9101 (0x2bad bytes) 2024-09-18T00:19:25.906 INFO:teuthology.orchestra.run.smithi097.stdout:5106 punch from 0x1bcad84 to 0x1bd7941, (0xcbbd bytes) 2024-09-18T00:19:25.907 INFO:teuthology.orchestra.run.smithi097.stdout:5107 punch from 0xcd6565d to 0xcd7367a, (0xe01d bytes) 2024-09-18T00:19:25.907 INFO:teuthology.orchestra.run.smithi097.stdout:5108 writesame 0x70c9a2b thru 0x70ccb10 data_size 0xb(0x30e6 bytes) 2024-09-18T00:19:25.916 INFO:teuthology.orchestra.run.smithi097.stdout:5109 read 0xd2d0252 thru 0xd2def67 (0xed16 bytes) 2024-09-18T00:19:25.916 INFO:teuthology.orchestra.run.smithi097.stdout:5110 punch from 0x33e10e to 0x343e26, (0x5d18 bytes) 2024-09-18T00:19:25.916 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:25.916 INFO:teuthology.orchestra.run.smithi097.stdout:5113 read 0xd8cf0ee thru 0xd8d88c5 (0x97d8 bytes) 2024-09-18T00:19:25.927 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:25.927 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:25.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:25.928 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:25.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.928 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2418 3.1a 3:59a8b40d:::rbd_data.112aa7ba4578.00000000000001b0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:19:25.928 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:25.928 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.190171767999999, 2024-09-18T00:19:25.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10488562999999999, 2024-09-18T00:19:25.928 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.928 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.929 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.929 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:25.929 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:25.929 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.929 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2418 2024-09-18T00:19:25.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.930 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:25.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:25.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735592+0000", 2024-09-18T00:19:25.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5229999999999998e-06 2024-09-18T00:19:25.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735604+0000", 2024-09-18T00:19:25.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2143e-05 2024-09-18T00:19:25.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735629+0000", 2024-09-18T00:19:25.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.497e-05 2024-09-18T00:19:25.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735648+0000", 2024-09-18T00:19:25.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8896999999999999e-05 2024-09-18T00:19:25.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:25.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735775+0000", 2024-09-18T00:19:25.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126935 2024-09-18T00:19:25.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:25.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.736118+0000", 2024-09-18T00:19:25.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034330800000000001 2024-09-18T00:19:25.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-18T00:19:25.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.741633+0000", 2024-09-18T00:19:25.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055144139999999996 2024-09-18T00:19:25.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:25.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840134+0000", 2024-09-18T00:19:25.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.098501618999999999 2024-09-18T00:19:25.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:25.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840184+0000", 2024-09-18T00:19:25.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9953000000000001e-05 2024-09-18T00:19:25.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:25.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840336+0000", 2024-09-18T00:19:25.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151476 2024-09-18T00:19:25.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:25.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840475+0000", 2024-09-18T00:19:25.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013939200000000001 2024-09-18T00:19:25.935 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.935 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:25.935 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.935 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2418 3.1a e37/23 ondisk, result = 0)", 2024-09-18T00:19:25.935 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:19:25.935 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.171581924999998, 2024-09-18T00:19:25.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.085979978999999998, 2024-09-18T00:19:25.935 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.935 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.935 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:25.935 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:19:25.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754179+0000", 2024-09-18T00:19:25.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754181+0000", 2024-09-18T00:19:25.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8530000000000001e-06 2024-09-18T00:19:25.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754194+0000", 2024-09-18T00:19:25.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.292e-05 2024-09-18T00:19:25.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754230+0000", 2024-09-18T00:19:25.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5602999999999998e-05 2024-09-18T00:19:25.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.754249+0000", 2024-09-18T00:19:25.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8850000000000001e-05 2024-09-18T00:19:25.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:25.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840070+0000", 2024-09-18T00:19:25.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.085821123999999999 2024-09-18T00:19:25.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:25.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840098+0000", 2024-09-18T00:19:25.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8546000000000001e-05 2024-09-18T00:19:25.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:25.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840159+0000", 2024-09-18T00:19:25.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1082999999999998e-05 2024-09-18T00:19:25.939 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.939 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:25.939 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.939 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12545 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:25.939 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:19:25.939 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.530519163000001, 2024-09-18T00:19:25.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045427642999999997, 2024-09-18T00:19:25.940 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.940 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.940 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:25.940 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:25.940 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:25.940 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.940 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12545 2024-09-18T00:19:25.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.940 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:19:25.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395242+0000", 2024-09-18T00:19:25.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395243+0000", 2024-09-18T00:19:25.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.206e-06 2024-09-18T00:19:25.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395249+0000", 2024-09-18T00:19:25.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3600000000000004e-06 2024-09-18T00:19:25.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395270+0000", 2024-09-18T00:19:25.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1223e-05 2024-09-18T00:19:25.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395285+0000", 2024-09-18T00:19:25.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4725999999999999e-05 2024-09-18T00:19:25.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:25.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440296+0000", 2024-09-18T00:19:25.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045011402999999998 2024-09-18T00:19:25.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:25.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440453+0000", 2024-09-18T00:19:25.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015742299999999999 2024-09-18T00:19:25.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:25.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440670+0000", 2024-09-18T00:19:25.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000216302 2024-09-18T00:19:25.943 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.943 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:25.943 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.944 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12546 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:25.944 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:19:25.944 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.530456954999998, 2024-09-18T00:19:25.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045915171999999997, 2024-09-18T00:19:25.944 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.944 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.944 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:25.944 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:25.944 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:25.944 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.944 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12546 2024-09-18T00:19:25.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.945 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:19:25.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395304+0000", 2024-09-18T00:19:25.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395306+0000", 2024-09-18T00:19:25.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.206e-06 2024-09-18T00:19:25.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395330+0000", 2024-09-18T00:19:25.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4686e-05 2024-09-18T00:19:25.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395356+0000", 2024-09-18T00:19:25.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5418999999999998e-05 2024-09-18T00:19:25.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395387+0000", 2024-09-18T00:19:25.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0942000000000003e-05 2024-09-18T00:19:25.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:25.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440743+0000", 2024-09-18T00:19:25.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045356592000000001 2024-09-18T00:19:25.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:25.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.440880+0000", 2024-09-18T00:19:25.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013641700000000001 2024-09-18T00:19:25.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:25.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441219+0000", 2024-09-18T00:19:25.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033991000000000002 2024-09-18T00:19:25.948 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.948 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:25.948 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.949 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12547 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 212992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:25.949 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:19:25.949 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.530347622000001, 2024-09-18T00:19:25.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046163691999999999, 2024-09-18T00:19:25.949 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.949 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.949 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:25.949 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:25.949 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:25.949 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.949 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12547 2024-09-18T00:19:25.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.950 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:19:25.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395414+0000", 2024-09-18T00:19:25.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395415+0000", 2024-09-18T00:19:25.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.553e-06 2024-09-18T00:19:25.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395426+0000", 2024-09-18T00:19:25.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0682999999999999e-05 2024-09-18T00:19:25.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395449+0000", 2024-09-18T00:19:25.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3082999999999999e-05 2024-09-18T00:19:25.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.952 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.952 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395469+0000", 2024-09-18T00:19:25.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0173000000000002e-05 2024-09-18T00:19:25.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:25.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441289+0000", 2024-09-18T00:19:25.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045820195000000001 2024-09-18T00:19:25.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:25.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441426+0000", 2024-09-18T00:19:25.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013712699999999999 2024-09-18T00:19:25.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:25.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441577+0000", 2024-09-18T00:19:25.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015087799999999999 2024-09-18T00:19:25.953 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.954 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:25.954 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.954 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12548 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:25.954 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:19:25.954 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.530273860999998, 2024-09-18T00:19:25.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046293932000000003, 2024-09-18T00:19:25.954 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.954 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.955 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:25.955 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:25.955 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:25.955 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.955 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12548 2024-09-18T00:19:25.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.955 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:19:25.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395487+0000", 2024-09-18T00:19:25.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395489+0000", 2024-09-18T00:19:25.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.22e-06 2024-09-18T00:19:25.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395512+0000", 2024-09-18T00:19:25.957 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3312e-05 2024-09-18T00:19:25.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.957 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.957 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395621+0000", 2024-09-18T00:19:25.957 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010859 2024-09-18T00:19:25.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.957 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.957 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395689+0000", 2024-09-18T00:19:25.957 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8232000000000007e-05 2024-09-18T00:19:25.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:25.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441610+0000", 2024-09-18T00:19:25.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045920751000000003 2024-09-18T00:19:25.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:25.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441695+0000", 2024-09-18T00:19:25.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5379999999999999e-05 2024-09-18T00:19:25.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:25.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441781+0000", 2024-09-18T00:19:25.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6446999999999994e-05 2024-09-18T00:19:25.959 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.959 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:25.959 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.959 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12549 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 294912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:25.959 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:19:25.959 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.530013209000003, 2024-09-18T00:19:25.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046410264999999999, 2024-09-18T00:19:25.959 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.959 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.960 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:25.960 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:25.960 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:25.960 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.960 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12549 2024-09-18T00:19:25.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.960 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.960 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:19:25.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.961 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.961 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.961 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395748+0000", 2024-09-18T00:19:25.961 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.961 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.961 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.961 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395750+0000", 2024-09-18T00:19:25.961 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5629999999999999e-06 2024-09-18T00:19:25.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.962 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395758+0000", 2024-09-18T00:19:25.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5259999999999993e-06 2024-09-18T00:19:25.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.962 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395793+0000", 2024-09-18T00:19:25.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4459000000000003e-05 2024-09-18T00:19:25.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395809+0000", 2024-09-18T00:19:25.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6433e-05 2024-09-18T00:19:25.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:25.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.441851+0000", 2024-09-18T00:19:25.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046041493000000003 2024-09-18T00:19:25.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:25.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442002+0000", 2024-09-18T00:19:25.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151433 2024-09-18T00:19:25.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:25.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442158+0000", 2024-09-18T00:19:25.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000156358 2024-09-18T00:19:25.964 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.964 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:25.964 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.965 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12550 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:25.965 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:19:25.965 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.529901719999998, 2024-09-18T00:19:25.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046498398000000003, 2024-09-18T00:19:25.965 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.965 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.965 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:25.965 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:25.965 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:25.965 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.965 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12550 2024-09-18T00:19:25.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.965 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.965 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:19:25.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395860+0000", 2024-09-18T00:19:25.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395861+0000", 2024-09-18T00:19:25.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.234e-06 2024-09-18T00:19:25.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395872+0000", 2024-09-18T00:19:25.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1409e-05 2024-09-18T00:19:25.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395892+0000", 2024-09-18T00:19:25.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9680000000000001e-05 2024-09-18T00:19:25.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.395911+0000", 2024-09-18T00:19:25.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9283000000000002e-05 2024-09-18T00:19:25.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:25.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442210+0000", 2024-09-18T00:19:25.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046298372999999997 2024-09-18T00:19:25.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:25.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442279+0000", 2024-09-18T00:19:25.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9055999999999997e-05 2024-09-18T00:19:25.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:25.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442358+0000", 2024-09-18T00:19:25.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9363000000000006e-05 2024-09-18T00:19:25.969 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.969 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:25.969 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.969 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12551 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:25.969 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:19:25.969 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.529229268000002, 2024-09-18T00:19:25.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046066902999999999, 2024-09-18T00:19:25.969 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.969 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.969 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:25.970 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:25.970 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:25.970 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.970 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12551 2024-09-18T00:19:25.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.970 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.970 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.970 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:19:25.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.970 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.970 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396532+0000", 2024-09-18T00:19:25.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396534+0000", 2024-09-18T00:19:25.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6869999999999999e-06 2024-09-18T00:19:25.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396543+0000", 2024-09-18T00:19:25.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3430000000000002e-06 2024-09-18T00:19:25.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396563+0000", 2024-09-18T00:19:25.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0063000000000001e-05 2024-09-18T00:19:25.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396579+0000", 2024-09-18T00:19:25.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5610000000000001e-05 2024-09-18T00:19:25.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:25.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442434+0000", 2024-09-18T00:19:25.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045855094999999998 2024-09-18T00:19:25.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:25.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442515+0000", 2024-09-18T00:19:25.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0988000000000005e-05 2024-09-18T00:19:25.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:25.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442599+0000", 2024-09-18T00:19:25.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4116999999999994e-05 2024-09-18T00:19:25.973 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.973 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:25.973 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.974 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12552 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:25.974 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:19:25.974 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.529154466000001, 2024-09-18T00:19:25.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046210804000000001, 2024-09-18T00:19:25.974 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.974 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.974 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:25.974 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:25.974 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:25.974 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.974 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12552 2024-09-18T00:19:25.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.975 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:19:25.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396607+0000", 2024-09-18T00:19:25.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396608+0000", 2024-09-18T00:19:25.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4729999999999999e-06 2024-09-18T00:19:25.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396614+0000", 2024-09-18T00:19:25.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1e-06 2024-09-18T00:19:25.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396633+0000", 2024-09-18T00:19:25.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9009e-05 2024-09-18T00:19:25.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396650+0000", 2024-09-18T00:19:25.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7045999999999999e-05 2024-09-18T00:19:25.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:25.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442662+0000", 2024-09-18T00:19:25.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046011170999999997 2024-09-18T00:19:25.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:25.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442737+0000", 2024-09-18T00:19:25.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5759000000000004e-05 2024-09-18T00:19:25.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:25.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442818+0000", 2024-09-18T00:19:25.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0246000000000004e-05 2024-09-18T00:19:25.978 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.978 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:25.978 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.978 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12553 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 458752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:25.979 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:19:25.979 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.529091762, 2024-09-18T00:19:25.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046339454000000002, 2024-09-18T00:19:25.979 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.979 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.979 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:25.979 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:25.979 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:25.979 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.979 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12553 2024-09-18T00:19:25.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.979 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:19:25.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396670+0000", 2024-09-18T00:19:25.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396671+0000", 2024-09-18T00:19:25.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.314e-06 2024-09-18T00:19:25.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396676+0000", 2024-09-18T00:19:25.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6659999999999996e-06 2024-09-18T00:19:25.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396695+0000", 2024-09-18T00:19:25.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8383e-05 2024-09-18T00:19:25.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396709+0000", 2024-09-18T00:19:25.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4309e-05 2024-09-18T00:19:25.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:25.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442871+0000", 2024-09-18T00:19:25.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046162295999999998 2024-09-18T00:19:25.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:25.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.442939+0000", 2024-09-18T00:19:25.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7607999999999994e-05 2024-09-18T00:19:25.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.983 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:25.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443009+0000", 2024-09-18T00:19:25.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9877999999999997e-05 2024-09-18T00:19:25.983 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.983 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:25.983 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.983 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12554 3.3 3:c09d8fd6:::rbd_data.112ab1fa102a.0000000000000001:9 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:25.983 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:19:25.984 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.529033839999997, 2024-09-18T00:19:25.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046504050999999998, 2024-09-18T00:19:25.984 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.984 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.984 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:25.984 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:25.984 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:25.984 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.984 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12554 2024-09-18T00:19:25.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.984 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:19:25.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396727+0000", 2024-09-18T00:19:25.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396729+0000", 2024-09-18T00:19:25.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2559999999999999e-06 2024-09-18T00:19:25.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396736+0000", 2024-09-18T00:19:25.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6140000000000003e-06 2024-09-18T00:19:25.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396755+0000", 2024-09-18T00:19:25.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8221999999999999e-05 2024-09-18T00:19:25.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.396769+0000", 2024-09-18T00:19:25.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4372999999999999e-05 2024-09-18T00:19:25.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:25.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443078+0000", 2024-09-18T00:19:25.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046308850999999998 2024-09-18T00:19:25.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:25.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443152+0000", 2024-09-18T00:19:25.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4083000000000002e-05 2024-09-18T00:19:25.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:25.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443231+0000", 2024-09-18T00:19:25.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9652000000000002e-05 2024-09-18T00:19:25.988 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.988 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:25.988 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.989 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12632 3.3 3:c21fac02:::rbd_data.112a7b1589a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1490944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:25.989 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:19:25.989 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.522475555, 2024-09-18T00:19:25.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045843916999999998, 2024-09-18T00:19:25.989 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.989 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.989 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.989 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:25.989 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:25.989 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.990 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12632 2024-09-18T00:19:25.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.990 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:19:25.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403286+0000", 2024-09-18T00:19:25.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403288+0000", 2024-09-18T00:19:25.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3630000000000001e-06 2024-09-18T00:19:25.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403886+0000", 2024-09-18T00:19:25.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00059840100000000003 2024-09-18T00:19:25.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403913+0000", 2024-09-18T00:19:25.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6376e-05 2024-09-18T00:19:25.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.403927+0000", 2024-09-18T00:19:25.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4188e-05 2024-09-18T00:19:25.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:25.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443268+0000", 2024-09-18T00:19:25.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.039340591000000001 2024-09-18T00:19:25.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:25.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443488+0000", 2024-09-18T00:19:25.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000220545 2024-09-18T00:19:25.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:25.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.443918+0000", 2024-09-18T00:19:25.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043021800000000001 2024-09-18T00:19:25.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:25.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.448558+0000", 2024-09-18T00:19:25.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046391449999999999 2024-09-18T00:19:25.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:25.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449015+0000", 2024-09-18T00:19:25.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045766600000000002 2024-09-18T00:19:25.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:25.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449085+0000", 2024-09-18T00:19:25.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0191999999999995e-05 2024-09-18T00:19:25.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:25.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.449130+0000", 2024-09-18T00:19:25.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4231999999999999e-05 2024-09-18T00:19:25.994 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.995 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:25.995 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:25.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.995 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:19386 3.1d 3:bcd89fc1:::rbd_data.112a7fe29ab4.00000000000000c8:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:25.995 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:53.699994+0000", 2024-09-18T00:19:25.995 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.225767413, 2024-09-18T00:19:25.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050613197999999998, 2024-09-18T00:19:25.995 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:25.995 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:25.995 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:25.995 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:25.995 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:25.995 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:25.995 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 19386 2024-09-18T00:19:25.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.996 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:25.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:25.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.699994+0000", 2024-09-18T00:19:25.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:25.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.699994+0000", 2024-09-18T00:19:25.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:25.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:25.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.699996+0000", 2024-09-18T00:19:25.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2330000000000001e-06 2024-09-18T00:19:25.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:25.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700007+0000", 2024-09-18T00:19:25.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1199000000000001e-05 2024-09-18T00:19:25.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:25.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700039+0000", 2024-09-18T00:19:25.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1362999999999998e-05 2024-09-18T00:19:25.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:25.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700054+0000", 2024-09-18T00:19:25.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4970000000000001e-05 2024-09-18T00:19:25.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:25.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700229+0000", 2024-09-18T00:19:25.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017526600000000001 2024-09-18T00:19:25.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:25.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700488+0000", 2024-09-18T00:19:25.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025918099999999998 2024-09-18T00:19:25.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:25.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750393+0000", 2024-09-18T00:19:25.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049904733 2024-09-18T00:19:25.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:25.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:25.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750476+0000", 2024-09-18T00:19:25.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3413999999999998e-05 2024-09-18T00:19:25.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:25.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750607+0000", 2024-09-18T00:19:26.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130839 2024-09-18T00:19:26.000 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.000 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.000 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.000 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:19387 3.1d 3:bc19744b:::rbd_data.112a7fe29ab4.00000000000000c9:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:26.000 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:53.700113+0000", 2024-09-18T00:19:26.000 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.225648003000003, 2024-09-18T00:19:26.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050693338999999997, 2024-09-18T00:19:26.000 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.000 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.001 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:26.001 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.001 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.001 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.001 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 19387 2024-09-18T00:19:26.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.001 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700113+0000", 2024-09-18T00:19:26.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700113+0000", 2024-09-18T00:19:26.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700115+0000", 2024-09-18T00:19:26.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3599999999999999e-06 2024-09-18T00:19:26.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700130+0000", 2024-09-18T00:19:26.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5396999999999998e-05 2024-09-18T00:19:26.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700147+0000", 2024-09-18T00:19:26.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7255e-05 2024-09-18T00:19:26.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.700158+0000", 2024-09-18T00:19:26.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.079e-05 2024-09-18T00:19:26.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.707161+0000", 2024-09-18T00:19:26.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070025820000000003 2024-09-18T00:19:26.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.707407+0000", 2024-09-18T00:19:26.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024667100000000001 2024-09-18T00:19:26.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:26.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750638+0000", 2024-09-18T00:19:26.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043230365 2024-09-18T00:19:26.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:26.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750704+0000", 2024-09-18T00:19:26.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6806999999999999e-05 2024-09-18T00:19:26.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:53.750807+0000", 2024-09-18T00:19:26.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102112 2024-09-18T00:19:26.005 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.005 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.005 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.005 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:22270 3.14 3:2badd31c:::rbd_data.112a948d5956.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:26.005 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.908651+0000", 2024-09-18T00:19:26.006 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.017110295999998, 2024-09-18T00:19:26.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.081875197999999996, 2024-09-18T00:19:26.006 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.006 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.006 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:26.006 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.006 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.006 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.006 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 22270 2024-09-18T00:19:26.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.006 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908651+0000", 2024-09-18T00:19:26.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908651+0000", 2024-09-18T00:19:26.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908653+0000", 2024-09-18T00:19:26.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2290000000000002e-06 2024-09-18T00:19:26.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908665+0000", 2024-09-18T00:19:26.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.225e-05 2024-09-18T00:19:26.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908697+0000", 2024-09-18T00:19:26.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1306000000000001e-05 2024-09-18T00:19:26.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908712+0000", 2024-09-18T00:19:26.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5466000000000001e-05 2024-09-18T00:19:26.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.908889+0000", 2024-09-18T00:19:26.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017653999999999999 2024-09-18T00:19:26.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.909023+0000", 2024-09-18T00:19:26.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013473700000000001 2024-09-18T00:19:26.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:26.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.913637+0000", 2024-09-18T00:19:26.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046132600000000001 2024-09-18T00:19:26.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:26.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990060+0000", 2024-09-18T00:19:26.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.076422834999999995 2024-09-18T00:19:26.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:26.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990206+0000", 2024-09-18T00:19:26.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014591499999999999 2024-09-18T00:19:26.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:26.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990391+0000", 2024-09-18T00:19:26.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018554300000000001 2024-09-18T00:19:26.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990526+0000", 2024-09-18T00:19:26.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000135117 2024-09-18T00:19:26.011 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.011 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.012 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.012 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:22270 3.14 e64/32 ondisk, result = 0)", 2024-09-18T00:19:26.012 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.926819+0000", 2024-09-18T00:19:26.012 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.998942031999999, 2024-09-18T00:19:26.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.063725764000000004, 2024-09-18T00:19:26.012 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.012 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.012 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.012 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926819+0000", 2024-09-18T00:19:26.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926819+0000", 2024-09-18T00:19:26.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926821+0000", 2024-09-18T00:19:26.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5540000000000001e-06 2024-09-18T00:19:26.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926828+0000", 2024-09-18T00:19:26.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9260000000000003e-06 2024-09-18T00:19:26.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926857+0000", 2024-09-18T00:19:26.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9357e-05 2024-09-18T00:19:26.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.926878+0000", 2024-09-18T00:19:26.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0571999999999999e-05 2024-09-18T00:19:26.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990155+0000", 2024-09-18T00:19:26.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.063277618999999993 2024-09-18T00:19:26.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990187+0000", 2024-09-18T00:19:26.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1322999999999998e-05 2024-09-18T00:19:26.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.990545+0000", 2024-09-18T00:19:26.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00035841300000000003 2024-09-18T00:19:26.016 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.016 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.016 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.016 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:22272 3.3 3:c617f123:::rbd_data.112a948d5956.0000000000000009:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:26.016 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:26.016 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.934663569000001, 2024-09-18T00:19:26.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10752577300000001, 2024-09-18T00:19:26.016 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.016 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.016 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:26.017 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.017 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.017 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.017 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 22272 2024-09-18T00:19:26.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.017 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:26.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:26.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991102+0000", 2024-09-18T00:19:26.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9369999999999997e-06 2024-09-18T00:19:26.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991118+0000", 2024-09-18T00:19:26.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6396e-05 2024-09-18T00:19:26.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991150+0000", 2024-09-18T00:19:26.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2132999999999999e-05 2024-09-18T00:19:26.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991170+0000", 2024-09-18T00:19:26.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0085000000000002e-05 2024-09-18T00:19:26.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991370+0000", 2024-09-18T00:19:26.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019963700000000001 2024-09-18T00:19:26.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991518+0000", 2024-09-18T00:19:26.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000148424 2024-09-18T00:19:26.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:26.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.998850+0000", 2024-09-18T00:19:26.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073315480000000002 2024-09-18T00:19:26.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:26.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098178+0000", 2024-09-18T00:19:26.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.099328067000000006 2024-09-18T00:19:26.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:26.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098314+0000", 2024-09-18T00:19:26.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000135678 2024-09-18T00:19:26.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:26.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098498+0000", 2024-09-18T00:19:26.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018479600000000001 2024-09-18T00:19:26.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098623+0000", 2024-09-18T00:19:26.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000125072 2024-09-18T00:19:26.022 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.022 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.022 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.023 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:22272 3.3 e64/23 ondisk, result = 0)", 2024-09-18T00:19:26.023 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:58.023226+0000", 2024-09-18T00:19:26.023 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.902535274000002, 2024-09-18T00:19:26.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.075416677000000001, 2024-09-18T00:19:26.023 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.023 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.023 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.023 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023226+0000", 2024-09-18T00:19:26.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023226+0000", 2024-09-18T00:19:26.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023228+0000", 2024-09-18T00:19:26.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0600000000000002e-06 2024-09-18T00:19:26.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023237+0000", 2024-09-18T00:19:26.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6430000000000001e-06 2024-09-18T00:19:26.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023271+0000", 2024-09-18T00:19:26.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4091999999999998e-05 2024-09-18T00:19:26.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.023293+0000", 2024-09-18T00:19:26.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2103000000000002e-05 2024-09-18T00:19:26.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098272+0000", 2024-09-18T00:19:26.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.074979263000000004 2024-09-18T00:19:26.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098297+0000", 2024-09-18T00:19:26.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5162999999999999e-05 2024-09-18T00:19:26.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098643+0000", 2024-09-18T00:19:26.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034535300000000001 2024-09-18T00:19:26.027 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.027 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.027 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.027 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:23689 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-18T00:19:26.027 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:26.027 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.073338791000001, 2024-09-18T00:19:26.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9562625809999998, 2024-09-18T00:19:26.028 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.028 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.028 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:26.028 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.028 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.028 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.028 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 23689 2024-09-18T00:19:26.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.028 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:26.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:26.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852423+0000", 2024-09-18T00:19:26.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9999999999999995e-07 2024-09-18T00:19:26.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852429+0000", 2024-09-18T00:19:26.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3279999999999996e-06 2024-09-18T00:19:26.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852442+0000", 2024-09-18T00:19:26.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3052e-05 2024-09-18T00:19:26.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852448+0000", 2024-09-18T00:19:26.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2779999999999999e-06 2024-09-18T00:19:26.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852523+0000", 2024-09-18T00:19:26.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4463999999999997e-05 2024-09-18T00:19:26.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:26.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852538+0000", 2024-09-18T00:19:26.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5466999999999999e-05 2024-09-18T00:19:26.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793478+0000", 2024-09-18T00:19:26.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.94094012699999996 2024-09-18T00:19:26.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:26.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793494+0000", 2024-09-18T00:19:26.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5807999999999998e-05 2024-09-18T00:19:26.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.804875+0000", 2024-09-18T00:19:26.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0113811939999999 2024-09-18T00:19:26.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805359+0000", 2024-09-18T00:19:26.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000484248 2024-09-18T00:19:26.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-18T00:19:26.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805872+0000", 2024-09-18T00:19:26.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00051218100000000001 2024-09-18T00:19:26.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:26.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.807790+0000", 2024-09-18T00:19:26.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001918763 2024-09-18T00:19:26.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:26.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808242+0000", 2024-09-18T00:19:26.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045126200000000002 2024-09-18T00:19:26.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:26.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808428+0000", 2024-09-18T00:19:26.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018633000000000001 2024-09-18T00:19:26.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808685+0000", 2024-09-18T00:19:26.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025707900000000001 2024-09-18T00:19:26.036 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.036 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.036 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.036 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.036 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.036 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:26.038 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:19:26.038 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:26.033+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:26.039 INFO:teuthology.orchestra.run.smithi097.stdout:5114 trunc from 0xece9102 to 0x62f57ef 2024-09-18T00:19:26.039 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:26.039 INFO:teuthology.orchestra.run.smithi097.stdout:5116 writesame 0x733e384 thru 0x734bf49 data_size 0x3(0xdbc6 bytes) 2024-09-18T00:19:26.039 INFO:teuthology.orchestra.run.smithi097.stdout:5117 trunc from 0x734bf4a to 0x93a50a7 2024-09-18T00:19:26.039 INFO:teuthology.orchestra.run.smithi097.stdout:5118 read 0x9376ebe thru 0x9385777 (0xe8ba bytes) 2024-09-18T00:19:26.039 INFO:teuthology.orchestra.run.smithi097.stdout:5119 read 0x4a763eb thru 0x4a84e4d (0xea63 bytes) 2024-09-18T00:19:26.039 INFO:teuthology.orchestra.run.smithi097.stdout:5121 trunc from 0x93a50a7 to 0x899020d 2024-09-18T00:19:26.039 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:26.039 INFO:teuthology.orchestra.run.smithi097.stdout:5124 writesame 0x8889d70 thru 0x888a97c data_size 0x1(0xc0d bytes) 2024-09-18T00:19:26.040 INFO:teuthology.orchestra.run.smithi097.stdout:5125 write 0x43485fe thru 0x43493b1 (0xdb4 bytes) 2024-09-18T00:19:26.138 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:26.137+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:26.154 INFO:teuthology.orchestra.run.smithi097.stdout:5126 read 0x30a0a14 thru 0x30a8ee6 (0x84d3 bytes) 2024-09-18T00:19:26.157 INFO:teuthology.orchestra.run.smithi097.stdout:5127 writesame 0x6fe6ae0 thru 0x6fe91cc data_size 0x1(0x26ed bytes) 2024-09-18T00:19:26.176 INFO:teuthology.orchestra.run.smithi097.stdout:5128 trunc from 0x899020d to 0x6c9d0cc 2024-09-18T00:19:26.203 INFO:teuthology.orchestra.run.smithi097.stdout:5129 read 0x2fa2445 thru 0x2fa54db (0x3097 bytes) 2024-09-18T00:19:26.204 INFO:teuthology.orchestra.run.smithi097.stdout:5130 read 0x4ca4c7b thru 0x4caf12d (0xa4b3 bytes) 2024-09-18T00:19:26.207 INFO:teuthology.orchestra.run.smithi097.stdout:5132 read 0x5f4bb95 thru 0x5f59d64 (0xe1d0 bytes) 2024-09-18T00:19:26.213 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:26.213 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:26.213 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:26.213 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:26.227 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:19:26.237 INFO:teuthology.orchestra.run.smithi097.stdout:5133 writesame 0xc7c07bc thru 0xc7c38f1 data_size 0x1(0x3136 bytes) 2024-09-18T00:19:26.237 INFO:teuthology.orchestra.run.smithi097.stdout:5134 write 0x8d3c8bd thru 0x8d490b8 (0xc7fc bytes) 2024-09-18T00:19:26.252 INFO:teuthology.orchestra.run.smithi097.stdout:5135 writesame 0x7eb188c thru 0x7eb2f55 data_size 0x1(0x16ca bytes) 2024-09-18T00:19:26.261 INFO:teuthology.orchestra.run.smithi097.stdout:5137 read 0x6b501eb thru 0x6b53415 (0x322b bytes) 2024-09-18T00:19:26.262 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:26.262 INFO:teuthology.orchestra.run.smithi097.stdout:5144 read 0x8aef257 thru 0x8af3e66 (0x4c10 bytes) 2024-09-18T00:19:26.263 INFO:teuthology.orchestra.run.smithi097.stdout:5145 read 0x6a8b8a6 thru 0x6a8efd6 (0x3731 bytes) 2024-09-18T00:19:26.264 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:26.264 INFO:teuthology.orchestra.run.smithi097.stdout:5147 write 0x2fc7411 thru 0x2fd03fb (0x8feb bytes) 2024-09-18T00:19:26.270 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:26.269+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:26.340 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:26.337+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:26.408 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:26.408 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:26.408 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:26.408 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:26.408 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:26.425 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:19:26.466 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:26.465+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:26.542 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:26.537+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:26.586 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:26.586 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:26.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2229 3.10 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 66.319927628000002, 2024-09-18T00:19:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048730086999999998, 2024-09-18T00:19:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2229 2024-09-18T00:19:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263362+0000", 2024-09-18T00:19:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3869999999999999e-06 2024-09-18T00:19:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263381+0000", 2024-09-18T00:19:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9153000000000001e-05 2024-09-18T00:19:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263418+0000", 2024-09-18T00:19:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6346000000000001e-05 2024-09-18T00:19:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263435+0000", 2024-09-18T00:19:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7309999999999999e-05 2024-09-18T00:19:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263589+0000", 2024-09-18T00:19:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015451199999999999 2024-09-18T00:19:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263902+0000", 2024-09-18T00:19:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031291799999999997 2024-09-18T00:19:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.265995+0000", 2024-09-18T00:19:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020931809999999999 2024-09-18T00:19:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311788+0000", 2024-09-18T00:19:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045792078 2024-09-18T00:19:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311903+0000", 2024-09-18T00:19:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011510699999999999 2024-09-18T00:19:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312009+0000", 2024-09-18T00:19:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010630599999999999 2024-09-18T00:19:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312089+0000", 2024-09-18T00:19:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9789000000000005e-05 2024-09-18T00:19:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2229 3.10 e36/23 ondisk, result = 0)", 2024-09-18T00:19:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 66.300781968999999, 2024-09-18T00:19:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029600955000000002, 2024-09-18T00:19:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282507+0000", 2024-09-18T00:19:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9160000000000001e-06 2024-09-18T00:19:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282522+0000", 2024-09-18T00:19:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5e-05 2024-09-18T00:19:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282558+0000", 2024-09-18T00:19:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5694000000000001e-05 2024-09-18T00:19:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282580+0000", 2024-09-18T00:19:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2323e-05 2024-09-18T00:19:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311873+0000", 2024-09-18T00:19:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029293013999999999 2024-09-18T00:19:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311892+0000", 2024-09-18T00:19:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8556e-05 2024-09-18T00:19:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312105+0000", 2024-09-18T00:19:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021345199999999999 2024-09-18T00:19:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5714 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.273079465999999, 2024-09-18T00:19:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027448727999999999, 2024-09-18T00:19:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5714 2024-09-18T00:19:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310211+0000", 2024-09-18T00:19:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5639999999999998e-06 2024-09-18T00:19:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312678+0000", 2024-09-18T00:19:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002466761 2024-09-18T00:19:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312746+0000", 2024-09-18T00:19:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8046000000000005e-05 2024-09-18T00:19:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312774+0000", 2024-09-18T00:19:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7813e-05 2024-09-18T00:19:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312927+0000", 2024-09-18T00:19:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015328499999999999 2024-09-18T00:19:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313214+0000", 2024-09-18T00:19:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028683099999999998 2024-09-18T00:19:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313708+0000", 2024-09-18T00:19:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049379100000000004 2024-09-18T00:19:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336895+0000", 2024-09-18T00:19:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023186801 2024-09-18T00:19:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337454+0000", 2024-09-18T00:19:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055979399999999996 2024-09-18T00:19:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337571+0000", 2024-09-18T00:19:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116821 2024-09-18T00:19:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337656+0000", 2024-09-18T00:19:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4221000000000006e-05 2024-09-18T00:19:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6801 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 60.546243017999998, 2024-09-18T00:19:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051953602000000002, 2024-09-18T00:19:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6801 2024-09-18T00:19:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037046+0000", 2024-09-18T00:19:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.19e-06 2024-09-18T00:19:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037062+0000", 2024-09-18T00:19:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586999999999998e-05 2024-09-18T00:19:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037096+0000", 2024-09-18T00:19:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4112000000000001e-05 2024-09-18T00:19:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037120+0000", 2024-09-18T00:19:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.355e-05 2024-09-18T00:19:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037246+0000", 2024-09-18T00:19:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012652200000000001 2024-09-18T00:19:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037385+0000", 2024-09-18T00:19:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013924599999999999 2024-09-18T00:19:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.040881+0000", 2024-09-18T00:19:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034954830000000002 2024-09-18T00:19:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088794+0000", 2024-09-18T00:19:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047912897000000003 2024-09-18T00:19:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088828+0000", 2024-09-18T00:19:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3880000000000001e-05 2024-09-18T00:19:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088916+0000", 2024-09-18T00:19:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.865e-05 2024-09-18T00:19:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088997+0000", 2024-09-18T00:19:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0484999999999999e-05 2024-09-18T00:19:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:6801 3.10 e42/23 ondisk, result = 0)", 2024-09-18T00:19:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 60.532435608, 2024-09-18T00:19:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037960394000000001, 2024-09-18T00:19:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050852+0000", 2024-09-18T00:19:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:19:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050860+0000", 2024-09-18T00:19:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7729999999999999e-06 2024-09-18T00:19:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050891+0000", 2024-09-18T00:19:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0617000000000002e-05 2024-09-18T00:19:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050910+0000", 2024-09-18T00:19:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.944e-05 2024-09-18T00:19:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088766+0000", 2024-09-18T00:19:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037855633999999999 2024-09-18T00:19:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088784+0000", 2024-09-18T00:19:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7767000000000001e-05 2024-09-18T00:19:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088811+0000", 2024-09-18T00:19:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7382999999999999e-05 2024-09-18T00:19:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12594 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.19165907, 2024-09-18T00:19:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026492600000000002, 2024-09-18T00:19:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12594 2024-09-18T00:19:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391629+0000", 2024-09-18T00:19:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.86e-06 2024-09-18T00:19:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391637+0000", 2024-09-18T00:19:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3999999999999992e-06 2024-09-18T00:19:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391664+0000", 2024-09-18T00:19:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6536000000000001e-05 2024-09-18T00:19:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391681+0000", 2024-09-18T00:19:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6563e-05 2024-09-18T00:19:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417791+0000", 2024-09-18T00:19:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026110936000000001 2024-09-18T00:19:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417933+0000", 2024-09-18T00:19:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014112099999999999 2024-09-18T00:19:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418120+0000", 2024-09-18T00:19:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018718399999999999 2024-09-18T00:19:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12595 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.191582871999998, 2024-09-18T00:19:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026819428999999999, 2024-09-18T00:19:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12595 2024-09-18T00:19:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391705+0000", 2024-09-18T00:19:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3999999999999999e-06 2024-09-18T00:19:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391711+0000", 2024-09-18T00:19:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7459999999999997e-06 2024-09-18T00:19:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391733+0000", 2024-09-18T00:19:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2923000000000001e-05 2024-09-18T00:19:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391748+0000", 2024-09-18T00:19:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4790000000000001e-05 2024-09-18T00:19:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418180+0000", 2024-09-18T00:19:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026431888000000001 2024-09-18T00:19:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418328+0000", 2024-09-18T00:19:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000147725 2024-09-18T00:19:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418523+0000", 2024-09-18T00:19:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000194957 2024-09-18T00:19:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12596 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.191519904000003, 2024-09-18T00:19:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027220296000000001, 2024-09-18T00:19:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12596 2024-09-18T00:19:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391768+0000", 2024-09-18T00:19:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.283e-06 2024-09-18T00:19:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391773+0000", 2024-09-18T00:19:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3970000000000002e-06 2024-09-18T00:19:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391795+0000", 2024-09-18T00:19:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1835e-05 2024-09-18T00:19:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391809+0000", 2024-09-18T00:19:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4107000000000001e-05 2024-09-18T00:19:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418582+0000", 2024-09-18T00:19:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026772918999999999 2024-09-18T00:19:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418780+0000", 2024-09-18T00:19:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000198042 2024-09-18T00:19:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418987+0000", 2024-09-18T00:19:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000206713 2024-09-18T00:19:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12597 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.191459385999998, 2024-09-18T00:19:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027612714, 2024-09-18T00:19:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12597 2024-09-18T00:19:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391828+0000", 2024-09-18T00:19:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.203e-06 2024-09-18T00:19:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391834+0000", 2024-09-18T00:19:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6529999999999999e-06 2024-09-18T00:19:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391856+0000", 2024-09-18T00:19:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2195999999999999e-05 2024-09-18T00:19:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391870+0000", 2024-09-18T00:19:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4063000000000001e-05 2024-09-18T00:19:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419049+0000", 2024-09-18T00:19:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027178751000000001 2024-09-18T00:19:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419239+0000", 2024-09-18T00:19:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019039400000000001 2024-09-18T00:19:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419440+0000", 2024-09-18T00:19:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000200454 2024-09-18T00:19:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12598 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.191396335, 2024-09-18T00:19:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028269056000000001, 2024-09-18T00:19:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12598 2024-09-18T00:19:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391891+0000", 2024-09-18T00:19:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.257e-06 2024-09-18T00:19:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391897+0000", 2024-09-18T00:19:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9560000000000002e-06 2024-09-18T00:19:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391920+0000", 2024-09-18T00:19:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3025999999999999e-05 2024-09-18T00:19:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391935+0000", 2024-09-18T00:19:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4853e-05 2024-09-18T00:19:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419527+0000", 2024-09-18T00:19:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027592378000000001 2024-09-18T00:19:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419681+0000", 2024-09-18T00:19:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015316 2024-09-18T00:19:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420159+0000", 2024-09-18T00:19:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000478426 2024-09-18T00:19:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12599 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.19133248, 2024-09-18T00:19:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028649857000000001, 2024-09-18T00:19:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12599 2024-09-18T00:19:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391955+0000", 2024-09-18T00:19:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-06 2024-09-18T00:19:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391961+0000", 2024-09-18T00:19:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7100000000000004e-06 2024-09-18T00:19:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391982+0000", 2024-09-18T00:19:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1492e-05 2024-09-18T00:19:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391996+0000", 2024-09-18T00:19:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4194e-05 2024-09-18T00:19:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420237+0000", 2024-09-18T00:19:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028240818000000001 2024-09-18T00:19:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420430+0000", 2024-09-18T00:19:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019228299999999999 2024-09-18T00:19:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420604+0000", 2024-09-18T00:19:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000174117 2024-09-18T00:19:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12600 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 270336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.191271485999998, 2024-09-18T00:19:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028964586, 2024-09-18T00:19:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12600 2024-09-18T00:19:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392016+0000", 2024-09-18T00:19:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.31e-06 2024-09-18T00:19:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392022+0000", 2024-09-18T00:19:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5600000000000001e-06 2024-09-18T00:19:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392045+0000", 2024-09-18T00:19:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3595999999999999e-05 2024-09-18T00:19:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392060+0000", 2024-09-18T00:19:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4727e-05 2024-09-18T00:19:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420669+0000", 2024-09-18T00:19:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028608855999999998 2024-09-18T00:19:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420813+0000", 2024-09-18T00:19:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014429500000000001 2024-09-18T00:19:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420979+0000", 2024-09-18T00:19:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016624200000000001 2024-09-18T00:19:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12601 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.191207581, 2024-09-18T00:19:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029292832000000001, 2024-09-18T00:19:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12601 2024-09-18T00:19:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392080+0000", 2024-09-18T00:19:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392086+0000", 2024-09-18T00:19:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7570000000000001e-06 2024-09-18T00:19:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392105+0000", 2024-09-18T00:19:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9415999999999998e-05 2024-09-18T00:19:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392132+0000", 2024-09-18T00:19:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7019e-05 2024-09-18T00:19:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421049+0000", 2024-09-18T00:19:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028916659000000001 2024-09-18T00:19:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421204+0000", 2024-09-18T00:19:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155224 2024-09-18T00:19:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421372+0000", 2024-09-18T00:19:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016754699999999999 2024-09-18T00:19:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12602 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.191134949999999, 2024-09-18T00:19:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029612833000000002, 2024-09-18T00:19:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12602 2024-09-18T00:19:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392152+0000", 2024-09-18T00:19:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0470000000000001e-06 2024-09-18T00:19:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392158+0000", 2024-09-18T00:19:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:19:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392181+0000", 2024-09-18T00:19:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2629e-05 2024-09-18T00:19:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392222+0000", 2024-09-18T00:19:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0772000000000002e-05 2024-09-18T00:19:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421441+0000", 2024-09-18T00:19:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029219327 2024-09-18T00:19:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421608+0000", 2024-09-18T00:19:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000166813 2024-09-18T00:19:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421764+0000", 2024-09-18T00:19:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015628500000000001 2024-09-18T00:19:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12603 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.191044556000001, 2024-09-18T00:19:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029902524, 2024-09-18T00:19:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12603 2024-09-18T00:19:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392243+0000", 2024-09-18T00:19:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392260+0000", 2024-09-18T00:19:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6860000000000001e-05 2024-09-18T00:19:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392282+0000", 2024-09-18T00:19:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2713000000000001e-05 2024-09-18T00:19:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392297+0000", 2024-09-18T00:19:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4375e-05 2024-09-18T00:19:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421830+0000", 2024-09-18T00:19:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029532668000000002 2024-09-18T00:19:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421985+0000", 2024-09-18T00:19:26.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155613 2024-09-18T00:19:26.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422144+0000", 2024-09-18T00:19:26.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000159102 2024-09-18T00:19:26.669 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.669 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.669 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.669 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12604 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:26.670 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:26.670 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.190969504999998, 2024-09-18T00:19:26.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030228695, 2024-09-18T00:19:26.670 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.670 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.670 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.670 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.670 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.670 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.670 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12604 2024-09-18T00:19:26.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.670 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:26.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:26.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392318+0000", 2024-09-18T00:19:26.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:26.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392337+0000", 2024-09-18T00:19:26.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:19:26.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392364+0000", 2024-09-18T00:19:26.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7288999999999999e-05 2024-09-18T00:19:26.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392379+0000", 2024-09-18T00:19:26.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:19:26.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422209+0000", 2024-09-18T00:19:26.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029829050999999999 2024-09-18T00:19:26.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422388+0000", 2024-09-18T00:19:26.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017988699999999999 2024-09-18T00:19:26.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422545+0000", 2024-09-18T00:19:26.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015701199999999999 2024-09-18T00:19:26.674 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.674 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.674 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.674 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12605 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:26.674 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:26.674 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.190884943999997, 2024-09-18T00:19:26.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030488674, 2024-09-18T00:19:26.674 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.674 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.674 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.674 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.674 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.674 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.675 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12605 2024-09-18T00:19:26.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.675 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:26.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:26.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392403+0000", 2024-09-18T00:19:26.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.883e-06 2024-09-18T00:19:26.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392421+0000", 2024-09-18T00:19:26.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7526000000000001e-05 2024-09-18T00:19:26.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392450+0000", 2024-09-18T00:19:26.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8972e-05 2024-09-18T00:19:26.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392488+0000", 2024-09-18T00:19:26.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8522000000000001e-05 2024-09-18T00:19:26.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422614+0000", 2024-09-18T00:19:26.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030125945000000001 2024-09-18T00:19:26.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422752+0000", 2024-09-18T00:19:26.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000138178 2024-09-18T00:19:26.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422890+0000", 2024-09-18T00:19:26.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013764800000000001 2024-09-18T00:19:26.678 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.678 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.678 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.678 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12606 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:26.679 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:26.679 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.190776284000002, 2024-09-18T00:19:26.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033727073000000003, 2024-09-18T00:19:26.679 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.679 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.679 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.679 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.679 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.679 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.679 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12606 2024-09-18T00:19:26.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.679 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:26.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:26.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392511+0000", 2024-09-18T00:19:26.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3090000000000001e-06 2024-09-18T00:19:26.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392517+0000", 2024-09-18T00:19:26.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.993e-06 2024-09-18T00:19:26.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392543+0000", 2024-09-18T00:19:26.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6152999999999998e-05 2024-09-18T00:19:26.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392558+0000", 2024-09-18T00:19:26.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4610000000000001e-05 2024-09-18T00:19:26.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.425959+0000", 2024-09-18T00:19:26.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033400484000000001 2024-09-18T00:19:26.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426106+0000", 2024-09-18T00:19:26.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014779800000000001 2024-09-18T00:19:26.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426237+0000", 2024-09-18T00:19:26.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013072600000000001 2024-09-18T00:19:26.682 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.683 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.683 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.683 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17562 3.a 3:5578626c:::rbd_data.112a6efb7564.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T00:19:26.683 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:26.683 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.625311123000003, 2024-09-18T00:19:26.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052733453999999999, 2024-09-18T00:19:26.683 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.683 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.683 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:26.683 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:26.683 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:26.683 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:26.683 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17562 2024-09-18T00:19:26.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.684 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:26.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:26.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957977+0000", 2024-09-18T00:19:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.119e-06 2024-09-18T00:19:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957988+0000", 2024-09-18T00:19:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0557000000000001e-05 2024-09-18T00:19:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958017+0000", 2024-09-18T00:19:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9156e-05 2024-09-18T00:19:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958036+0000", 2024-09-18T00:19:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8919e-05 2024-09-18T00:19:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958144+0000", 2024-09-18T00:19:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107905 2024-09-18T00:19:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958284+0000", 2024-09-18T00:19:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140216 2024-09-18T00:19:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.960572+0000", 2024-09-18T00:19:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022881860000000002 2024-09-18T00:19:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010537+0000", 2024-09-18T00:19:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049964664999999998 2024-09-18T00:19:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010590+0000", 2024-09-18T00:19:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3094999999999999e-05 2024-09-18T00:19:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010642+0000", 2024-09-18T00:19:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2073000000000001e-05 2024-09-18T00:19:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010709+0000", 2024-09-18T00:19:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6563000000000006e-05 2024-09-18T00:19:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17562 3.a e60/23 ondisk, result = 0)", 2024-09-18T00:19:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.615107752, 2024-09-18T00:19:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042539307999999998, 2024-09-18T00:19:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968180+0000", 2024-09-18T00:19:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-18T00:19:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968188+0000", 2024-09-18T00:19:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7419999999999997e-06 2024-09-18T00:19:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968217+0000", 2024-09-18T00:19:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8926999999999999e-05 2024-09-18T00:19:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968231+0000", 2024-09-18T00:19:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4168999999999999e-05 2024-09-18T00:19:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010575+0000", 2024-09-18T00:19:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042343923999999998 2024-09-18T00:19:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010584+0000", 2024-09-18T00:19:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-18T00:19:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010718+0000", 2024-09-18T00:19:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013344499999999999 2024-09-18T00:19:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:26.694 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:26.694 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:19:26.697 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:19:26.698 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:26.689+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:26.796 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:26.793+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:26.850 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:26.866 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:26.867 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:26.867 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:26.867 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:26.867 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:26.867 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:26.881 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:19:26.884 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:26.884 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:26.902 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:26.902 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:26.920 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:26.922 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:26.921+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:26.939 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:26.939 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:26.957 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:26.957 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:26.961 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:26.961 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:26.998 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:26.997+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:27.046 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:27.046 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:27.060 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:19:27.099 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:27.097+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:27.200 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:27.197+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:27.225 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:27.225 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:27.239 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:19:27.326 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:27.325+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:27.402 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:27.397+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:27.404 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:27.404 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:27.419 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:19:27.439 INFO:teuthology.orchestra.run.smithi097.stdout:5148 punch from 0x1c2d277 to 0x1c36186, (0x8f0f bytes) 2024-09-18T00:19:27.439 INFO:teuthology.orchestra.run.smithi097.stdout:5149 punch from 0x9072c0d to 0x907df1f, (0xb312 bytes) 2024-09-18T00:19:27.440 INFO:teuthology.orchestra.run.smithi097.stdout:5150 write 0x8f80eb4 thru 0x8f8e5a3 (0xd6f0 bytes) 2024-09-18T00:19:27.442 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:19:27.445 INFO:teuthology.orchestra.run.smithi097.stdout:5152 read 0x8b6f6fe thru 0x8b71f81 (0x2884 bytes) 2024-09-18T00:19:27.473 INFO:teuthology.orchestra.run.smithi097.stdout:5153 trunc from 0xc7c38f2 to 0x23de5f6 2024-09-18T00:19:27.502 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:27.497+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:27.593 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:27.593 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:27.603 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:27.597+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:27.607 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:19:27.704 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:27.701+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:27.724 INFO:teuthology.orchestra.run.smithi097.stdout:5154 trunc from 0x23de5f6 to 0xa0980c1 2024-09-18T00:19:27.731 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:27.731 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:27.735 INFO:teuthology.orchestra.run.smithi097.stdout:5157 punch from 0x8e9c91f to 0x8ea61ee, (0x98cf bytes) 2024-09-18T00:19:27.741 INFO:teuthology.orchestra.run.smithi097.stdout:5158 trunc from 0xa0980c1 to 0xce97e98 2024-09-18T00:19:27.746 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:19:27.749 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:27.749 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:27.749 INFO:teuthology.orchestra.run.smithi097.stdout:5161 read 0xa38b0db thru 0xa396c61 (0xbb87 bytes) 2024-09-18T00:19:27.750 INFO:teuthology.orchestra.run.smithi097.stdout:5162 read 0x84deb43 thru 0x84e199d (0x2e5b bytes) 2024-09-18T00:19:27.750 INFO:teuthology.orchestra.run.smithi097.stdout:5163 punch from 0x2a23711 to 0x2a32b57, (0xf446 bytes) 2024-09-18T00:19:27.750 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:27.750 INFO:teuthology.orchestra.run.smithi097.stdout:5166 read 0x50c45e0 thru 0x50d1498 (0xceb9 bytes) 2024-09-18T00:19:27.750 INFO:teuthology.orchestra.run.smithi097.stdout:5167 writesame 0xc044986 thru 0xc04cd70 data_size 0x1(0x83eb bytes) 2024-09-18T00:19:27.801 INFO:teuthology.orchestra.run.smithi097.stdout:5168 writesame 0xd46cc33 thru 0xd46f28d data_size 0x3(0x265b bytes) 2024-09-18T00:19:27.801 INFO:teuthology.orchestra.run.smithi097.stdout:5169 writesame 0xbc126d thru 0xbc2af1 data_size 0x1(0x1885 bytes) 2024-09-18T00:19:27.804 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:27.801+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:27.876 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:27.876 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:27.890 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:19:27.937 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:27.933+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:27.990 INFO:teuthology.orchestra.run.smithi097.stdout:5170 trunc from 0xd46f28e to 0x3d99e6f 2024-09-18T00:19:28.006 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:28.001+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:28.033 INFO:teuthology.orchestra.run.smithi097.stdout:5171 write 0x9d89b93 thru 0x9d97758 (0xdbc6 bytes) 2024-09-18T00:19:28.059 INFO:teuthology.orchestra.run.smithi097.stdout:5172 trunc from 0x9d97759 to 0xe9ac99c 2024-09-18T00:19:28.069 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:28.069 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:28.073 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:28.073 INFO:teuthology.orchestra.run.smithi097.stdout:5175 write 0xd8c526f thru 0xd8d1b94 (0xc926 bytes) 2024-09-18T00:19:28.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:28.094 INFO:teuthology.orchestra.run.smithi097.stdout:5176 write 0x11bc970 thru 0x11c9746 (0xcdd7 bytes) 2024-09-18T00:19:28.137 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:28.133+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:28.208 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:28.201+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:28.236 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:28.236 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:28.254 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:19:28.306 INFO:teuthology.orchestra.run.smithi097.stdout:5177 write 0x44ed6d6 thru 0x44fa66b (0xcf96 bytes) 2024-09-18T00:19:28.309 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:28.305+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:28.328 INFO:teuthology.orchestra.run.smithi097.stdout:5178 trunc from 0xe9ac99c to 0xb245a6f 2024-09-18T00:19:28.353 INFO:teuthology.orchestra.run.smithi097.stdout:5179 read 0xae4573 thru 0xae52d6 (0xd64 bytes) 2024-09-18T00:19:28.354 INFO:teuthology.orchestra.run.smithi097.stdout:5180 write 0x3940648 thru 0x3947124 (0x6add bytes) 2024-09-18T00:19:28.383 INFO:teuthology.orchestra.run.smithi097.stdout:5181 write 0xc65ca7a thru 0xc66679e (0x9d25 bytes) 2024-09-18T00:19:28.396 INFO:teuthology.orchestra.run.smithi097.stdout:5182 read 0xa758eee thru 0xa758f87 (0x9a bytes) 2024-09-18T00:19:28.410 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:28.409+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:28.411 INFO:teuthology.orchestra.run.smithi097.stdout:5184 writesame 0xda3b7a5 thru 0xda40f9a data_size 0x1b(0x57f6 bytes) 2024-09-18T00:19:28.411 INFO:teuthology.orchestra.run.smithi097.stdout:5185 write 0xd462d2a thru 0xd46d857 (0xab2e bytes) 2024-09-18T00:19:28.416 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 7] out_osds: [6, 1] dead_osds: [1] live_osds: [0, 2, 3, 4, 6, 7, 5] 2024-09-18T00:19:28.416 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:19:28.416 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-18T00:19:28.417 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-18T00:19:28.418 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:28.418 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:28.425 INFO:teuthology.orchestra.run.smithi097.stdout:5187 writesame 0xb9a3211 thru 0xb9a73ed data_size 0x1(0x41dd bytes) 2024-09-18T00:19:28.435 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:19:28.447 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is set 2024-09-18T00:19:28.458 INFO:teuthology.orchestra.run.smithi097.stdout:5188 write 0xce6f1d6 thru 0xce7a915 (0xb740 bytes) 2024-09-18T00:19:28.487 INFO:teuthology.orchestra.run.smithi097.stdout:5189 writesame 0xecb0d05 thru 0xecb735a data_size 0x1(0x6656 bytes) 2024-09-18T00:19:28.487 INFO:teuthology.orchestra.run.smithi097.stdout:5191 read 0x7f95b93 thru 0x7f9b061 (0x54cf bytes) 2024-09-18T00:19:28.487 INFO:teuthology.orchestra.run.smithi097.stdout:5194 punch from 0x60fe9a7 to 0x6103e87, (0x54e0 bytes) 2024-09-18T00:19:28.487 INFO:teuthology.orchestra.run.smithi097.stdout:5196 write 0x7747528 thru 0x774a742 (0x321b bytes) 2024-09-18T00:19:28.511 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:28.505+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:28.519 INFO:teuthology.orchestra.run.smithi097.stdout:5197 trunc from 0xecb735b to 0x103744c 2024-09-18T00:19:28.558 INFO:teuthology.orchestra.run.smithi097.stdout:5198 read 0x266a37 thru 0x274fd3 (0xe59d bytes) 2024-09-18T00:19:28.560 INFO:teuthology.orchestra.run.smithi097.stdout:5201 punch from 0xbe2328 to 0xbe8846, (0x651e bytes) 2024-09-18T00:19:28.560 INFO:teuthology.orchestra.run.smithi097.stdout:5202 punch from 0xad0341 to 0xade285, (0xdf44 bytes) 2024-09-18T00:19:28.588 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:28.588 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:28.592 INFO:teuthology.orchestra.run.smithi097.stdout:5204 write 0xe2231ed thru 0xe230a48 (0xd85c bytes) 2024-09-18T00:19:28.601 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-18T00:19:28.605 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:19:28.614 INFO:teuthology.orchestra.run.smithi097.stdout:5205 trunc from 0xe230a49 to 0x91fef5b 2024-09-18T00:19:28.616 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:28.609+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:28.657 INFO:teuthology.orchestra.run.smithi097.stdout:5208 writesame 0xe222a1f thru 0xe232294 data_size 0x3(0xf876 bytes) 2024-09-18T00:19:28.657 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:28.657 INFO:teuthology.orchestra.run.smithi097.stdout:5210 writesame 0x5d6196 thru 0x5e4c63 data_size 0x5(0xeace bytes) 2024-09-18T00:19:28.712 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:28.705+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:28.759 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:28.759 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:28.775 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:19:28.793 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-18T00:19:28.813 INFO:teuthology.orchestra.run.smithi097.stdout:5212 write 0x2a5ca8d thru 0x2a68599 (0xbb0d bytes) 2024-09-18T00:19:28.814 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:28.809+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:28.829 INFO:teuthology.orchestra.run.smithi097.stdout:5213 read 0xae024a3 thru 0xae067c0 (0x431e bytes) 2024-09-18T00:19:28.829 INFO:teuthology.orchestra.run.smithi097.stdout:5214 trunc from 0xe232295 to 0xdf030f8 2024-09-18T00:19:28.840 INFO:teuthology.orchestra.run.smithi097.stdout:5215 write 0x26a4e18 thru 0x26a8643 (0x382c bytes) 2024-09-18T00:19:28.853 INFO:teuthology.orchestra.run.smithi097.stdout:5216 read 0x742c05e thru 0x7431b8a (0x5b2d bytes) 2024-09-18T00:19:28.853 INFO:teuthology.orchestra.run.smithi097.stdout:5218 write 0x72f1f24 thru 0x72fc6f9 (0xa7d6 bytes) 2024-09-18T00:19:28.868 INFO:teuthology.orchestra.run.smithi097.stdout:5219 write 0xbb23203 thru 0xbb252ae (0x20ac bytes) 2024-09-18T00:19:28.877 INFO:teuthology.orchestra.run.smithi097.stdout:5221 writesame 0x6e5344a thru 0x6e5d61e data_size 0x1(0xa1d5 bytes) 2024-09-18T00:19:28.914 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:28.913+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:28.925 INFO:teuthology.orchestra.run.smithi097.stdout:5222 punch from 0xa493196 to 0xa498905, (0x576f bytes) 2024-09-18T00:19:28.925 INFO:teuthology.orchestra.run.smithi097.stdout:5225 read 0x629564c thru 0x6296d44 (0x16f9 bytes) 2024-09-18T00:19:28.925 INFO:teuthology.orchestra.run.smithi097.stdout:5226 read 0xa21713b thru 0xa21f1ca (0x8090 bytes) 2024-09-18T00:19:28.925 INFO:teuthology.orchestra.run.smithi097.stdout:5227 punch from 0xbdddd67 to 0xbdea32c, (0xc5c5 bytes) 2024-09-18T00:19:28.925 INFO:teuthology.orchestra.run.smithi097.stdout:5228 punch from 0x49873ee to 0x498a95e, (0x3570 bytes) 2024-09-18T00:19:28.925 INFO:teuthology.orchestra.run.smithi097.stdout:5229 writesame 0x81d265b thru 0x81d9dd8 data_size 0x2e(0x777e bytes) 2024-09-18T00:19:28.930 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:28.930 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:28.939 INFO:teuthology.orchestra.run.smithi097.stdout:5230 writesame 0x5728f9f thru 0x572d80c data_size 0x2(0x486e bytes) 2024-09-18T00:19:28.946 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:19:28.953 INFO:teuthology.orchestra.run.smithi097.stdout:5231 read 0x29985e0 thru 0x299ac9d (0x26be bytes) 2024-09-18T00:19:28.954 INFO:teuthology.orchestra.run.smithi097.stdout:5232 write 0xcc3eff8 thru 0xcc4d44a (0xe453 bytes) 2024-09-18T00:19:28.963 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:19:28.967 INFO:teuthology.orchestra.run.smithi097.stdout:5233 write 0x2d9493a thru 0x2da255a (0xdc21 bytes) 2024-09-18T00:19:28.972 INFO:teuthology.orchestra.run.smithi097.stdout:5234 write 0x73a976a thru 0x73af849 (0x60e0 bytes) 2024-09-18T00:19:28.976 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-18T00:19:28.977 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:28.978 INFO:teuthology.orchestra.run.smithi097.stdout:5238 writesame 0x48abd21 thru 0x48b8404 data_size 0x6(0xc6e4 bytes) 2024-09-18T00:19:28.996 INFO:teuthology.orchestra.run.smithi097.stdout:5239 trunc from 0xdf030f8 to 0xbc2bfac 2024-09-18T00:19:29.015 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:29.009+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:29.020 INFO:teuthology.orchestra.run.smithi097.stdout:5241 write 0x47652ae thru 0x476d4c6 (0x8219 bytes) 2024-09-18T00:19:29.049 INFO:teuthology.orchestra.run.smithi097.stdout:5242 write 0xe75d54c thru 0xe75d664 (0x119 bytes) 2024-09-18T00:19:29.056 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:29.056 INFO:teuthology.orchestra.run.smithi097.stdout:5246 punch from 0x84996c5 to 0x849f773, (0x60ae bytes) 2024-09-18T00:19:29.056 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:29.057 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:29.057 INFO:teuthology.orchestra.run.smithi097.stdout:5251 writesame 0xc6fb6d3 thru 0xc702dcf data_size 0x1(0x76fd bytes) 2024-09-18T00:19:29.076 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:29.076 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:29.092 INFO:teuthology.orchestra.run.smithi097.stdout:5252 writesame 0x6c3b52d thru 0x6c48d77 data_size 0x1(0xd84b bytes) 2024-09-18T00:19:29.101 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:19:29.120 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:29.120 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:29.121 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:29.113+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:29.121 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:29.134 INFO:teuthology.orchestra.run.smithi097.stdout:5253 write 0x77fe968 thru 0x780088f (0x1f28 bytes) 2024-09-18T00:19:29.140 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:29.140 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:29.140 INFO:teuthology.orchestra.run.smithi097.stdout:5256 writesame 0xe4928 thru 0xe91ad data_size 0x1(0x4886 bytes) 2024-09-18T00:19:29.145 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:29.145 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:29.150 INFO:teuthology.orchestra.run.smithi097.stdout:5257 writesame 0x4c9ce8a thru 0x4cacae8 data_size 0x1(0xfc5f bytes) 2024-09-18T00:19:29.164 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-18T00:19:29.165 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:29.165 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:29.192 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:29.192 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:29.212 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:29.212 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:29.217 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:29.213+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:29.233 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:29.233 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:29.241 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:29.241 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:29.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:29.241 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:29.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.241 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:19:29.241 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:29.241 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 76.439533381000004, 2024-09-18T00:19:29.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:19:29.242 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.242 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.242 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.242 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.242 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.242 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.242 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:19:29.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.242 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:29.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:29.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:19:29.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:19:29.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:19:29.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:19:29.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:19:29.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:19:29.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:19:29.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:19:29.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:19:29.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:19:29.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:19:29.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:19:29.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:29.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:19:29.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:19:29.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:29.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:19:29.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:19:29.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:29.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:19:29.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:19:29.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:29.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:19:29.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:19:29.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:19:29.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:19:29.247 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.247 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.247 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.247 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:19:29.247 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:29.247 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 76.423325508000005, 2024-09-18T00:19:29.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:19:29.248 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.248 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.248 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:29.248 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:29.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:29.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:19:29.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:29.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:19:29.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:19:29.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:19:29.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:19:29.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:19:29.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:19:29.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:19:29.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:19:29.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:19:29.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:19:29.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:19:29.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:19:29.251 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.251 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.251 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.251 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:29.252 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:29.252 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 63.396284188000003, 2024-09-18T00:19:29.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:19:29.252 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.252 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.252 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.252 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:19:29.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.252 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:29.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:29.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:19:29.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:19:29.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:19:29.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:19:29.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:19:29.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:19:29.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:19:29.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:19:29.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:19:29.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:19:29.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:19:29.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:19:29.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:29.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:19:29.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:19:29.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:29.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:19:29.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:19:29.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:29.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:19:29.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:19:29.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:29.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:19:29.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:19:29.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:19:29.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:19:29.257 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.257 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.257 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.257 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:19:29.257 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:29.257 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 63.375007635999999, 2024-09-18T00:19:29.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:19:29.258 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.258 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.258 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:29.258 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:29.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:29.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:19:29.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:19:29.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:19:29.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:19:29.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:19:29.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:19:29.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:19:29.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:19:29.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:19:29.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:19:29.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:19:29.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:19:29.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:19:29.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:19:29.261 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.261 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.261 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.262 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11460 3.19 3:9e578b6c:::rbd_data.112ab1fa102a.0000000000000153:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:29.262 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:29.262 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 53.972636184000002, 2024-09-18T00:19:29.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041989011999999999, 2024-09-18T00:19:29.262 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.262 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.262 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.262 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.262 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.262 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.262 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11460 2024-09-18T00:19:29.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.263 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:29.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:29.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267067+0000", 2024-09-18T00:19:29.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:19:29.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267088+0000", 2024-09-18T00:19:29.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0908999999999999e-05 2024-09-18T00:19:29.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267133+0000", 2024-09-18T00:19:29.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5232000000000003e-05 2024-09-18T00:19:29.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267157+0000", 2024-09-18T00:19:29.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3833e-05 2024-09-18T00:19:29.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267300+0000", 2024-09-18T00:19:29.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142282 2024-09-18T00:19:29.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267447+0000", 2024-09-18T00:19:29.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147851 2024-09-18T00:19:29.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:29.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308889+0000", 2024-09-18T00:19:29.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041441779999999998 2024-09-18T00:19:29.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:29.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308956+0000", 2024-09-18T00:19:29.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7136000000000002e-05 2024-09-18T00:19:29.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.309053+0000", 2024-09-18T00:19:29.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6552999999999995e-05 2024-09-18T00:19:29.266 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.267 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.267 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.267 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:16947 3.18 3:18b58c7b:::rbd_data.112a6efb7564.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:19:29.267 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:29.267 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 40.622858483000002, 2024-09-18T00:19:29.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.101692484, 2024-09-18T00:19:29.267 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.267 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.267 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.267 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.267 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.267 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.268 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 16947 2024-09-18T00:19:29.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.268 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:29.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:29.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616843+0000", 2024-09-18T00:19:29.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0780000000000001e-06 2024-09-18T00:19:29.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617785+0000", 2024-09-18T00:19:29.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094204100000000002 2024-09-18T00:19:29.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617801+0000", 2024-09-18T00:19:29.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6436999999999999e-05 2024-09-18T00:19:29.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617809+0000", 2024-09-18T00:19:29.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9049999999999997e-06 2024-09-18T00:19:29.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617904+0000", 2024-09-18T00:19:29.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5204999999999994e-05 2024-09-18T00:19:29.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.618071+0000", 2024-09-18T00:19:29.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016640000000000001 2024-09-18T00:19:29.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:29.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.624534+0000", 2024-09-18T00:19:29.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064634499999999999 2024-09-18T00:19:29.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:29.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718299+0000", 2024-09-18T00:19:29.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093764739 2024-09-18T00:19:29.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:29.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718375+0000", 2024-09-18T00:19:29.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6532999999999994e-05 2024-09-18T00:19:29.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:29.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718466+0000", 2024-09-18T00:19:29.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0581000000000006e-05 2024-09-18T00:19:29.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718534+0000", 2024-09-18T00:19:29.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8114999999999994e-05 2024-09-18T00:19:29.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.273 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.273 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.273 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:16947 3.18 e59/23 ondisk, result = 0)", 2024-09-18T00:19:29.273 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:29.273 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 40.597641789000001, 2024-09-18T00:19:29.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076488069000000006, 2024-09-18T00:19:29.273 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.273 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.273 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:29.273 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:29.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:29.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642060+0000", 2024-09-18T00:19:29.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6640000000000001e-06 2024-09-18T00:19:29.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642071+0000", 2024-09-18T00:19:29.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1204e-05 2024-09-18T00:19:29.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642091+0000", 2024-09-18T00:19:29.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0313999999999999e-05 2024-09-18T00:19:29.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642103+0000", 2024-09-18T00:19:29.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1214e-05 2024-09-18T00:19:29.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718350+0000", 2024-09-18T00:19:29.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076247681999999997 2024-09-18T00:19:29.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718366+0000", 2024-09-18T00:19:29.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5420000000000001e-05 2024-09-18T00:19:29.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718546+0000", 2024-09-18T00:19:29.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000180571 2024-09-18T00:19:29.277 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.277 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.277 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.277 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18686 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:29.277 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:29.277 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.290964418999998, 2024-09-18T00:19:29.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041132573999999998, 2024-09-18T00:19:29.278 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.278 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.278 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.278 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.278 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18686 2024-09-18T00:19:29.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.278 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:29.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:29.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948737+0000", 2024-09-18T00:19:29.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3769999999999999e-06 2024-09-18T00:19:29.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949968+0000", 2024-09-18T00:19:29.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001230493 2024-09-18T00:19:29.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949993+0000", 2024-09-18T00:19:29.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5282e-05 2024-09-18T00:19:29.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950011+0000", 2024-09-18T00:19:29.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8199999999999999e-05 2024-09-18T00:19:29.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950180+0000", 2024-09-18T00:19:29.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016867600000000001 2024-09-18T00:19:29.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950337+0000", 2024-09-18T00:19:29.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156902 2024-09-18T00:19:29.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950818+0000", 2024-09-18T00:19:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048192599999999998 2024-09-18T00:19:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978491+0000", 2024-09-18T00:19:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027672568000000002 2024-09-18T00:19:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989759+0000", 2024-09-18T00:19:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011268144000000001 2024-09-18T00:19:29.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:29.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989824+0000", 2024-09-18T00:19:29.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4556999999999999e-05 2024-09-18T00:19:29.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989868+0000", 2024-09-18T00:19:29.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4449000000000001e-05 2024-09-18T00:19:29.283 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.283 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.283 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.284 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18687 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:29.284 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:29.284 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.289636653999999, 2024-09-18T00:19:29.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040034156000000001, 2024-09-18T00:19:29.284 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.284 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.284 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.284 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.284 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.284 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.284 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18687 2024-09-18T00:19:29.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.285 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:29.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:29.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950065+0000", 2024-09-18T00:19:29.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5820000000000001e-06 2024-09-18T00:19:29.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950072+0000", 2024-09-18T00:19:29.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1029999999999996e-06 2024-09-18T00:19:29.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950092+0000", 2024-09-18T00:19:29.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0288e-05 2024-09-18T00:19:29.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950107+0000", 2024-09-18T00:19:29.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4290999999999999e-05 2024-09-18T00:19:29.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975820+0000", 2024-09-18T00:19:29.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025713573999999999 2024-09-18T00:19:29.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975980+0000", 2024-09-18T00:19:29.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016002799999999999 2024-09-18T00:19:29.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:29.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.976299+0000", 2024-09-18T00:19:29.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031853300000000002 2024-09-18T00:19:29.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:29.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989936+0000", 2024-09-18T00:19:29.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013637046999999999 2024-09-18T00:19:29.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:29.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990025+0000", 2024-09-18T00:19:29.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9040000000000001e-05 2024-09-18T00:19:29.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:29.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990064+0000", 2024-09-18T00:19:29.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9394000000000003e-05 2024-09-18T00:19:29.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990098+0000", 2024-09-18T00:19:29.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3275999999999998e-05 2024-09-18T00:19:29.290 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.290 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.290 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.290 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18688 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:29.290 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:29.290 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.289535301999997, 2024-09-18T00:19:29.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040041259000000003, 2024-09-18T00:19:29.290 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.290 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.290 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.291 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.291 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18688 2024-09-18T00:19:29.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.291 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:29.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:29.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950166+0000", 2024-09-18T00:19:29.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:19:29.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951295+0000", 2024-09-18T00:19:29.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011293480000000001 2024-09-18T00:19:29.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951328+0000", 2024-09-18T00:19:29.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2524999999999998e-05 2024-09-18T00:19:29.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951346+0000", 2024-09-18T00:19:29.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8170999999999999e-05 2024-09-18T00:19:29.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978546+0000", 2024-09-18T00:19:29.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027200418000000001 2024-09-18T00:19:29.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978666+0000", 2024-09-18T00:19:29.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119913 2024-09-18T00:19:29.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:29.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.979098+0000", 2024-09-18T00:19:29.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00043169399999999999 2024-09-18T00:19:29.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:29.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990000+0000", 2024-09-18T00:19:29.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010901719000000001 2024-09-18T00:19:29.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:29.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990109+0000", 2024-09-18T00:19:29.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010964299999999999 2024-09-18T00:19:29.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:29.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990152+0000", 2024-09-18T00:19:29.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2877000000000003e-05 2024-09-18T00:19:29.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990206+0000", 2024-09-18T00:19:29.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3717000000000001e-05 2024-09-18T00:19:29.296 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.296 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.296 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.296 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18689 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:29.296 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:29.296 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.288323603999999, 2024-09-18T00:19:29.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051022338, 2024-09-18T00:19:29.297 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.297 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.297 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.297 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.297 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.297 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.297 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18689 2024-09-18T00:19:29.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.297 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:29.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:29.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951378+0000", 2024-09-18T00:19:29.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:19:29.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951385+0000", 2024-09-18T00:19:29.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-18T00:19:29.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951402+0000", 2024-09-18T00:19:29.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6538999999999999e-05 2024-09-18T00:19:29.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951565+0000", 2024-09-18T00:19:29.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016307999999999999 2024-09-18T00:19:29.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988232+0000", 2024-09-18T00:19:29.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036667656999999999 2024-09-18T00:19:29.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988318+0000", 2024-09-18T00:19:29.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5348999999999999e-05 2024-09-18T00:19:29.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:29.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988574+0000", 2024-09-18T00:19:29.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025670800000000002 2024-09-18T00:19:29.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:29.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002254+0000", 2024-09-18T00:19:29.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013679274 2024-09-18T00:19:29.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:29.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002326+0000", 2024-09-18T00:19:29.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2720999999999997e-05 2024-09-18T00:19:29.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:29.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002364+0000", 2024-09-18T00:19:29.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7314000000000002e-05 2024-09-18T00:19:29.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002399+0000", 2024-09-18T00:19:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5162e-05 2024-09-18T00:19:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18690 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.288106585999998, 2024-09-18T00:19:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050872134999999999, 2024-09-18T00:19:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.303 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.303 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.303 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.303 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.303 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.303 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18690 2024-09-18T00:19:29.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.303 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:29.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:29.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951595+0000", 2024-09-18T00:19:29.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4479999999999999e-06 2024-09-18T00:19:29.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952167+0000", 2024-09-18T00:19:29.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00057188400000000004 2024-09-18T00:19:29.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952192+0000", 2024-09-18T00:19:29.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4936999999999999e-05 2024-09-18T00:19:29.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952206+0000", 2024-09-18T00:19:29.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4565e-05 2024-09-18T00:19:29.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990303+0000", 2024-09-18T00:19:29.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038096954000000002 2024-09-18T00:19:29.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990403+0000", 2024-09-18T00:19:29.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9486000000000003e-05 2024-09-18T00:19:29.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:29.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990773+0000", 2024-09-18T00:19:29.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037037799999999998 2024-09-18T00:19:29.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:29.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002316+0000", 2024-09-18T00:19:29.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011542581 2024-09-18T00:19:29.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:29.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002407+0000", 2024-09-18T00:19:29.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0968e-05 2024-09-18T00:19:29.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:29.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002442+0000", 2024-09-18T00:19:29.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4962000000000002e-05 2024-09-18T00:19:29.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002466+0000", 2024-09-18T00:19:29.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3972e-05 2024-09-18T00:19:29.308 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.308 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.308 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.308 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18691 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:29.308 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:29.309 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.287473708999997, 2024-09-18T00:19:29.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051495530999999997, 2024-09-18T00:19:29.309 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.309 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.309 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.309 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.309 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.309 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.309 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18691 2024-09-18T00:19:29.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.309 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:29.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:29.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952228+0000", 2024-09-18T00:19:29.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:19:29.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952234+0000", 2024-09-18T00:19:29.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0310000000000004e-06 2024-09-18T00:19:29.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952248+0000", 2024-09-18T00:19:29.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4577e-05 2024-09-18T00:19:29.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952260+0000", 2024-09-18T00:19:29.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1659e-05 2024-09-18T00:19:29.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.000989+0000", 2024-09-18T00:19:29.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048729445000000003 2024-09-18T00:19:29.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001076+0000", 2024-09-18T00:19:29.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6323999999999995e-05 2024-09-18T00:19:29.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:29.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001295+0000", 2024-09-18T00:19:29.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021944299999999999 2024-09-18T00:19:29.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:29.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003502+0000", 2024-09-18T00:19:29.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022065209999999999 2024-09-18T00:19:29.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:29.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003581+0000", 2024-09-18T00:19:29.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9404000000000001e-05 2024-09-18T00:19:29.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:29.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003652+0000", 2024-09-18T00:19:29.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0506999999999994e-05 2024-09-18T00:19:29.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003722+0000", 2024-09-18T00:19:29.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0300000000000001e-05 2024-09-18T00:19:29.314 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.314 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.314 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.315 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18692 3.7 3:e69124d0:::rbd_data.112a6efb7564.0000000000000001:f [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:29.315 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:29.315 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.287425648000003, 2024-09-18T00:19:29.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050418114999999999, 2024-09-18T00:19:29.315 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.315 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.315 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:29.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.315 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18692 2024-09-18T00:19:29.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.315 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:29.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:29.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952275+0000", 2024-09-18T00:19:29.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4e-07 2024-09-18T00:19:29.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952281+0000", 2024-09-18T00:19:29.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2000000000000002e-06 2024-09-18T00:19:29.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952295+0000", 2024-09-18T00:19:29.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4731e-05 2024-09-18T00:19:29.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952307+0000", 2024-09-18T00:19:29.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1877e-05 2024-09-18T00:19:29.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002506+0000", 2024-09-18T00:19:29.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050199230999999997 2024-09-18T00:19:29.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002574+0000", 2024-09-18T00:19:29.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8076000000000003e-05 2024-09-18T00:19:29.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002693+0000", 2024-09-18T00:19:29.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011806 2024-09-18T00:19:29.319 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.319 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.319 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.319 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19383 3.5 3:a1859bb0:::rbd_data.112a7fe29ab4.000000000000015b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:29.319 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:29.319 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 35.617414050999997, 2024-09-18T00:19:29.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056760192000000001, 2024-09-18T00:19:29.319 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.319 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.320 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.320 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.320 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.320 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.320 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19383 2024-09-18T00:19:29.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.320 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:29.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:29.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622287+0000", 2024-09-18T00:19:29.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-18T00:19:29.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622292+0000", 2024-09-18T00:19:29.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6110000000000004e-06 2024-09-18T00:19:29.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622304+0000", 2024-09-18T00:19:29.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2106000000000001e-05 2024-09-18T00:19:29.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622309+0000", 2024-09-18T00:19:29.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.818e-06 2024-09-18T00:19:29.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622360+0000", 2024-09-18T00:19:29.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0430999999999999e-05 2024-09-18T00:19:29.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622438+0000", 2024-09-18T00:19:29.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8362999999999995e-05 2024-09-18T00:19:29.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:29.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678811+0000", 2024-09-18T00:19:29.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056372894 2024-09-18T00:19:29.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:29.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678906+0000", 2024-09-18T00:19:29.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5078e-05 2024-09-18T00:19:29.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.679046+0000", 2024-09-18T00:19:29.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139987 2024-09-18T00:19:29.324 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.324 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.324 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.324 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22019 3.c 3:37b77e94:::rbd_data.112a948d5956.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:29.324 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:19:29.324 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 31.740439221999999, 2024-09-18T00:19:29.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.039176065000000003, 2024-09-18T00:19:29.325 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.325 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.325 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.325 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.325 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22019 2024-09-18T00:19:29.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.325 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:19:29.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499261+0000", 2024-09-18T00:19:29.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499262+0000", 2024-09-18T00:19:29.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:19:29.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499268+0000", 2024-09-18T00:19:29.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8780000000000003e-06 2024-09-18T00:19:29.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499283+0000", 2024-09-18T00:19:29.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5184000000000001e-05 2024-09-18T00:19:29.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499291+0000", 2024-09-18T00:19:29.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5959999999999998e-06 2024-09-18T00:19:29.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499364+0000", 2024-09-18T00:19:29.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2855000000000006e-05 2024-09-18T00:19:29.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.499427+0000", 2024-09-18T00:19:29.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3645999999999993e-05 2024-09-18T00:19:29.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:29.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538252+0000", 2024-09-18T00:19:29.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038825207 2024-09-18T00:19:29.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:29.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538328+0000", 2024-09-18T00:19:29.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5938000000000004e-05 2024-09-18T00:19:29.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.538437+0000", 2024-09-18T00:19:29.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000108527 2024-09-18T00:19:29.329 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.329 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.329 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.330 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22468 3.5 3:a250f1ef:::rbd_data.112a948d5956.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:29.330 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:29.330 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 30.468332645, 2024-09-18T00:19:29.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.055599787999999997, 2024-09-18T00:19:29.330 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.330 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.330 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.330 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.330 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.330 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.330 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22468 2024-09-18T00:19:29.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.330 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:29.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:29.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771370+0000", 2024-09-18T00:19:29.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9629999999999998e-06 2024-09-18T00:19:29.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771385+0000", 2024-09-18T00:19:29.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5053e-05 2024-09-18T00:19:29.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771429+0000", 2024-09-18T00:19:29.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3192999999999998e-05 2024-09-18T00:19:29.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771453+0000", 2024-09-18T00:19:29.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3995999999999999e-05 2024-09-18T00:19:29.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771623+0000", 2024-09-18T00:19:29.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000170846 2024-09-18T00:19:29.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771838+0000", 2024-09-18T00:19:29.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021416600000000001 2024-09-18T00:19:29.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:29.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826866+0000", 2024-09-18T00:19:29.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.05502866 2024-09-18T00:19:29.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:29.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826906+0000", 2024-09-18T00:19:29.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9805999999999998e-05 2024-09-18T00:19:29.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826967+0000", 2024-09-18T00:19:29.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1105000000000005e-05 2024-09-18T00:19:29.334 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.334 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.335 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.335 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22470 3.2 3:474595b6:::rbd_data.112a948d5956.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:29.335 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:29.335 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 30.412141661, 2024-09-18T00:19:29.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088321282000000001, 2024-09-18T00:19:29.335 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.335 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.335 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.335 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.335 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.335 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.335 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22470 2024-09-18T00:19:29.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.336 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:29.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:29.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827560+0000", 2024-09-18T00:19:29.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.762e-06 2024-09-18T00:19:29.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827569+0000", 2024-09-18T00:19:29.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8740000000000001e-06 2024-09-18T00:19:29.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827585+0000", 2024-09-18T00:19:29.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5962000000000001e-05 2024-09-18T00:19:29.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827595+0000", 2024-09-18T00:19:29.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5910000000000002e-06 2024-09-18T00:19:29.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827811+0000", 2024-09-18T00:19:29.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000216858 2024-09-18T00:19:29.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.338 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:19:29.338 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827818+0000", 2024-09-18T00:19:29.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.99e-06 2024-09-18T00:19:29.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829799+0000", 2024-09-18T00:19:29.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019803849999999999 2024-09-18T00:19:29.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829884+0000", 2024-09-18T00:19:29.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4852999999999995e-05 2024-09-18T00:19:29.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:29.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915616+0000", 2024-09-18T00:19:29.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085732808999999993 2024-09-18T00:19:29.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:29.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915711+0000", 2024-09-18T00:19:29.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4578000000000001e-05 2024-09-18T00:19:29.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915880+0000", 2024-09-18T00:19:29.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016861999999999999 2024-09-18T00:19:29.340 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.341 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.341 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.341 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:42788 3.7 3:e5a8432b:::rbd_data.112a3496573c.0000000000000027:17 [read 309248~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:29.341 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:29.341 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.5207298609999995, 2024-09-18T00:19:29.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.039621847000000002, 2024-09-18T00:19:29.341 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.341 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.341 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:29.341 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.342 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.342 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.342 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 42788 2024-09-18T00:19:29.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.342 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:29.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:29.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718971+0000", 2024-09-18T00:19:29.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2500000000000001e-07 2024-09-18T00:19:29.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718973+0000", 2024-09-18T00:19:29.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2510000000000001e-06 2024-09-18T00:19:29.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718979+0000", 2024-09-18T00:19:29.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1700000000000002e-06 2024-09-18T00:19:29.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718984+0000", 2024-09-18T00:19:29.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9110000000000001e-06 2024-09-18T00:19:29.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.749126+0000", 2024-09-18T00:19:29.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030142164999999999 2024-09-18T00:19:29.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.749199+0000", 2024-09-18T00:19:29.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2985999999999995e-05 2024-09-18T00:19:29.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.758592+0000", 2024-09-18T00:19:29.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093929389999999995 2024-09-18T00:19:29.345 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.345 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.345 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.346 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:42820 3.7 3:e2363f9c:::rbd_data.112a3496573c.0000000000000028:17 [read 446464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:29.346 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.719971+0000", 2024-09-18T00:19:29.346 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.5197293649999999, 2024-09-18T00:19:29.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034904399000000003, 2024-09-18T00:19:29.346 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:29.346 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:29.346 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:29.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:29.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:29.346 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.346 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 42820 2024-09-18T00:19:29.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.346 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:29.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:29.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719971+0000", 2024-09-18T00:19:29.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:29.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719971+0000", 2024-09-18T00:19:29.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:29.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:29.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719971+0000", 2024-09-18T00:19:29.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5999999999999999e-07 2024-09-18T00:19:29.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:29.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719973+0000", 2024-09-18T00:19:29.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.069e-06 2024-09-18T00:19:29.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:29.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719980+0000", 2024-09-18T00:19:29.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5379999999999999e-06 2024-09-18T00:19:29.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.719985+0000", 2024-09-18T00:19:29.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1080000000000001e-06 2024-09-18T00:19:29.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:29.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.754731+0000", 2024-09-18T00:19:29.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.034745725999999998 2024-09-18T00:19:29.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:29.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.754804+0000", 2024-09-18T00:19:29.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3396e-05 2024-09-18T00:19:29.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:29.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:29.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:29.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.754875+0000", 2024-09-18T00:19:29.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1101999999999998e-05 2024-09-18T00:19:29.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.350 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:29.350 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:29.350 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:29.351 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:29.351 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:29.352 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:19:29.353 INFO:teuthology.orchestra.run.smithi097.stdout:5258 write 0x8e8b47c thru 0x8e8ca6e (0x15f3 bytes) 2024-09-18T00:19:29.353 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:29.354 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:29.354 INFO:teuthology.orchestra.run.smithi097.stdout:5259 read 0x22833d8 thru 0x228fa04 (0xc62d bytes) 2024-09-18T00:19:29.354 INFO:teuthology.orchestra.run.smithi097.stdout:5260 read 0xe7736f thru 0xe814ca (0xa15c bytes) 2024-09-18T00:19:29.355 INFO:teuthology.orchestra.run.smithi097.stdout:5261 write 0x2f5bcf2 thru 0x2f61d71 (0x6080 bytes) 2024-09-18T00:19:29.355 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:29.355 INFO:teuthology.orchestra.run.smithi097.stdout:5263 read 0x32ee72d thru 0x32f4283 (0x5b57 bytes) 2024-09-18T00:19:29.355 INFO:teuthology.orchestra.run.smithi097.stdout:5264 trunc from 0xe75d665 to 0xcd9c009 2024-09-18T00:19:29.355 INFO:teuthology.orchestra.run.smithi097.stdout:5265 writesame 0x18e81ee thru 0x18f482e data_size 0x1(0xc641 bytes) 2024-09-18T00:19:29.355 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:29.355 INFO:teuthology.orchestra.run.smithi097.stdout:5268 punch from 0x49b79d2 to 0x49bba3e, (0x406c bytes) 2024-09-18T00:19:29.355 INFO:teuthology.orchestra.run.smithi097.stdout:5269 trunc from 0xcd9c009 to 0xbfba9a8 2024-09-18T00:19:29.356 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-18T00:19:29.357 INFO:teuthology.orchestra.run.smithi097.stdout:5270 punch from 0x751b764 to 0x751f242, (0x3ade bytes) 2024-09-18T00:19:29.357 INFO:teuthology.orchestra.run.smithi097.stdout:5271 write 0xd527318 thru 0xd536e14 (0xfafd bytes) 2024-09-18T00:19:29.357 INFO:teuthology.orchestra.run.smithi097.stdout:5272 read 0x6e8f068 thru 0x6e932e4 (0x427d bytes) 2024-09-18T00:19:29.357 INFO:teuthology.orchestra.run.smithi097.stdout:5274 read 0xd1f9bb3 thru 0xd1ff22a (0x5678 bytes) 2024-09-18T00:19:29.357 INFO:teuthology.orchestra.run.smithi097.stdout:5276 read 0x397207b thru 0x397b9d8 (0x995e bytes) 2024-09-18T00:19:29.357 INFO:teuthology.orchestra.run.smithi097.stdout:5277 write 0x960f2cf thru 0x9610771 (0x14a3 bytes) 2024-09-18T00:19:29.357 INFO:teuthology.orchestra.run.smithi097.stdout:5278 writesame 0x713b207 thru 0x713e929 data_size 0x5(0x3723 bytes) 2024-09-18T00:19:29.357 INFO:teuthology.orchestra.run.smithi097.stdout:5279 read 0x11b7614 thru 0x11b8bf4 (0x15e1 bytes) 2024-09-18T00:19:29.358 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:29.349+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:29.364 INFO:teuthology.orchestra.run.smithi097.stdout:5280 write 0xdfaca2b thru 0xdfb41df (0x77b5 bytes) 2024-09-18T00:19:29.375 INFO:teuthology.orchestra.run.smithi097.stdout:5281 write 0x89e3d45 thru 0x89e4c56 (0xf12 bytes) 2024-09-18T00:19:29.377 INFO:teuthology.orchestra.run.smithi097.stdout:5282 write 0x7caf4bd thru 0x7cb470f (0x5253 bytes) 2024-09-18T00:19:29.384 INFO:teuthology.orchestra.run.smithi097.stdout:5283 read 0xb1271e5 thru 0xb12d115 (0x5f31 bytes) 2024-09-18T00:19:29.384 INFO:teuthology.orchestra.run.smithi097.stdout:5285 write 0x331ba7f thru 0x3325e8b (0xa40d bytes) 2024-09-18T00:19:29.391 INFO:teuthology.orchestra.run.smithi097.stdout:5287 punch from 0x86269df to 0x86357fd, (0xee1e bytes) 2024-09-18T00:19:29.391 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:29.391 INFO:teuthology.orchestra.run.smithi097.stdout:5289 read 0x77c9d1f thru 0x77d5400 (0xb6e2 bytes) 2024-09-18T00:19:29.391 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:29.392 INFO:teuthology.orchestra.run.smithi097.stdout:5291 writesame 0x1a077ed thru 0x1a12082 data_size 0x3(0xa896 bytes) 2024-09-18T00:19:29.399 INFO:teuthology.orchestra.run.smithi097.stdout:5292 read 0x1975dc8 thru 0x1981df0 (0xc029 bytes) 2024-09-18T00:19:29.399 INFO:teuthology.orchestra.run.smithi097.stdout:5293 write 0x77ddb85 thru 0x77eca56 (0xeed2 bytes) 2024-09-18T00:19:29.402 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:29.402 INFO:teuthology.orchestra.run.smithi097.stdout:5296 read 0x179f1d1 thru 0x17a8de6 (0x9c16 bytes) 2024-09-18T00:19:29.413 INFO:teuthology.orchestra.run.smithi097.stdout:5297 trunc from 0xdfb41e0 to 0x615e9c1 2024-09-18T00:19:29.434 INFO:teuthology.orchestra.run.smithi097.stdout:5298 read 0x584f973 thru 0x584ff71 (0x5ff bytes) 2024-09-18T00:19:29.434 INFO:teuthology.orchestra.run.smithi097.stdout:5299 read 0x39e091f thru 0x39ec209 (0xb8eb bytes) 2024-09-18T00:19:29.434 INFO:teuthology.orchestra.run.smithi097.stdout:5300 write 0xd0c98 thru 0xda8fc (0x9c65 bytes) 2024-09-18T00:19:29.436 INFO:teuthology.orchestra.run.smithi097.stdout:5301 punch from 0x1bd9518 to 0x1be8c22, (0xf70a bytes) 2024-09-18T00:19:29.443 INFO:teuthology.orchestra.run.smithi097.stdout:5302 trunc from 0x615e9c1 to 0xc1c4993 2024-09-18T00:19:29.453 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:29.449+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:29.460 INFO:teuthology.orchestra.run.smithi097.stdout:5303 trunc from 0xc1c4993 to 0x44d23cb 2024-09-18T00:19:29.490 INFO:teuthology.orchestra.run.smithi097.stdout:5305 write 0x89c07b7 thru 0x89c92c0 (0x8b0a bytes) 2024-09-18T00:19:29.499 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:29.499 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:29.499 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:29.499 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:29.506 INFO:teuthology.orchestra.run.smithi097.stdout:5306 trunc from 0x89c92c1 to 0x28b1863 2024-09-18T00:19:29.515 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:19:29.539 INFO:teuthology.orchestra.run.smithi097.stdout:5307 write 0x7193374 thru 0x719d809 (0xa496 bytes) 2024-09-18T00:19:29.545 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:29.545 INFO:teuthology.orchestra.run.smithi097.stdout:5310 writesame 0x5d29253 thru 0x5d2b604 data_size 0x6(0x23b2 bytes) 2024-09-18T00:19:29.549 INFO:teuthology.orchestra.run.smithi097.stdout:5311 read 0x2347d20 thru 0x2349146 (0x1427 bytes) 2024-09-18T00:19:29.549 INFO:teuthology.orchestra.run.smithi097.stdout:5312 read 0x6ed947a thru 0x6eda896 (0x141d bytes) 2024-09-18T00:19:29.554 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:29.549+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:29.557 INFO:teuthology.orchestra.run.smithi097.stdout:5313 trunc from 0x719d80a to 0x9f6c7d 2024-09-18T00:19:29.655 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:29.649+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:29.677 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:29.677 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:29.677 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:29.677 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:29.677 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:29.692 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:19:29.698 INFO:teuthology.orchestra.run.smithi097.stdout:5314 write 0x5e0bed9 thru 0x5e11333 (0x545b bytes) 2024-09-18T00:19:29.706 INFO:teuthology.orchestra.run.smithi097.stdout:5316 punch from 0x4f6c4c to 0x4fcf15, (0x62c9 bytes) 2024-09-18T00:19:29.713 INFO:teuthology.orchestra.run.smithi097.stdout:5317 trunc from 0x5e11334 to 0x28dcb5d 2024-09-18T00:19:29.728 INFO:teuthology.orchestra.run.smithi097.stdout:5318 punch from 0xf0d4e6 to 0xf1a29b, (0xcdb5 bytes) 2024-09-18T00:19:29.734 INFO:teuthology.orchestra.run.smithi097.stdout:5319 writesame 0x2a8b8a3 thru 0x2a95f62 data_size 0x5c(0xa6c0 bytes) 2024-09-18T00:19:29.751 INFO:teuthology.orchestra.run.smithi097.stdout:5320 writesame 0x63e3c56 thru 0x63e4446 data_size 0x1(0x7f1 bytes) 2024-09-18T00:19:29.752 INFO:teuthology.orchestra.run.smithi097.stdout:5322 punch from 0xd0c955 to 0xd0d334, (0x9df bytes) 2024-09-18T00:19:29.752 INFO:teuthology.orchestra.run.smithi097.stdout:5323 punch from 0x2fa2cb0 to 0x2fa841f, (0x576f bytes) 2024-09-18T00:19:29.752 INFO:teuthology.orchestra.run.smithi097.stdout:5325 read 0x3ea4981 thru 0x3eb09c4 (0xc044 bytes) 2024-09-18T00:19:29.752 INFO:teuthology.orchestra.run.smithi097.stdout:5328 read 0x3e8fcf9 thru 0x3e9c965 (0xcc6d bytes) 2024-09-18T00:19:29.756 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:29.749+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:29.768 INFO:teuthology.orchestra.run.smithi097.stdout:5329 writesame 0x9adb010 thru 0x9adbefa data_size 0x13(0xeeb bytes) 2024-09-18T00:19:29.768 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:29.769 INFO:teuthology.orchestra.run.smithi097.stdout:5331 read 0x33b6787 thru 0x33bbcbc (0x5536 bytes) 2024-09-18T00:19:29.769 INFO:teuthology.orchestra.run.smithi097.stdout:5332 writesame 0x475b1ab thru 0x4764b25 data_size 0x7(0x997b bytes) 2024-09-18T00:19:29.804 INFO:teuthology.orchestra.run.smithi097.stdout:5333 trunc from 0x9adbefb to 0x3c166c 2024-09-18T00:19:29.826 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:29.826 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:29.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:29.826 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:29.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.826 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2418 3.1a 3:59a8b40d:::rbd_data.112aa7ba4578.00000000000001b0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:19:29.826 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:29.826 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.086278187999994, 2024-09-18T00:19:29.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10488562999999999, 2024-09-18T00:19:29.827 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.827 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.827 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.827 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.827 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.827 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.827 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2418 2024-09-18T00:19:29.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.828 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:29.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:29.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735592+0000", 2024-09-18T00:19:29.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5229999999999998e-06 2024-09-18T00:19:29.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735604+0000", 2024-09-18T00:19:29.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2143e-05 2024-09-18T00:19:29.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735629+0000", 2024-09-18T00:19:29.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.497e-05 2024-09-18T00:19:29.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735648+0000", 2024-09-18T00:19:29.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8896999999999999e-05 2024-09-18T00:19:29.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735775+0000", 2024-09-18T00:19:29.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126935 2024-09-18T00:19:29.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.736118+0000", 2024-09-18T00:19:29.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034330800000000001 2024-09-18T00:19:29.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-18T00:19:29.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.741633+0000", 2024-09-18T00:19:29.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055144139999999996 2024-09-18T00:19:29.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:29.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840134+0000", 2024-09-18T00:19:29.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.098501618999999999 2024-09-18T00:19:29.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:29.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840184+0000", 2024-09-18T00:19:29.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9953000000000001e-05 2024-09-18T00:19:29.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:29.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840336+0000", 2024-09-18T00:19:29.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151476 2024-09-18T00:19:29.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840475+0000", 2024-09-18T00:19:29.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013939200000000001 2024-09-18T00:19:29.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.833 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.833 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:22272 3.3 3:c617f123:::rbd_data.112a948d5956.0000000000000009:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:29.833 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:29.833 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.830769989, 2024-09-18T00:19:29.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10752577300000001, 2024-09-18T00:19:29.834 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.834 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.834 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.834 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.834 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.834 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.834 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 22272 2024-09-18T00:19:29.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.834 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:29.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:29.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991102+0000", 2024-09-18T00:19:29.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9369999999999997e-06 2024-09-18T00:19:29.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991118+0000", 2024-09-18T00:19:29.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6396e-05 2024-09-18T00:19:29.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991150+0000", 2024-09-18T00:19:29.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2132999999999999e-05 2024-09-18T00:19:29.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991170+0000", 2024-09-18T00:19:29.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0085000000000002e-05 2024-09-18T00:19:29.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991370+0000", 2024-09-18T00:19:29.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019963700000000001 2024-09-18T00:19:29.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991518+0000", 2024-09-18T00:19:29.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000148424 2024-09-18T00:19:29.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:29.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.998850+0000", 2024-09-18T00:19:29.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073315480000000002 2024-09-18T00:19:29.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:29.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098178+0000", 2024-09-18T00:19:29.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.099328067000000006 2024-09-18T00:19:29.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:29.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098314+0000", 2024-09-18T00:19:29.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000135678 2024-09-18T00:19:29.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:29.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098498+0000", 2024-09-18T00:19:29.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018479600000000001 2024-09-18T00:19:29.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098623+0000", 2024-09-18T00:19:29.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000125072 2024-09-18T00:19:29.839 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.839 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.839 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.840 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:23689 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-18T00:19:29.840 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:29.840 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.969445211, 2024-09-18T00:19:29.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9562625809999998, 2024-09-18T00:19:29.840 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.840 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.840 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:29.840 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.840 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.840 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.840 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 23689 2024-09-18T00:19:29.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.841 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:29.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:29.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852423+0000", 2024-09-18T00:19:29.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9999999999999995e-07 2024-09-18T00:19:29.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852429+0000", 2024-09-18T00:19:29.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3279999999999996e-06 2024-09-18T00:19:29.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852442+0000", 2024-09-18T00:19:29.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3052e-05 2024-09-18T00:19:29.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852448+0000", 2024-09-18T00:19:29.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2779999999999999e-06 2024-09-18T00:19:29.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852523+0000", 2024-09-18T00:19:29.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4463999999999997e-05 2024-09-18T00:19:29.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:29.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852538+0000", 2024-09-18T00:19:29.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5466999999999999e-05 2024-09-18T00:19:29.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793478+0000", 2024-09-18T00:19:29.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.94094012699999996 2024-09-18T00:19:29.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:29.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793494+0000", 2024-09-18T00:19:29.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5807999999999998e-05 2024-09-18T00:19:29.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.804875+0000", 2024-09-18T00:19:29.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0113811939999999 2024-09-18T00:19:29.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805359+0000", 2024-09-18T00:19:29.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000484248 2024-09-18T00:19:29.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-18T00:19:29.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805872+0000", 2024-09-18T00:19:29.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00051218100000000001 2024-09-18T00:19:29.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:29.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.807790+0000", 2024-09-18T00:19:29.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001918763 2024-09-18T00:19:29.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:29.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808242+0000", 2024-09-18T00:19:29.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045126200000000002 2024-09-18T00:19:29.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:29.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808428+0000", 2024-09-18T00:19:29.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018633000000000001 2024-09-18T00:19:29.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808685+0000", 2024-09-18T00:19:29.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025707900000000001 2024-09-18T00:19:29.847 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.847 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.848 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.848 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47924 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7831552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.848 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032564+0000", 2024-09-18T00:19:29.848 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.789303232, 2024-09-18T00:19:29.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086266190000000006, 2024-09-18T00:19:29.848 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.848 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.848 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.848 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.848 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.848 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.848 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47924 2024-09-18T00:19:29.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.849 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032564+0000", 2024-09-18T00:19:29.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032564+0000", 2024-09-18T00:19:29.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032565+0000", 2024-09-18T00:19:29.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9599999999999999e-07 2024-09-18T00:19:29.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032567+0000", 2024-09-18T00:19:29.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.142e-06 2024-09-18T00:19:29.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032573+0000", 2024-09-18T00:19:29.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2659999999999998e-06 2024-09-18T00:19:29.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032578+0000", 2024-09-18T00:19:29.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9019999999999998e-06 2024-09-18T00:19:29.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118632+0000", 2024-09-18T00:19:29.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086053942999999994 2024-09-18T00:19:29.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118706+0000", 2024-09-18T00:19:29.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4266999999999999e-05 2024-09-18T00:19:29.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118831+0000", 2024-09-18T00:19:29.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012417400000000001 2024-09-18T00:19:29.852 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.852 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.852 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.852 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47925 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7864320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.852 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:19:29.852 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.78928311, 2024-09-18T00:19:29.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086484302999999998, 2024-09-18T00:19:29.853 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.853 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.853 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.853 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47925 2024-09-18T00:19:29.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.853 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:19:29.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:19:29.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:19:29.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.82e-07 2024-09-18T00:19:29.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032587+0000", 2024-09-18T00:19:29.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.058e-06 2024-09-18T00:19:29.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032593+0000", 2024-09-18T00:19:29.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1380000000000004e-06 2024-09-18T00:19:29.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032598+0000", 2024-09-18T00:19:29.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9819999999999999e-06 2024-09-18T00:19:29.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118873+0000", 2024-09-18T00:19:29.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086274609000000002 2024-09-18T00:19:29.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118950+0000", 2024-09-18T00:19:29.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6687999999999995e-05 2024-09-18T00:19:29.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119069+0000", 2024-09-18T00:19:29.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011934600000000001 2024-09-18T00:19:29.856 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.856 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.857 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.857 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47926 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7897088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.857 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032604+0000", 2024-09-18T00:19:29.857 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.7892632339999999, 2024-09-18T00:19:29.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086712168000000006, 2024-09-18T00:19:29.857 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.857 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.857 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.858 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47926 2024-09-18T00:19:29.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.858 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032604+0000", 2024-09-18T00:19:29.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032604+0000", 2024-09-18T00:19:29.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032605+0000", 2024-09-18T00:19:29.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:19:29.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032611+0000", 2024-09-18T00:19:29.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4860000000000002e-06 2024-09-18T00:19:29.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032618+0000", 2024-09-18T00:19:29.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6699999999999997e-06 2024-09-18T00:19:29.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032623+0000", 2024-09-18T00:19:29.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0860000000000001e-06 2024-09-18T00:19:29.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119111+0000", 2024-09-18T00:19:29.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086488067000000002 2024-09-18T00:19:29.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119185+0000", 2024-09-18T00:19:29.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3758999999999996e-05 2024-09-18T00:19:29.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119317+0000", 2024-09-18T00:19:29.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013159599999999999 2024-09-18T00:19:29.861 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.861 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.861 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.861 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47927 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7929856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.861 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:19:29.862 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.7892378930000001, 2024-09-18T00:19:29.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086929532000000004, 2024-09-18T00:19:29.862 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.862 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.862 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.862 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47927 2024-09-18T00:19:29.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.862 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:19:29.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:19:29.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:19:29.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-18T00:19:29.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032632+0000", 2024-09-18T00:19:29.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1320000000000001e-06 2024-09-18T00:19:29.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032639+0000", 2024-09-18T00:19:29.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1840000000000004e-06 2024-09-18T00:19:29.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032644+0000", 2024-09-18T00:19:29.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2519999999999999e-06 2024-09-18T00:19:29.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119358+0000", 2024-09-18T00:19:29.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086714075000000002 2024-09-18T00:19:29.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119438+0000", 2024-09-18T00:19:29.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9756e-05 2024-09-18T00:19:29.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119559+0000", 2024-09-18T00:19:29.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121579 2024-09-18T00:19:29.866 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.866 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.866 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.866 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47928 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7962624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.866 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032650+0000", 2024-09-18T00:19:29.866 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.789217469, 2024-09-18T00:19:29.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087152732999999996, 2024-09-18T00:19:29.866 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.866 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.866 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.866 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.866 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.867 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.867 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47928 2024-09-18T00:19:29.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.867 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032650+0000", 2024-09-18T00:19:29.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032650+0000", 2024-09-18T00:19:29.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032651+0000", 2024-09-18T00:19:29.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:19:29.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032653+0000", 2024-09-18T00:19:29.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.182e-06 2024-09-18T00:19:29.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032659+0000", 2024-09-18T00:19:29.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2319999999999998e-06 2024-09-18T00:19:29.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032664+0000", 2024-09-18T00:19:29.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0420000000000002e-06 2024-09-18T00:19:29.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119602+0000", 2024-09-18T00:19:29.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086938213 2024-09-18T00:19:29.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119676+0000", 2024-09-18T00:19:29.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3708000000000006e-05 2024-09-18T00:19:29.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119803+0000", 2024-09-18T00:19:29.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126852 2024-09-18T00:19:29.870 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.870 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.870 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.871 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47929 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7995392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.871 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032670+0000", 2024-09-18T00:19:29.871 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.7891972109999998, 2024-09-18T00:19:29.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087373082000000005, 2024-09-18T00:19:29.871 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.871 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.871 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.871 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47929 2024-09-18T00:19:29.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.871 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032670+0000", 2024-09-18T00:19:29.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032670+0000", 2024-09-18T00:19:29.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032671+0000", 2024-09-18T00:19:29.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:19:29.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032673+0000", 2024-09-18T00:19:29.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.928e-06 2024-09-18T00:19:29.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032679+0000", 2024-09-18T00:19:29.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.02e-06 2024-09-18T00:19:29.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032684+0000", 2024-09-18T00:19:29.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1660000000000002e-06 2024-09-18T00:19:29.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119844+0000", 2024-09-18T00:19:29.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087160248999999995 2024-09-18T00:19:29.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119920+0000", 2024-09-18T00:19:29.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5181999999999999e-05 2024-09-18T00:19:29.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120044+0000", 2024-09-18T00:19:29.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012403300000000001 2024-09-18T00:19:29.875 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.875 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.875 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.875 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47930 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8028160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.875 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032690+0000", 2024-09-18T00:19:29.875 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.789177359, 2024-09-18T00:19:29.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087595941999999996, 2024-09-18T00:19:29.876 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.876 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.876 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.876 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.876 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.876 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.876 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47930 2024-09-18T00:19:29.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.876 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032690+0000", 2024-09-18T00:19:29.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032690+0000", 2024-09-18T00:19:29.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032691+0000", 2024-09-18T00:19:29.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9399999999999995e-07 2024-09-18T00:19:29.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032693+0000", 2024-09-18T00:19:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1299999999999999e-06 2024-09-18T00:19:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032699+0000", 2024-09-18T00:19:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1879999999999999e-06 2024-09-18T00:19:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032704+0000", 2024-09-18T00:19:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0939999999999998e-06 2024-09-18T00:19:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120087+0000", 2024-09-18T00:19:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087383049000000004 2024-09-18T00:19:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120164+0000", 2024-09-18T00:19:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6679000000000002e-05 2024-09-18T00:19:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120286+0000", 2024-09-18T00:19:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012230800000000001 2024-09-18T00:19:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47931 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8060928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:19:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.7891571160000002, 2024-09-18T00:19:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087823688999999996, 2024-09-18T00:19:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47931 2024-09-18T00:19:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:19:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:19:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:19:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9200000000000001e-07 2024-09-18T00:19:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032713+0000", 2024-09-18T00:19:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1239999999999999e-06 2024-09-18T00:19:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032719+0000", 2024-09-18T00:19:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1659999999999999e-06 2024-09-18T00:19:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032724+0000", 2024-09-18T00:19:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9359999999999998e-06 2024-09-18T00:19:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120330+0000", 2024-09-18T00:19:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087605559 2024-09-18T00:19:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120403+0000", 2024-09-18T00:19:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-05 2024-09-18T00:19:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120534+0000", 2024-09-18T00:19:29.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013121199999999999 2024-09-18T00:19:29.884 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.884 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.884 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.884 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47932 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8093696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.884 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032730+0000", 2024-09-18T00:19:29.884 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.789137218, 2024-09-18T00:19:29.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088052486999999999, 2024-09-18T00:19:29.884 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.885 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.885 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.885 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.885 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.885 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.885 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47932 2024-09-18T00:19:29.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.885 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032730+0000", 2024-09-18T00:19:29.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032730+0000", 2024-09-18T00:19:29.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032731+0000", 2024-09-18T00:19:29.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2200000000000004e-07 2024-09-18T00:19:29.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032743+0000", 2024-09-18T00:19:29.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1796e-05 2024-09-18T00:19:29.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032749+0000", 2024-09-18T00:19:29.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6959999999999999e-06 2024-09-18T00:19:29.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032755+0000", 2024-09-18T00:19:29.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.22e-06 2024-09-18T00:19:29.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120578+0000", 2024-09-18T00:19:29.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087823017000000003 2024-09-18T00:19:29.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120654+0000", 2024-09-18T00:19:29.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6476000000000004e-05 2024-09-18T00:19:29.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120783+0000", 2024-09-18T00:19:29.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012876000000000001 2024-09-18T00:19:29.888 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.889 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.889 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.889 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47933 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8126464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.889 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032761+0000", 2024-09-18T00:19:29.889 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.7891062, 2024-09-18T00:19:29.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088263233999999996, 2024-09-18T00:19:29.889 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.889 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.889 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.889 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.889 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47933 2024-09-18T00:19:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032761+0000", 2024-09-18T00:19:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032761+0000", 2024-09-18T00:19:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032762+0000", 2024-09-18T00:19:29.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:19:29.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032764+0000", 2024-09-18T00:19:29.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1519999999999999e-06 2024-09-18T00:19:29.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032770+0000", 2024-09-18T00:19:29.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1700000000000002e-06 2024-09-18T00:19:29.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032778+0000", 2024-09-18T00:19:29.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2400000000000001e-06 2024-09-18T00:19:29.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120825+0000", 2024-09-18T00:19:29.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088047046000000004 2024-09-18T00:19:29.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120898+0000", 2024-09-18T00:19:29.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3011999999999998e-05 2024-09-18T00:19:29.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121025+0000", 2024-09-18T00:19:29.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012710999999999999 2024-09-18T00:19:29.893 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.893 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.893 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.893 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47934 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8159232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.893 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032784+0000", 2024-09-18T00:19:29.894 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.7890836490000002, 2024-09-18T00:19:29.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088479458999999996, 2024-09-18T00:19:29.894 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.894 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.894 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.894 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.894 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.894 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.894 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47934 2024-09-18T00:19:29.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.894 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032784+0000", 2024-09-18T00:19:29.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032784+0000", 2024-09-18T00:19:29.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032785+0000", 2024-09-18T00:19:29.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9599999999999999e-07 2024-09-18T00:19:29.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032787+0000", 2024-09-18T00:19:29.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.052e-06 2024-09-18T00:19:29.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032793+0000", 2024-09-18T00:19:29.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2439999999999998e-06 2024-09-18T00:19:29.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032798+0000", 2024-09-18T00:19:29.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9080000000000003e-06 2024-09-18T00:19:29.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121066+0000", 2024-09-18T00:19:29.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088267795999999996 2024-09-18T00:19:29.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121139+0000", 2024-09-18T00:19:29.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2910999999999994e-05 2024-09-18T00:19:29.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121263+0000", 2024-09-18T00:19:29.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000124852 2024-09-18T00:19:29.898 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.898 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.898 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.898 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47935 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8192000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.898 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032841+0000", 2024-09-18T00:19:29.898 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.7890265689999998, 2024-09-18T00:19:29.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088666069, 2024-09-18T00:19:29.898 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.898 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.898 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.899 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.899 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.899 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.899 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47935 2024-09-18T00:19:29.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.899 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032841+0000", 2024-09-18T00:19:29.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032841+0000", 2024-09-18T00:19:29.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032842+0000", 2024-09-18T00:19:29.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6400000000000002e-07 2024-09-18T00:19:29.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032845+0000", 2024-09-18T00:19:29.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8959999999999999e-06 2024-09-18T00:19:29.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032852+0000", 2024-09-18T00:19:29.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1640000000000004e-06 2024-09-18T00:19:29.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032857+0000", 2024-09-18T00:19:29.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3340000000000001e-06 2024-09-18T00:19:29.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121304+0000", 2024-09-18T00:19:29.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088447001999999997 2024-09-18T00:19:29.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121376+0000", 2024-09-18T00:19:29.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2121999999999998e-05 2024-09-18T00:19:29.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121507+0000", 2024-09-18T00:19:29.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130987 2024-09-18T00:19:29.902 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.902 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.902 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.903 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47936 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.903 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:19:29.903 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.7890038480000001, 2024-09-18T00:19:29.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088878349999999995, 2024-09-18T00:19:29.903 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.903 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.903 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.903 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.903 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.903 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.903 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47936 2024-09-18T00:19:29.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.904 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:19:29.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:19:29.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:19:29.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8999999999999997e-07 2024-09-18T00:19:29.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032866+0000", 2024-09-18T00:19:29.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.142e-06 2024-09-18T00:19:29.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032873+0000", 2024-09-18T00:19:29.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:19:29.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032878+0000", 2024-09-18T00:19:29.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1220000000000003e-06 2024-09-18T00:19:29.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121548+0000", 2024-09-18T00:19:29.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088670530999999997 2024-09-18T00:19:29.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121621+0000", 2024-09-18T00:19:29.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2941999999999994e-05 2024-09-18T00:19:29.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121742+0000", 2024-09-18T00:19:29.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120903 2024-09-18T00:19:29.907 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.907 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.907 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.907 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47937 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8257536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.907 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:19:29.907 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.7889837399999999, 2024-09-18T00:19:29.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089098572000000001, 2024-09-18T00:19:29.907 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.907 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.908 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.908 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.908 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.908 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.908 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47937 2024-09-18T00:19:29.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.908 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:19:29.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:19:29.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:19:29.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9200000000000001e-07 2024-09-18T00:19:29.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032886+0000", 2024-09-18T00:19:29.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9999999999999999e-06 2024-09-18T00:19:29.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032893+0000", 2024-09-18T00:19:29.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2020000000000001e-06 2024-09-18T00:19:29.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032898+0000", 2024-09-18T00:19:29.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9860000000000002e-06 2024-09-18T00:19:29.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121785+0000", 2024-09-18T00:19:29.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088887318000000007 2024-09-18T00:19:29.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121858+0000", 2024-09-18T00:19:29.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2900999999999999e-05 2024-09-18T00:19:29.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121983+0000", 2024-09-18T00:19:29.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000124673 2024-09-18T00:19:29.911 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.911 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.911 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.912 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47938 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8290304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.912 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:19:29.912 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.7889639559999999, 2024-09-18T00:19:29.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089335780000000004, 2024-09-18T00:19:29.912 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.912 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.912 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.912 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.912 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.912 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.912 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47938 2024-09-18T00:19:29.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.912 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:19:29.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:19:29.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:19:29.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2799999999999996e-07 2024-09-18T00:19:29.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032923+0000", 2024-09-18T00:19:29.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8218000000000001e-05 2024-09-18T00:19:29.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032930+0000", 2024-09-18T00:19:29.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5440000000000001e-06 2024-09-18T00:19:29.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032936+0000", 2024-09-18T00:19:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4720000000000003e-06 2024-09-18T00:19:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122023+0000", 2024-09-18T00:19:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089087728000000005 2024-09-18T00:19:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122117+0000", 2024-09-18T00:19:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3295999999999996e-05 2024-09-18T00:19:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122240+0000", 2024-09-18T00:19:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012289399999999999 2024-09-18T00:19:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47939 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8323072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032943+0000", 2024-09-18T00:19:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.7889244030000002, 2024-09-18T00:19:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089547929999999998, 2024-09-18T00:19:29.916 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.917 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.917 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.917 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.917 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.917 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.917 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47939 2024-09-18T00:19:29.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.917 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032943+0000", 2024-09-18T00:19:29.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032943+0000", 2024-09-18T00:19:29.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032944+0000", 2024-09-18T00:19:29.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6199999999999998e-07 2024-09-18T00:19:29.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032946+0000", 2024-09-18T00:19:29.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2840000000000001e-06 2024-09-18T00:19:29.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032952+0000", 2024-09-18T00:19:29.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2700000000000001e-06 2024-09-18T00:19:29.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032957+0000", 2024-09-18T00:19:29.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0200000000000002e-06 2024-09-18T00:19:29.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122280+0000", 2024-09-18T00:19:29.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089322330000000005 2024-09-18T00:19:29.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122354+0000", 2024-09-18T00:19:29.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3794000000000005e-05 2024-09-18T00:19:29.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122491+0000", 2024-09-18T00:19:29.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013767 2024-09-18T00:19:29.920 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.920 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.920 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.921 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47940 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8355840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:29.921 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:19:29.921 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.7889039750000002, 2024-09-18T00:19:29.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089758663000000002, 2024-09-18T00:19:29.921 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:29.921 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:29.921 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:29.921 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:29.921 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:29.921 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:29.921 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47940 2024-09-18T00:19:29.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:19:29.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8599999999999998e-07 2024-09-18T00:19:29.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:29.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032966+0000", 2024-09-18T00:19:29.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1160000000000002e-06 2024-09-18T00:19:29.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:29.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032972+0000", 2024-09-18T00:19:29.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1639999999999998e-06 2024-09-18T00:19:29.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:29.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032978+0000", 2024-09-18T00:19:29.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1320000000000002e-06 2024-09-18T00:19:29.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:29.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122520+0000", 2024-09-18T00:19:29.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089541930000000006 2024-09-18T00:19:29.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:29.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122601+0000", 2024-09-18T00:19:29.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1207999999999999e-05 2024-09-18T00:19:29.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:29.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:29.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:29.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122722+0000", 2024-09-18T00:19:29.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012162700000000001 2024-09-18T00:19:29.925 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.925 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.925 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.925 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:29.925 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:29.925 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:29.927 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:19:29.928 INFO:teuthology.orchestra.run.smithi097.stdout:5336 write 0x3475388 thru 0x34834f5 (0xe16e bytes) 2024-09-18T00:19:29.928 INFO:teuthology.orchestra.run.smithi097.stdout:5337 read 0x1c17542 thru 0x1c257cf (0xe28e bytes) 2024-09-18T00:19:29.928 INFO:teuthology.orchestra.run.smithi097.stdout:5338 punch from 0xfa9d1 to 0xfe730, (0x3d5f bytes) 2024-09-18T00:19:29.928 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:29.928 INFO:teuthology.orchestra.run.smithi097.stdout:5341 trunc from 0x34834f6 to 0xea22ffc 2024-09-18T00:19:29.928 INFO:teuthology.orchestra.run.smithi097.stdout:5343 punch from 0xd484634 to 0xd4911c2, (0xcb8e bytes) 2024-09-18T00:19:29.928 INFO:teuthology.orchestra.run.smithi097.stdout:5344 read 0xde5c581 thru 0xde5e5e8 (0x2068 bytes) 2024-09-18T00:19:29.928 INFO:teuthology.orchestra.run.smithi097.stdout:5345 trunc from 0xea22ffc to 0x67e6814 2024-09-18T00:19:29.929 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:29.921+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:29.932 INFO:teuthology.orchestra.run.smithi097.stdout:5346 read 0x16a323a thru 0x16a873f (0x5506 bytes) 2024-09-18T00:19:29.932 INFO:teuthology.orchestra.run.smithi097.stdout:5347 write 0x48fd9be thru 0x490a327 (0xc96a bytes) 2024-09-18T00:19:29.948 INFO:teuthology.orchestra.run.smithi097.stdout:5348 trunc from 0x67e6814 to 0x66f89fb 2024-09-18T00:19:29.988 INFO:teuthology.orchestra.run.smithi097.stdout:5349 write 0xdc8df7a thru 0xdc8e49e (0x525 bytes) 2024-09-18T00:19:29.993 INFO:teuthology.orchestra.run.smithi097.stdout:5350 write 0xcab3ce1 thru 0xcab9bb1 (0x5ed1 bytes) 2024-09-18T00:19:30.028 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:30.025+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:30.031 INFO:teuthology.orchestra.run.smithi097.stdout:5352 trunc from 0xdc8e49f to 0x371ede3 2024-09-18T00:19:30.052 INFO:teuthology.orchestra.run.smithi097.stdout:5354 read 0x33952e9 thru 0x33a1d7e (0xca96 bytes) 2024-09-18T00:19:30.053 INFO:teuthology.orchestra.run.smithi097.stdout:5355 read 0x30be296 thru 0x30cb4b4 (0xd21f bytes) 2024-09-18T00:19:30.061 INFO:teuthology.orchestra.run.smithi097.stdout:5356 trunc from 0x371ede3 to 0x5ffb449 2024-09-18T00:19:30.070 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:30.070 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:30.070 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:30.070 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:30.074 INFO:teuthology.orchestra.run.smithi097.stdout:5357 read 0x40582c2 thru 0x4063a9c (0xb7db bytes) 2024-09-18T00:19:30.074 INFO:teuthology.orchestra.run.smithi097.stdout:5358 read 0x1793365 thru 0x17a255a (0xf1f6 bytes) 2024-09-18T00:19:30.075 INFO:teuthology.orchestra.run.smithi097.stdout:5359 read 0x55adb75 thru 0x55b4f12 (0x739e bytes) 2024-09-18T00:19:30.084 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:19:30.093 INFO:teuthology.orchestra.run.smithi097.stdout:5361 write 0x9457b72 thru 0x945bc32 (0x40c1 bytes) 2024-09-18T00:19:30.102 INFO:teuthology.orchestra.run.smithi097.stdout:5362 read 0xafd3a7 thru 0xb06143 (0x8d9d bytes) 2024-09-18T00:19:30.102 INFO:teuthology.orchestra.run.smithi097.stdout:5363 write 0xdb6b2f thru 0xdb9fee (0x34c0 bytes) 2024-09-18T00:19:30.118 INFO:teuthology.orchestra.run.smithi097.stdout:5364 trunc from 0x945bc33 to 0x5ec4bf5 2024-09-18T00:19:30.128 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:30.121+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:30.133 INFO:teuthology.orchestra.run.smithi097.stdout:5365 read 0x12e8a1f thru 0x12ef153 (0x6735 bytes) 2024-09-18T00:19:30.148 INFO:teuthology.orchestra.run.smithi097.stdout:5366 write 0x7b01202 thru 0x7b100d8 (0xeed7 bytes) 2024-09-18T00:19:30.157 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:30.157 INFO:teuthology.orchestra.run.smithi097.stdout:5368 write 0x7081406 thru 0x708ef80 (0xdb7b bytes) 2024-09-18T00:19:30.167 INFO:teuthology.orchestra.run.smithi097.stdout:5369 read 0x7a878b3 thru 0x7a96b36 (0xf284 bytes) 2024-09-18T00:19:30.168 INFO:teuthology.orchestra.run.smithi097.stdout:5370 punch from 0x32c6a0e to 0x32ce9c3, (0x7fb5 bytes) 2024-09-18T00:19:30.168 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:30.186 INFO:teuthology.orchestra.run.smithi097.stdout:5372 write 0xa7b5195 thru 0xa7c30f0 (0xdf5c bytes) 2024-09-18T00:19:30.198 INFO:teuthology.orchestra.run.smithi097.stdout:5373 punch from 0x9f4a4cb to 0x9f4fa78, (0x55ad bytes) 2024-09-18T00:19:30.212 INFO:teuthology.orchestra.run.smithi097.stdout:5374 write 0xbda5ce5 thru 0xbdaf70b (0x9a27 bytes) 2024-09-18T00:19:30.224 INFO:teuthology.orchestra.run.smithi097.stdout:5375 read 0x3b1f57a thru 0x3b20456 (0xedd bytes) 2024-09-18T00:19:30.224 INFO:teuthology.orchestra.run.smithi097.stdout:5376 read 0x53680f2 thru 0x53697e0 (0x16ef bytes) 2024-09-18T00:19:30.224 INFO:teuthology.orchestra.run.smithi097.stdout:5377 read 0x9b382ca thru 0x9b41777 (0x94ae bytes) 2024-09-18T00:19:30.229 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:30.225+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:30.248 INFO:teuthology.orchestra.run.smithi097.stdout:5379 trunc from 0xbdaf70c to 0x272c07f 2024-09-18T00:19:30.268 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:30.268 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:30.268 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:30.268 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:30.268 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:30.279 INFO:teuthology.orchestra.run.smithi097.stdout:5380 punch from 0x1029b2f to 0x102e03b, (0x450c bytes) 2024-09-18T00:19:30.283 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:19:30.309 INFO:teuthology.orchestra.run.smithi097.stdout:5381 writesame 0xa17ff85 thru 0xa186535 data_size 0x1(0x65b1 bytes) 2024-09-18T00:19:30.309 INFO:teuthology.orchestra.run.smithi097.stdout:5382 punch from 0x5f5879f to 0x5f66ca6, (0xe507 bytes) 2024-09-18T00:19:30.309 INFO:teuthology.orchestra.run.smithi097.stdout:5383 write 0x2d2659e thru 0x2d2e25f (0x7cc2 bytes) 2024-09-18T00:19:30.318 INFO:teuthology.orchestra.run.smithi097.stdout:5384 read 0x78e0d85 thru 0x78e5525 (0x47a1 bytes) 2024-09-18T00:19:30.318 INFO:teuthology.orchestra.run.smithi097.stdout:5386 write 0x60a0b7e thru 0x60a9e6f (0x92f2 bytes) 2024-09-18T00:19:30.330 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:30.329+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:30.333 INFO:teuthology.orchestra.run.smithi097.stdout:5388 trunc from 0xa186536 to 0xbd5c24b 2024-09-18T00:19:30.348 INFO:teuthology.orchestra.run.smithi097.stdout:5389 trunc from 0xbd5c24b to 0x97899f1 2024-09-18T00:19:30.357 INFO:teuthology.orchestra.run.smithi097.stdout:5390 read 0x455ee65 thru 0x456bc92 (0xce2e bytes) 2024-09-18T00:19:30.357 INFO:teuthology.orchestra.run.smithi097.stdout:5391 write 0x351f643 thru 0x35247b7 (0x5175 bytes) 2024-09-18T00:19:30.366 INFO:teuthology.orchestra.run.smithi097.stdout:5392 read 0x52a7c40 thru 0x52a9cd0 (0x2091 bytes) 2024-09-18T00:19:30.367 INFO:teuthology.orchestra.run.smithi097.stdout:5393 clone 24 order 23 su 65536 sc 2 2024-09-18T00:19:30.414 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:30.414 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:30.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:30.415 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:30.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.415 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2229 3.10 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:30.415 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:30.415 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.149884338999996, 2024-09-18T00:19:30.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048730086999999998, 2024-09-18T00:19:30.415 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.415 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.415 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:30.415 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.415 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.415 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.416 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2229 2024-09-18T00:19:30.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.416 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:30.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:30.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263362+0000", 2024-09-18T00:19:30.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3869999999999999e-06 2024-09-18T00:19:30.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263381+0000", 2024-09-18T00:19:30.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9153000000000001e-05 2024-09-18T00:19:30.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263418+0000", 2024-09-18T00:19:30.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6346000000000001e-05 2024-09-18T00:19:30.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263435+0000", 2024-09-18T00:19:30.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7309999999999999e-05 2024-09-18T00:19:30.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263589+0000", 2024-09-18T00:19:30.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015451199999999999 2024-09-18T00:19:30.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263902+0000", 2024-09-18T00:19:30.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031291799999999997 2024-09-18T00:19:30.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:30.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.265995+0000", 2024-09-18T00:19:30.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020931809999999999 2024-09-18T00:19:30.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:30.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311788+0000", 2024-09-18T00:19:30.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045792078 2024-09-18T00:19:30.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:30.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311903+0000", 2024-09-18T00:19:30.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011510699999999999 2024-09-18T00:19:30.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:30.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312009+0000", 2024-09-18T00:19:30.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010630599999999999 2024-09-18T00:19:30.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312089+0000", 2024-09-18T00:19:30.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9789000000000005e-05 2024-09-18T00:19:30.420 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.421 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.421 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.421 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2229 3.10 e36/23 ondisk, result = 0)", 2024-09-18T00:19:30.421 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:30.421 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.130738679999993, 2024-09-18T00:19:30.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029600955000000002, 2024-09-18T00:19:30.421 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.421 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.421 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:30.421 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.421 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:30.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:30.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282507+0000", 2024-09-18T00:19:30.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9160000000000001e-06 2024-09-18T00:19:30.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282522+0000", 2024-09-18T00:19:30.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5e-05 2024-09-18T00:19:30.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282558+0000", 2024-09-18T00:19:30.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5694000000000001e-05 2024-09-18T00:19:30.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282580+0000", 2024-09-18T00:19:30.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2323e-05 2024-09-18T00:19:30.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311873+0000", 2024-09-18T00:19:30.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029293013999999999 2024-09-18T00:19:30.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311892+0000", 2024-09-18T00:19:30.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8556e-05 2024-09-18T00:19:30.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312105+0000", 2024-09-18T00:19:30.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021345199999999999 2024-09-18T00:19:30.425 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.425 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.425 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.425 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5714 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:30.425 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:30.425 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.103036177000007, 2024-09-18T00:19:30.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027448727999999999, 2024-09-18T00:19:30.425 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.425 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.425 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:30.425 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.426 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.426 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.426 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5714 2024-09-18T00:19:30.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.426 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:30.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:30.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310211+0000", 2024-09-18T00:19:30.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5639999999999998e-06 2024-09-18T00:19:30.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312678+0000", 2024-09-18T00:19:30.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002466761 2024-09-18T00:19:30.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312746+0000", 2024-09-18T00:19:30.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8046000000000005e-05 2024-09-18T00:19:30.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312774+0000", 2024-09-18T00:19:30.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7813e-05 2024-09-18T00:19:30.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312927+0000", 2024-09-18T00:19:30.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015328499999999999 2024-09-18T00:19:30.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313214+0000", 2024-09-18T00:19:30.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028683099999999998 2024-09-18T00:19:30.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:30.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313708+0000", 2024-09-18T00:19:30.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049379100000000004 2024-09-18T00:19:30.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:30.430 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336895+0000", 2024-09-18T00:19:30.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023186801 2024-09-18T00:19:30.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.430 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:30.430 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337454+0000", 2024-09-18T00:19:30.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055979399999999996 2024-09-18T00:19:30.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.430 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:30.430 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337571+0000", 2024-09-18T00:19:30.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116821 2024-09-18T00:19:30.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337656+0000", 2024-09-18T00:19:30.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4221000000000006e-05 2024-09-18T00:19:30.431 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.431 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.431 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.431 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6801 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:30.431 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:30.431 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.376199729000007, 2024-09-18T00:19:30.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051953602000000002, 2024-09-18T00:19:30.431 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.431 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.432 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:30.432 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.432 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.432 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.432 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6801 2024-09-18T00:19:30.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.432 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:30.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:30.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037046+0000", 2024-09-18T00:19:30.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.19e-06 2024-09-18T00:19:30.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037062+0000", 2024-09-18T00:19:30.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586999999999998e-05 2024-09-18T00:19:30.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037096+0000", 2024-09-18T00:19:30.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4112000000000001e-05 2024-09-18T00:19:30.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037120+0000", 2024-09-18T00:19:30.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.355e-05 2024-09-18T00:19:30.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037246+0000", 2024-09-18T00:19:30.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012652200000000001 2024-09-18T00:19:30.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037385+0000", 2024-09-18T00:19:30.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013924599999999999 2024-09-18T00:19:30.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:30.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.040881+0000", 2024-09-18T00:19:30.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034954830000000002 2024-09-18T00:19:30.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:30.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088794+0000", 2024-09-18T00:19:30.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047912897000000003 2024-09-18T00:19:30.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:30.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088828+0000", 2024-09-18T00:19:30.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3880000000000001e-05 2024-09-18T00:19:30.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:30.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088916+0000", 2024-09-18T00:19:30.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.865e-05 2024-09-18T00:19:30.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088997+0000", 2024-09-18T00:19:30.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0484999999999999e-05 2024-09-18T00:19:30.437 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.437 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.437 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.437 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:6801 3.10 e42/23 ondisk, result = 0)", 2024-09-18T00:19:30.437 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:30.437 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.362392318999994, 2024-09-18T00:19:30.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037960394000000001, 2024-09-18T00:19:30.437 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.437 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.437 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:30.437 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:30.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:30.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050852+0000", 2024-09-18T00:19:30.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:19:30.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050860+0000", 2024-09-18T00:19:30.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7729999999999999e-06 2024-09-18T00:19:30.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050891+0000", 2024-09-18T00:19:30.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0617000000000002e-05 2024-09-18T00:19:30.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050910+0000", 2024-09-18T00:19:30.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.944e-05 2024-09-18T00:19:30.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088766+0000", 2024-09-18T00:19:30.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037855633999999999 2024-09-18T00:19:30.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088784+0000", 2024-09-18T00:19:30.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7767000000000001e-05 2024-09-18T00:19:30.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088811+0000", 2024-09-18T00:19:30.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7382999999999999e-05 2024-09-18T00:19:30.441 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.441 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.441 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.441 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12594 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:30.441 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:30.441 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 54.021615781000001, 2024-09-18T00:19:30.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026492600000000002, 2024-09-18T00:19:30.441 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.441 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.441 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:30.442 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.442 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.442 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.442 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12594 2024-09-18T00:19:30.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.442 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:30.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:30.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391629+0000", 2024-09-18T00:19:30.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.86e-06 2024-09-18T00:19:30.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391637+0000", 2024-09-18T00:19:30.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3999999999999992e-06 2024-09-18T00:19:30.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.444 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.444 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391664+0000", 2024-09-18T00:19:30.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6536000000000001e-05 2024-09-18T00:19:30.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.444 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.444 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391681+0000", 2024-09-18T00:19:30.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6563e-05 2024-09-18T00:19:30.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.444 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.444 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417791+0000", 2024-09-18T00:19:30.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026110936000000001 2024-09-18T00:19:30.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417933+0000", 2024-09-18T00:19:30.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014112099999999999 2024-09-18T00:19:30.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418120+0000", 2024-09-18T00:19:30.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018718399999999999 2024-09-18T00:19:30.445 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.445 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.445 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.445 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12595 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:30.446 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:30.446 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 54.021539582999999, 2024-09-18T00:19:30.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026819428999999999, 2024-09-18T00:19:30.446 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.446 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.446 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:30.446 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.446 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.446 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.446 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12595 2024-09-18T00:19:30.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.446 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:30.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:30.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391705+0000", 2024-09-18T00:19:30.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3999999999999999e-06 2024-09-18T00:19:30.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391711+0000", 2024-09-18T00:19:30.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7459999999999997e-06 2024-09-18T00:19:30.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391733+0000", 2024-09-18T00:19:30.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2923000000000001e-05 2024-09-18T00:19:30.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391748+0000", 2024-09-18T00:19:30.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4790000000000001e-05 2024-09-18T00:19:30.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418180+0000", 2024-09-18T00:19:30.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026431888000000001 2024-09-18T00:19:30.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418328+0000", 2024-09-18T00:19:30.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000147725 2024-09-18T00:19:30.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418523+0000", 2024-09-18T00:19:30.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000194957 2024-09-18T00:19:30.450 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.450 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.450 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.450 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12596 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:30.450 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:30.450 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 54.021476614999997, 2024-09-18T00:19:30.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027220296000000001, 2024-09-18T00:19:30.451 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.451 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.451 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:30.451 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.451 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.451 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.451 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12596 2024-09-18T00:19:30.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.451 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:30.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:30.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391768+0000", 2024-09-18T00:19:30.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.283e-06 2024-09-18T00:19:30.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391773+0000", 2024-09-18T00:19:30.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3970000000000002e-06 2024-09-18T00:19:30.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391795+0000", 2024-09-18T00:19:30.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1835e-05 2024-09-18T00:19:30.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391809+0000", 2024-09-18T00:19:30.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4107000000000001e-05 2024-09-18T00:19:30.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418582+0000", 2024-09-18T00:19:30.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026772918999999999 2024-09-18T00:19:30.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418780+0000", 2024-09-18T00:19:30.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000198042 2024-09-18T00:19:30.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418987+0000", 2024-09-18T00:19:30.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000206713 2024-09-18T00:19:30.454 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.454 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.454 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.455 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12597 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:30.455 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:30.455 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 54.021416096999999, 2024-09-18T00:19:30.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027612714, 2024-09-18T00:19:30.455 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.455 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.455 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:30.455 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.455 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.455 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.455 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12597 2024-09-18T00:19:30.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.456 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:30.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:30.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391828+0000", 2024-09-18T00:19:30.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.203e-06 2024-09-18T00:19:30.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391834+0000", 2024-09-18T00:19:30.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6529999999999999e-06 2024-09-18T00:19:30.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391856+0000", 2024-09-18T00:19:30.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2195999999999999e-05 2024-09-18T00:19:30.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391870+0000", 2024-09-18T00:19:30.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4063000000000001e-05 2024-09-18T00:19:30.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419049+0000", 2024-09-18T00:19:30.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027178751000000001 2024-09-18T00:19:30.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419239+0000", 2024-09-18T00:19:30.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019039400000000001 2024-09-18T00:19:30.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419440+0000", 2024-09-18T00:19:30.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000200454 2024-09-18T00:19:30.459 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.459 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.459 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.459 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12598 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:30.459 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:30.459 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 54.021353046000002, 2024-09-18T00:19:30.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028269056000000001, 2024-09-18T00:19:30.459 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.459 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.459 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:30.460 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.460 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.460 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.460 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12598 2024-09-18T00:19:30.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.460 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:30.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:30.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391891+0000", 2024-09-18T00:19:30.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.257e-06 2024-09-18T00:19:30.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391897+0000", 2024-09-18T00:19:30.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9560000000000002e-06 2024-09-18T00:19:30.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391920+0000", 2024-09-18T00:19:30.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3025999999999999e-05 2024-09-18T00:19:30.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391935+0000", 2024-09-18T00:19:30.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4853e-05 2024-09-18T00:19:30.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419527+0000", 2024-09-18T00:19:30.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027592378000000001 2024-09-18T00:19:30.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419681+0000", 2024-09-18T00:19:30.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015316 2024-09-18T00:19:30.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420159+0000", 2024-09-18T00:19:30.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000478426 2024-09-18T00:19:30.463 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.463 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.463 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.463 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12599 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:30.464 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:30.464 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 54.021289191000001, 2024-09-18T00:19:30.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028649857000000001, 2024-09-18T00:19:30.464 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.464 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.464 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:30.464 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.464 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.464 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.464 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12599 2024-09-18T00:19:30.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.464 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:30.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:30.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391955+0000", 2024-09-18T00:19:30.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-06 2024-09-18T00:19:30.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391961+0000", 2024-09-18T00:19:30.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7100000000000004e-06 2024-09-18T00:19:30.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391982+0000", 2024-09-18T00:19:30.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1492e-05 2024-09-18T00:19:30.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391996+0000", 2024-09-18T00:19:30.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4194e-05 2024-09-18T00:19:30.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420237+0000", 2024-09-18T00:19:30.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028240818000000001 2024-09-18T00:19:30.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420430+0000", 2024-09-18T00:19:30.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019228299999999999 2024-09-18T00:19:30.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420604+0000", 2024-09-18T00:19:30.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000174117 2024-09-18T00:19:30.467 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.467 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.467 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.467 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12600 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 270336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:30.468 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:30.468 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 54.021228196999999, 2024-09-18T00:19:30.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028964586, 2024-09-18T00:19:30.468 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.468 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.468 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:30.468 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.468 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.468 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.468 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12600 2024-09-18T00:19:30.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.468 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:30.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:30.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392016+0000", 2024-09-18T00:19:30.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.31e-06 2024-09-18T00:19:30.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392022+0000", 2024-09-18T00:19:30.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5600000000000001e-06 2024-09-18T00:19:30.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392045+0000", 2024-09-18T00:19:30.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3595999999999999e-05 2024-09-18T00:19:30.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392060+0000", 2024-09-18T00:19:30.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4727e-05 2024-09-18T00:19:30.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420669+0000", 2024-09-18T00:19:30.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028608855999999998 2024-09-18T00:19:30.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420813+0000", 2024-09-18T00:19:30.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014429500000000001 2024-09-18T00:19:30.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420979+0000", 2024-09-18T00:19:30.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016624200000000001 2024-09-18T00:19:30.471 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.472 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.472 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.472 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12601 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:30.472 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:30.472 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 54.021164292000002, 2024-09-18T00:19:30.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029292832000000001, 2024-09-18T00:19:30.472 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.472 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.472 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:30.472 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.472 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.472 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.473 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12601 2024-09-18T00:19:30.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.473 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:30.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:30.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392080+0000", 2024-09-18T00:19:30.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:30.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392086+0000", 2024-09-18T00:19:30.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7570000000000001e-06 2024-09-18T00:19:30.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392105+0000", 2024-09-18T00:19:30.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9415999999999998e-05 2024-09-18T00:19:30.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392132+0000", 2024-09-18T00:19:30.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7019e-05 2024-09-18T00:19:30.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421049+0000", 2024-09-18T00:19:30.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028916659000000001 2024-09-18T00:19:30.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421204+0000", 2024-09-18T00:19:30.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155224 2024-09-18T00:19:30.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421372+0000", 2024-09-18T00:19:30.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016754699999999999 2024-09-18T00:19:30.476 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.476 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.476 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.477 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12602 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:30.477 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:30.477 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 54.021091661, 2024-09-18T00:19:30.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029612833000000002, 2024-09-18T00:19:30.477 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.477 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.477 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:30.477 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.477 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.477 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.477 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12602 2024-09-18T00:19:30.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.477 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:30.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:30.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392152+0000", 2024-09-18T00:19:30.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0470000000000001e-06 2024-09-18T00:19:30.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392158+0000", 2024-09-18T00:19:30.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:19:30.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392181+0000", 2024-09-18T00:19:30.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2629e-05 2024-09-18T00:19:30.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392222+0000", 2024-09-18T00:19:30.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0772000000000002e-05 2024-09-18T00:19:30.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421441+0000", 2024-09-18T00:19:30.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029219327 2024-09-18T00:19:30.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421608+0000", 2024-09-18T00:19:30.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000166813 2024-09-18T00:19:30.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421764+0000", 2024-09-18T00:19:30.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015628500000000001 2024-09-18T00:19:30.481 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.481 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.481 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.481 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12603 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:30.481 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:30.481 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 54.021001267000003, 2024-09-18T00:19:30.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029902524, 2024-09-18T00:19:30.481 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.481 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.481 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:30.482 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.482 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.482 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.482 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12603 2024-09-18T00:19:30.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.482 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:30.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:30.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392243+0000", 2024-09-18T00:19:30.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:30.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392260+0000", 2024-09-18T00:19:30.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6860000000000001e-05 2024-09-18T00:19:30.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392282+0000", 2024-09-18T00:19:30.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2713000000000001e-05 2024-09-18T00:19:30.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392297+0000", 2024-09-18T00:19:30.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4375e-05 2024-09-18T00:19:30.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421830+0000", 2024-09-18T00:19:30.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029532668000000002 2024-09-18T00:19:30.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421985+0000", 2024-09-18T00:19:30.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155613 2024-09-18T00:19:30.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422144+0000", 2024-09-18T00:19:30.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000159102 2024-09-18T00:19:30.485 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.485 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.485 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.486 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12604 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:30.486 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:30.486 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 54.020926215999999, 2024-09-18T00:19:30.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030228695, 2024-09-18T00:19:30.486 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.486 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.486 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:30.486 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.486 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.486 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.486 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12604 2024-09-18T00:19:30.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.486 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:30.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:30.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392318+0000", 2024-09-18T00:19:30.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:30.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392337+0000", 2024-09-18T00:19:30.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:19:30.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392364+0000", 2024-09-18T00:19:30.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7288999999999999e-05 2024-09-18T00:19:30.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392379+0000", 2024-09-18T00:19:30.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:19:30.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422209+0000", 2024-09-18T00:19:30.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029829050999999999 2024-09-18T00:19:30.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422388+0000", 2024-09-18T00:19:30.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017988699999999999 2024-09-18T00:19:30.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422545+0000", 2024-09-18T00:19:30.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015701199999999999 2024-09-18T00:19:30.490 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.490 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.490 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.490 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12605 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:30.490 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:30.490 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 54.020841654999998, 2024-09-18T00:19:30.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030488674, 2024-09-18T00:19:30.490 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.490 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.490 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:30.491 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.491 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.491 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.491 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12605 2024-09-18T00:19:30.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.491 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:30.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:30.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392403+0000", 2024-09-18T00:19:30.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.883e-06 2024-09-18T00:19:30.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392421+0000", 2024-09-18T00:19:30.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7526000000000001e-05 2024-09-18T00:19:30.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392450+0000", 2024-09-18T00:19:30.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8972e-05 2024-09-18T00:19:30.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392488+0000", 2024-09-18T00:19:30.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8522000000000001e-05 2024-09-18T00:19:30.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422614+0000", 2024-09-18T00:19:30.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030125945000000001 2024-09-18T00:19:30.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422752+0000", 2024-09-18T00:19:30.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000138178 2024-09-18T00:19:30.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422890+0000", 2024-09-18T00:19:30.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013764800000000001 2024-09-18T00:19:30.494 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.494 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.495 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.495 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12606 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:30.495 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:30.495 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 54.020732995000003, 2024-09-18T00:19:30.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033727073000000003, 2024-09-18T00:19:30.495 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.495 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.495 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:30.495 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.495 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.495 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.495 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12606 2024-09-18T00:19:30.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.496 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:30.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:30.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392511+0000", 2024-09-18T00:19:30.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3090000000000001e-06 2024-09-18T00:19:30.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392517+0000", 2024-09-18T00:19:30.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.993e-06 2024-09-18T00:19:30.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392543+0000", 2024-09-18T00:19:30.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6152999999999998e-05 2024-09-18T00:19:30.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392558+0000", 2024-09-18T00:19:30.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4610000000000001e-05 2024-09-18T00:19:30.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.425959+0000", 2024-09-18T00:19:30.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033400484000000001 2024-09-18T00:19:30.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426106+0000", 2024-09-18T00:19:30.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014779800000000001 2024-09-18T00:19:30.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426237+0000", 2024-09-18T00:19:30.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013072600000000001 2024-09-18T00:19:30.499 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.499 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.499 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.499 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17562 3.a 3:5578626c:::rbd_data.112a6efb7564.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T00:19:30.499 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:30.499 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 40.455267833999997, 2024-09-18T00:19:30.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052733453999999999, 2024-09-18T00:19:30.500 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.500 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.500 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:30.500 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:30.500 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:30.500 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:30.500 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17562 2024-09-18T00:19:30.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.500 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:30.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:30.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957977+0000", 2024-09-18T00:19:30.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.119e-06 2024-09-18T00:19:30.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957988+0000", 2024-09-18T00:19:30.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0557000000000001e-05 2024-09-18T00:19:30.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958017+0000", 2024-09-18T00:19:30.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9156e-05 2024-09-18T00:19:30.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958036+0000", 2024-09-18T00:19:30.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8919e-05 2024-09-18T00:19:30.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958144+0000", 2024-09-18T00:19:30.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107905 2024-09-18T00:19:30.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958284+0000", 2024-09-18T00:19:30.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140216 2024-09-18T00:19:30.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:30.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.960572+0000", 2024-09-18T00:19:30.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022881860000000002 2024-09-18T00:19:30.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:30.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010537+0000", 2024-09-18T00:19:30.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049964664999999998 2024-09-18T00:19:30.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:30.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010590+0000", 2024-09-18T00:19:30.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3094999999999999e-05 2024-09-18T00:19:30.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:30.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010642+0000", 2024-09-18T00:19:30.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2073000000000001e-05 2024-09-18T00:19:30.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010709+0000", 2024-09-18T00:19:30.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6563000000000006e-05 2024-09-18T00:19:30.505 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.506 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.506 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.506 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17562 3.a e60/23 ondisk, result = 0)", 2024-09-18T00:19:30.506 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:30.506 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 40.445064463000001, 2024-09-18T00:19:30.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042539307999999998, 2024-09-18T00:19:30.506 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:30.506 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:30.506 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:30.506 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:30.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:30.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:30.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:30.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:30.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:30.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:30.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968180+0000", 2024-09-18T00:19:30.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-18T00:19:30.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:30.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968188+0000", 2024-09-18T00:19:30.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7419999999999997e-06 2024-09-18T00:19:30.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:30.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968217+0000", 2024-09-18T00:19:30.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8926999999999999e-05 2024-09-18T00:19:30.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:30.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968231+0000", 2024-09-18T00:19:30.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4168999999999999e-05 2024-09-18T00:19:30.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:30.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010575+0000", 2024-09-18T00:19:30.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042343923999999998 2024-09-18T00:19:30.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:30.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010584+0000", 2024-09-18T00:19:30.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-18T00:19:30.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:30.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:30.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:30.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010718+0000", 2024-09-18T00:19:30.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013344499999999999 2024-09-18T00:19:30.509 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.510 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.510 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.510 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:30.510 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:30.510 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:30.510 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:19:30.511 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:30.509+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:30.512 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:19:30.606 INFO:teuthology.orchestra.run.smithi097.stdout:truncating image image_client.0-clone23 from 0x97899f1 (overlap 0x3c166c) to 0x0 2024-09-18T00:19:30.612 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:30.609+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:30.696 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-18T00:19:30.698 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:30.698 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:30.698 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:30.698 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:30.712 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:19:30.713 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:30.709+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:30.794 INFO:teuthology.orchestra.run.smithi097.stdout:5394 writesame 0x8d24d3 thru 0x8d7771 data_size 0x1(0x529f bytes) 2024-09-18T00:19:30.813 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:30.809+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:30.876 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:30.876 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:30.877 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:30.877 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:30.877 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:30.890 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:19:30.914 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:30.915 INFO:teuthology.orchestra.run.smithi097.stdout:5396 read 0x7dd7bc7 thru 0x7ddac43 (0x307d bytes) 2024-09-18T00:19:30.915 INFO:teuthology.orchestra.run.smithi097.stdout:skipping zero size read 2024-09-18T00:19:30.926 INFO:teuthology.orchestra.run.smithi097.stdout:5400 writesame 0x9cf9aa5 thru 0x9cfa314 data_size 0x10(0x870 bytes) 2024-09-18T00:19:30.932 INFO:teuthology.orchestra.run.smithi097.stdout:5402 write 0x2161cd0 thru 0x21658ea (0x3c1b bytes) 2024-09-18T00:19:30.934 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:30.933+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:30.991 INFO:teuthology.orchestra.run.smithi097.stdout:5403 punch from 0x8c4e89 to 0x8c63e7, (0x155e bytes) 2024-09-18T00:19:31.006 INFO:teuthology.orchestra.run.smithi097.stdout:5406 write 0xacf6b3c thru 0xacf88cd (0x1d92 bytes) 2024-09-18T00:19:31.012 INFO:teuthology.orchestra.run.smithi097.stdout:5408 read 0x57fe2cb thru 0x580b6f3 (0xd429 bytes) 2024-09-18T00:19:31.015 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:31.013+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:31.016 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:31.017 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:31.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:31.017 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:31.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.017 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2228 3.15 3:aa2fc059:::rbd_data.112aa7ba4578.000000000000014a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:31.017 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:31.017 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.750267891999997, 2024-09-18T00:19:31.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.068966377999999995, 2024-09-18T00:19:31.017 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.017 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.017 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:31.017 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.018 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.018 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.018 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2228 2024-09-18T00:19:31.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.018 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:31.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:31.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263099+0000", 2024-09-18T00:19:31.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2359999999999999e-06 2024-09-18T00:19:31.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263115+0000", 2024-09-18T00:19:31.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5996999999999999e-05 2024-09-18T00:19:31.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263156+0000", 2024-09-18T00:19:31.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:19:31.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263183+0000", 2024-09-18T00:19:31.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7233000000000002e-05 2024-09-18T00:19:31.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263304+0000", 2024-09-18T00:19:31.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121209 2024-09-18T00:19:31.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263528+0000", 2024-09-18T00:19:31.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022401800000000001 2024-09-18T00:19:31.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:19:31.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.266458+0000", 2024-09-18T00:19:31.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029296980000000001 2024-09-18T00:19:31.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:31.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331628+0000", 2024-09-18T00:19:31.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.065170045999999995 2024-09-18T00:19:31.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:31.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331819+0000", 2024-09-18T00:19:31.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000190819 2024-09-18T00:19:31.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:31.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331971+0000", 2024-09-18T00:19:31.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151972 2024-09-18T00:19:31.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332063+0000", 2024-09-18T00:19:31.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2083999999999995e-05 2024-09-18T00:19:31.023 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.023 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.023 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.023 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2854 3.f 3:f02b0bca:::rbd_data.112aa7ba4578.0000000000000135:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:19:31.023 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:31.023 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.143295402000007, 2024-09-18T00:19:31.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.082582802999999996, 2024-09-18T00:19:31.023 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.023 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.023 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:31.023 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.024 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.024 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.024 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2854 2024-09-18T00:19:31.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.024 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:31.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:31.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870071+0000", 2024-09-18T00:19:31.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.227e-06 2024-09-18T00:19:31.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870083+0000", 2024-09-18T00:19:31.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1909999999999999e-05 2024-09-18T00:19:31.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870114+0000", 2024-09-18T00:19:31.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0576e-05 2024-09-18T00:19:31.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870129+0000", 2024-09-18T00:19:31.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.485e-05 2024-09-18T00:19:31.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870256+0000", 2024-09-18T00:19:31.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012691300000000001 2024-09-18T00:19:31.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870493+0000", 2024-09-18T00:19:31.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023737399999999999 2024-09-18T00:19:31.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:31.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.874933+0000", 2024-09-18T00:19:31.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044398299999999996 2024-09-18T00:19:31.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:31.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952452+0000", 2024-09-18T00:19:31.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.077518918000000006 2024-09-18T00:19:31.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:31.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952486+0000", 2024-09-18T00:19:31.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4035999999999997e-05 2024-09-18T00:19:31.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:31.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952575+0000", 2024-09-18T00:19:31.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.954e-05 2024-09-18T00:19:31.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952652+0000", 2024-09-18T00:19:31.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6629000000000001e-05 2024-09-18T00:19:31.029 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.029 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.029 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.029 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2854 3.f e38/32 ondisk, result = 0)", 2024-09-18T00:19:31.029 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:31.029 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.125764587000006, 2024-09-18T00:19:31.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064868625999999999, 2024-09-18T00:19:31.029 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.029 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.029 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.029 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:31.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:31.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887602+0000", 2024-09-18T00:19:31.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.677e-06 2024-09-18T00:19:31.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887609+0000", 2024-09-18T00:19:31.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6729999999999992e-06 2024-09-18T00:19:31.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887637+0000", 2024-09-18T00:19:31.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8206000000000001e-05 2024-09-18T00:19:31.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887651+0000", 2024-09-18T00:19:31.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3387000000000001e-05 2024-09-18T00:19:31.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952422+0000", 2024-09-18T00:19:31.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064771366999999996 2024-09-18T00:19:31.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952441+0000", 2024-09-18T00:19:31.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8737e-05 2024-09-18T00:19:31.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952469+0000", 2024-09-18T00:19:31.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7579000000000001e-05 2024-09-18T00:19:31.033 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.033 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.033 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.033 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15879 3.16 3:692b47b6:::rbd_object_map.112a13ec2d6c.000000000000000b:head [call] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:19:31.033 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:31.033 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.358600680000002, 2024-09-18T00:19:31.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.011184018, 2024-09-18T00:19:31.033 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.034 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.034 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.034 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.034 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.034 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.034 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15879 2024-09-18T00:19:31.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.034 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:31.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:31.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654768+0000", 2024-09-18T00:19:31.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9129999999999996e-06 2024-09-18T00:19:31.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654786+0000", 2024-09-18T00:19:31.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8695999999999999e-05 2024-09-18T00:19:31.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654832+0000", 2024-09-18T00:19:31.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5735000000000002e-05 2024-09-18T00:19:31.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654860+0000", 2024-09-18T00:19:31.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7789000000000001e-05 2024-09-18T00:19:31.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655048+0000", 2024-09-18T00:19:31.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018781900000000001 2024-09-18T00:19:31.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:19:31.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655070+0000", 2024-09-18T00:19:31.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1916000000000002e-05 2024-09-18T00:19:31.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663335+0000", 2024-09-18T00:19:31.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0082651210000000003 2024-09-18T00:19:31.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:31.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663358+0000", 2024-09-18T00:19:31.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2986e-05 2024-09-18T00:19:31.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663739+0000", 2024-09-18T00:19:31.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0003814820000001 2024-09-18T00:19:31.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663920+0000", 2024-09-18T00:19:31.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018116900000000001 2024-09-18T00:19:31.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.665948+0000", 2024-09-18T00:19:31.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020273919999999998 2024-09-18T00:19:31.039 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.039 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.039 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.039 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15881 3.16 3:6cc14920:::rbd_header.112a3285400b:head [watch reconnect cookie 140685950184608 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:19:31.039 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:31.039 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.358439685, 2024-09-18T00:19:31.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0138723599999999, 2024-09-18T00:19:31.039 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.039 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.039 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.039 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.040 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.040 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.040 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15881 2024-09-18T00:19:31.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.040 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:31.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:31.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654926+0000", 2024-09-18T00:19:31.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6470000000000001e-06 2024-09-18T00:19:31.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654935+0000", 2024-09-18T00:19:31.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8859999999999993e-06 2024-09-18T00:19:31.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654960+0000", 2024-09-18T00:19:31.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4689999999999999e-05 2024-09-18T00:19:31.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654976+0000", 2024-09-18T00:19:31.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6319000000000001e-05 2024-09-18T00:19:31.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655238+0000", 2024-09-18T00:19:31.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026145999999999998 2024-09-18T00:19:31.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:31.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655255+0000", 2024-09-18T00:19:31.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7249999999999999e-05 2024-09-18T00:19:31.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663416+0000", 2024-09-18T00:19:31.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081607959999999997 2024-09-18T00:19:31.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:31.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663436+0000", 2024-09-18T00:19:31.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0384e-05 2024-09-18T00:19:31.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666184+0000", 2024-09-18T00:19:31.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002748121 2024-09-18T00:19:31.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666452+0000", 2024-09-18T00:19:31.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026801700000000001 2024-09-18T00:19:31.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.668797+0000", 2024-09-18T00:19:31.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023447899999999998 2024-09-18T00:19:31.045 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.045 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.045 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.046 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17089 3.f 3:f7681386:::rbd_data.112a6efb7564.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:19:31.046 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:31.046 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.133512588999999, 2024-09-18T00:19:31.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.12573889399999999, 2024-09-18T00:19:31.046 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.046 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.046 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:31.046 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.046 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.046 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.046 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17089 2024-09-18T00:19:31.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.046 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:31.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:31.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879854+0000", 2024-09-18T00:19:31.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:19:31.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879867+0000", 2024-09-18T00:19:31.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3103e-05 2024-09-18T00:19:31.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879898+0000", 2024-09-18T00:19:31.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0103e-05 2024-09-18T00:19:31.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879912+0000", 2024-09-18T00:19:31.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4636e-05 2024-09-18T00:19:31.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879992+0000", 2024-09-18T00:19:31.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9814000000000005e-05 2024-09-18T00:19:31.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.880101+0000", 2024-09-18T00:19:31.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109172 2024-09-18T00:19:31.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:31.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.887269+0000", 2024-09-18T00:19:31.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071679300000000003 2024-09-18T00:19:31.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:31.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005269+0000", 2024-09-18T00:19:31.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.117999621 2024-09-18T00:19:31.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:31.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005317+0000", 2024-09-18T00:19:31.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8279000000000003e-05 2024-09-18T00:19:31.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:31.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005464+0000", 2024-09-18T00:19:31.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014661200000000001 2024-09-18T00:19:31.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005591+0000", 2024-09-18T00:19:31.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000127107 2024-09-18T00:19:31.051 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.052 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.052 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.052 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17089 3.f e59/32 ondisk, result = 0)", 2024-09-18T00:19:31.052 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:31.052 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.109405723999998, 2024-09-18T00:19:31.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101334403, 2024-09-18T00:19:31.052 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.052 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.052 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.052 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:31.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:31.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903960+0000", 2024-09-18T00:19:31.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.33e-06 2024-09-18T00:19:31.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903966+0000", 2024-09-18T00:19:31.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5929999999999996e-06 2024-09-18T00:19:31.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903986+0000", 2024-09-18T00:19:31.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.018e-05 2024-09-18T00:19:31.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903997+0000", 2024-09-18T00:19:31.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0988999999999999e-05 2024-09-18T00:19:31.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005171+0000", 2024-09-18T00:19:31.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101174267 2024-09-18T00:19:31.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005200+0000", 2024-09-18T00:19:31.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8540000000000001e-05 2024-09-18T00:19:31.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005293+0000", 2024-09-18T00:19:31.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504000000000005e-05 2024-09-18T00:19:31.056 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.056 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.056 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.056 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24529 3.16 3:6857ce1a:::rbd_data.112a990280ef.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:31.056 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:31.056 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.376594312999998, 2024-09-18T00:19:31.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10693805200000001, 2024-09-18T00:19:31.056 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.056 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.057 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:31.057 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.057 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.057 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.057 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24529 2024-09-18T00:19:31.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.057 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:31.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636772+0000", 2024-09-18T00:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1100000000000001e-06 2024-09-18T00:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636783+0000", 2024-09-18T00:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1005999999999999e-05 2024-09-18T00:19:31.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636814+0000", 2024-09-18T00:19:31.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1047000000000003e-05 2024-09-18T00:19:31.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636829+0000", 2024-09-18T00:19:31.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4363e-05 2024-09-18T00:19:31.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636980+0000", 2024-09-18T00:19:31.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015181099999999999 2024-09-18T00:19:31.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.637237+0000", 2024-09-18T00:19:31.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000256352 2024-09-18T00:19:31.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:31.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743536+0000", 2024-09-18T00:19:31.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.106298825 2024-09-18T00:19:31.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:31.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743599+0000", 2024-09-18T00:19:31.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3449999999999997e-05 2024-09-18T00:19:31.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743708+0000", 2024-09-18T00:19:31.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109088 2024-09-18T00:19:31.061 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.061 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.061 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.061 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24805 3.16 3:6de5e15d:::rbd_data.112a990280ef.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:31.062 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:31.062 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.790791080999998, 2024-09-18T00:19:31.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14775107700000001, 2024-09-18T00:19:31.062 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.062 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.062 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:31.062 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.062 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.062 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.062 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24805 2024-09-18T00:19:31.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.062 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:31.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:31.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222574+0000", 2024-09-18T00:19:31.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0210000000000001e-06 2024-09-18T00:19:31.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222581+0000", 2024-09-18T00:19:31.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.144e-06 2024-09-18T00:19:31.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222595+0000", 2024-09-18T00:19:31.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4567e-05 2024-09-18T00:19:31.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222603+0000", 2024-09-18T00:19:31.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6529999999999994e-06 2024-09-18T00:19:31.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222687+0000", 2024-09-18T00:19:31.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4587999999999997e-05 2024-09-18T00:19:31.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222828+0000", 2024-09-18T00:19:31.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140178 2024-09-18T00:19:31.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:31.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370231+0000", 2024-09-18T00:19:31.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14740378700000001 2024-09-18T00:19:31.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:31.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370269+0000", 2024-09-18T00:19:31.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7886999999999998e-05 2024-09-18T00:19:31.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370324+0000", 2024-09-18T00:19:31.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5251999999999999e-05 2024-09-18T00:19:31.066 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.067 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.067 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.067 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24852 3.13 3:cc64f0fd:::rbd_data.112a990280ef.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:31.067 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:31.067 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.558051611, 2024-09-18T00:19:31.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.066392943999999995, 2024-09-18T00:19:31.067 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.067 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.067 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:31.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.068 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24852 2024-09-18T00:19:31.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.068 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:31.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:31.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455315+0000", 2024-09-18T00:19:31.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.705e-06 2024-09-18T00:19:31.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455324+0000", 2024-09-18T00:19:31.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8120000000000002e-06 2024-09-18T00:19:31.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455349+0000", 2024-09-18T00:19:31.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4250999999999999e-05 2024-09-18T00:19:31.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455361+0000", 2024-09-18T00:19:31.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2805e-05 2024-09-18T00:19:31.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455425+0000", 2024-09-18T00:19:31.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3981999999999998e-05 2024-09-18T00:19:31.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455480+0000", 2024-09-18T00:19:31.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4792e-05 2024-09-18T00:19:31.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:31.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.460671+0000", 2024-09-18T00:19:31.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051907300000000002 2024-09-18T00:19:31.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:31.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521355+0000", 2024-09-18T00:19:31.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060684208000000003 2024-09-18T00:19:31.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:31.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521493+0000", 2024-09-18T00:19:31.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013803200000000001 2024-09-18T00:19:31.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:31.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521590+0000", 2024-09-18T00:19:31.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6339e-05 2024-09-18T00:19:31.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521706+0000", 2024-09-18T00:19:31.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116288 2024-09-18T00:19:31.072 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.072 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.072 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.073 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:25269 3.f 3:f6adfd48:::rbd_data.112aa34d8ec3.0000000000000145:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T00:19:31.073 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:31.073 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.40846165, 2024-09-18T00:19:31.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.075209085999999994, 2024-09-18T00:19:31.073 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.073 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.073 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:31.073 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.073 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.073 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.074 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 25269 2024-09-18T00:19:31.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.074 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:31.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:31.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604904+0000", 2024-09-18T00:19:31.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:19:31.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604910+0000", 2024-09-18T00:19:31.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.964e-06 2024-09-18T00:19:31.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604925+0000", 2024-09-18T00:19:31.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4894e-05 2024-09-18T00:19:31.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604932+0000", 2024-09-18T00:19:31.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-18T00:19:31.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604990+0000", 2024-09-18T00:19:31.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8143000000000002e-05 2024-09-18T00:19:31.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.605104+0000", 2024-09-18T00:19:31.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011404399999999999 2024-09-18T00:19:31.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:31.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.608755+0000", 2024-09-18T00:19:31.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003651081 2024-09-18T00:19:31.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:31.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679920+0000", 2024-09-18T00:19:31.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.071164543999999996 2024-09-18T00:19:31.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:31.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679978+0000", 2024-09-18T00:19:31.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8585000000000002e-05 2024-09-18T00:19:31.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:31.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680036+0000", 2024-09-18T00:19:31.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7978e-05 2024-09-18T00:19:31.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680112+0000", 2024-09-18T00:19:31.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5527999999999999e-05 2024-09-18T00:19:31.078 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:25269 3.f e72/32 ondisk, result = 0)", 2024-09-18T00:19:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.395251264999999, 2024-09-18T00:19:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062014761000000002, 2024-09-18T00:19:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618114+0000", 2024-09-18T00:19:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1650000000000001e-06 2024-09-18T00:19:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618119+0000", 2024-09-18T00:19:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-18T00:19:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618134+0000", 2024-09-18T00:19:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5770999999999999e-05 2024-09-18T00:19:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618143+0000", 2024-09-18T00:19:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.551e-06 2024-09-18T00:19:31.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679967+0000", 2024-09-18T00:19:31.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061823937000000002 2024-09-18T00:19:31.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679975+0000", 2024-09-18T00:19:31.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7340000000000008e-06 2024-09-18T00:19:31.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680128+0000", 2024-09-18T00:19:31.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015338299999999999 2024-09-18T00:19:31.083 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.083 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.083 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.083 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48483 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 15671296~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:31.083 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.367214+0000", 2024-09-18T00:19:31.083 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6461505619999999, 2024-09-18T00:19:31.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058250745999999999, 2024-09-18T00:19:31.083 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.083 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.083 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.084 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.084 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.084 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.084 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48483 2024-09-18T00:19:31.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.084 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367214+0000", 2024-09-18T00:19:31.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367214+0000", 2024-09-18T00:19:31.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367280+0000", 2024-09-18T00:19:31.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6088999999999997e-05 2024-09-18T00:19:31.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367284+0000", 2024-09-18T00:19:31.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0489999999999997e-06 2024-09-18T00:19:31.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367295+0000", 2024-09-18T00:19:31.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0908e-05 2024-09-18T00:19:31.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367303+0000", 2024-09-18T00:19:31.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2199999999999992e-06 2024-09-18T00:19:31.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425271+0000", 2024-09-18T00:19:31.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.057968325000000001 2024-09-18T00:19:31.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425358+0000", 2024-09-18T00:19:31.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6157999999999997e-05 2024-09-18T00:19:31.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425465+0000", 2024-09-18T00:19:31.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010699700000000001 2024-09-18T00:19:31.087 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.087 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.087 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.087 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48484 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 15704064~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:31.088 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:19:31.088 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6460491560000001, 2024-09-18T00:19:31.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058450876999999998, 2024-09-18T00:19:31.088 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.088 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.088 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.088 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.088 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.088 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.088 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48484 2024-09-18T00:19:31.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.088 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:19:31.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:19:31.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370429+0000", 2024-09-18T00:19:31.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031138530000000002 2024-09-18T00:19:31.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370435+0000", 2024-09-18T00:19:31.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6269999999999997e-06 2024-09-18T00:19:31.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370455+0000", 2024-09-18T00:19:31.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0426999999999999e-05 2024-09-18T00:19:31.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370465+0000", 2024-09-18T00:19:31.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0118999999999999e-05 2024-09-18T00:19:31.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425525+0000", 2024-09-18T00:19:31.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055059943 2024-09-18T00:19:31.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425645+0000", 2024-09-18T00:19:31.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119392 2024-09-18T00:19:31.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425766+0000", 2024-09-18T00:19:31.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012151599999999999 2024-09-18T00:19:31.092 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.092 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.092 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.092 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48502 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16293888~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:31.092 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371167+0000", 2024-09-18T00:19:31.092 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6421971519999996, 2024-09-18T00:19:31.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058236384000000002, 2024-09-18T00:19:31.093 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.093 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.093 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.093 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.093 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.093 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.093 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48502 2024-09-18T00:19:31.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.093 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371167+0000", 2024-09-18T00:19:31.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371167+0000", 2024-09-18T00:19:31.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371168+0000", 2024-09-18T00:19:31.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8400000000000004e-07 2024-09-18T00:19:31.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371171+0000", 2024-09-18T00:19:31.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9189999999999999e-06 2024-09-18T00:19:31.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371180+0000", 2024-09-18T00:19:31.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7320000000000004e-06 2024-09-18T00:19:31.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371187+0000", 2024-09-18T00:19:31.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0700000000000001e-06 2024-09-18T00:19:31.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429295+0000", 2024-09-18T00:19:31.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058108805999999999 2024-09-18T00:19:31.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429353+0000", 2024-09-18T00:19:31.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7445000000000003e-05 2024-09-18T00:19:31.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429404+0000", 2024-09-18T00:19:31.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0828000000000002e-05 2024-09-18T00:19:31.096 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.096 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.097 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.097 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48503 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16326656~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:31.097 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:31.097 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6421687550000001, 2024-09-18T00:19:31.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058310395000000001, 2024-09-18T00:19:31.097 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.097 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.097 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.097 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.097 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.097 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.097 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48503 2024-09-18T00:19:31.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0100000000000004e-07 2024-09-18T00:19:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371199+0000", 2024-09-18T00:19:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8899999999999999e-06 2024-09-18T00:19:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371211+0000", 2024-09-18T00:19:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1707e-05 2024-09-18T00:19:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371218+0000", 2024-09-18T00:19:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.131e-06 2024-09-18T00:19:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429426+0000", 2024-09-18T00:19:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058207771999999998 2024-09-18T00:19:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429467+0000", 2024-09-18T00:19:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1532999999999997e-05 2024-09-18T00:19:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429506+0000", 2024-09-18T00:19:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8661000000000001e-05 2024-09-18T00:19:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48504 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16359424~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.642136936, 2024-09-18T00:19:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058412775, 2024-09-18T00:19:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.101 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48504 2024-09-18T00:19:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:31.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:31.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6899999999999996e-07 2024-09-18T00:19:31.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371231+0000", 2024-09-18T00:19:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6709999999999999e-06 2024-09-18T00:19:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371240+0000", 2024-09-18T00:19:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7309999999999999e-06 2024-09-18T00:19:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371249+0000", 2024-09-18T00:19:31.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4280000000000002e-06 2024-09-18T00:19:31.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429533+0000", 2024-09-18T00:19:31.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058283852999999997 2024-09-18T00:19:31.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429590+0000", 2024-09-18T00:19:31.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6824999999999999e-05 2024-09-18T00:19:31.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429640+0000", 2024-09-18T00:19:31.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0498000000000003e-05 2024-09-18T00:19:31.106 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.106 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.106 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.107 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48505 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16392192~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:31.107 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:31.107 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6421039630000003, 2024-09-18T00:19:31.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058482777999999999, 2024-09-18T00:19:31.107 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.107 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.107 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.107 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48505 2024-09-18T00:19:31.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7800000000000001e-07 2024-09-18T00:19:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371264+0000", 2024-09-18T00:19:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-18T00:19:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371274+0000", 2024-09-18T00:19:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4690000000000003e-06 2024-09-18T00:19:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371281+0000", 2024-09-18T00:19:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4120000000000004e-06 2024-09-18T00:19:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429665+0000", 2024-09-18T00:19:31.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058384108999999997 2024-09-18T00:19:31.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429705+0000", 2024-09-18T00:19:31.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0299000000000002e-05 2024-09-18T00:19:31.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429743+0000", 2024-09-18T00:19:31.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7951000000000003e-05 2024-09-18T00:19:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48506 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16424960~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:19:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.642074751, 2024-09-18T00:19:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058604485999999997, 2024-09-18T00:19:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.112 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48506 2024-09-18T00:19:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:19:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:19:31.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371355+0000", 2024-09-18T00:19:31.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5549000000000005e-05 2024-09-18T00:19:31.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371359+0000", 2024-09-18T00:19:31.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1049999999999997e-06 2024-09-18T00:19:31.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371380+0000", 2024-09-18T00:19:31.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0205000000000001e-05 2024-09-18T00:19:31.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371388+0000", 2024-09-18T00:19:31.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3229999999999998e-06 2024-09-18T00:19:31.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429771+0000", 2024-09-18T00:19:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058382878999999999 2024-09-18T00:19:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429842+0000", 2024-09-18T00:19:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1110000000000002e-05 2024-09-18T00:19:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429894+0000", 2024-09-18T00:19:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2315000000000003e-05 2024-09-18T00:19:31.116 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.117 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.117 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.117 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48507 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16457728~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:31.117 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:19:31.117 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.641963391, 2024-09-18T00:19:31.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058597275999999997, 2024-09-18T00:19:31.117 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.117 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48507 2024-09-18T00:19:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:19:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:19:31.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374394+0000", 2024-09-18T00:19:31.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029924449999999998 2024-09-18T00:19:31.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374400+0000", 2024-09-18T00:19:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0229999999999998e-06 2024-09-18T00:19:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374419+0000", 2024-09-18T00:19:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9488e-05 2024-09-18T00:19:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374432+0000", 2024-09-18T00:19:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2492e-05 2024-09-18T00:19:31.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429917+0000", 2024-09-18T00:19:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055485087000000002 2024-09-18T00:19:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429958+0000", 2024-09-18T00:19:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1258999999999999e-05 2024-09-18T00:19:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429998+0000", 2024-09-18T00:19:31.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0482000000000003e-05 2024-09-18T00:19:31.122 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.122 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.122 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.122 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.122 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.122 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:31.124 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:19:31.125 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:31.121+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:31.126 INFO:teuthology.orchestra.run.smithi097.stdout:5409 writesame 0xbd7028d thru 0xbd77bfa data_size 0x42(0x796e bytes) 2024-09-18T00:19:31.126 INFO:teuthology.orchestra.run.smithi097.stdout:5411 read 0x907f53c thru 0x908cfce (0xda93 bytes) 2024-09-18T00:19:31.126 INFO:teuthology.orchestra.run.smithi097.stdout:5412 write 0x9d46426 thru 0x9d4e780 (0x835b bytes) 2024-09-18T00:19:31.126 INFO:teuthology.orchestra.run.smithi097.stdout:5414 read 0x1047951 thru 0x1051277 (0x9927 bytes) 2024-09-18T00:19:31.126 INFO:teuthology.orchestra.run.smithi097.stdout:5416 read 0x243bf26 thru 0x24495d6 (0xd6b1 bytes) 2024-09-18T00:19:31.126 INFO:teuthology.orchestra.run.smithi097.stdout:5417 trunc from 0xbd77bfb to 0x9f335ea 2024-09-18T00:19:31.126 INFO:teuthology.orchestra.run.smithi097.stdout:5418 read 0x7ef16bd thru 0x7ef4ad3 (0x3417 bytes) 2024-09-18T00:19:31.126 INFO:teuthology.orchestra.run.smithi097.stdout:5419 write 0x2847c74 thru 0x284e822 (0x6baf bytes) 2024-09-18T00:19:31.126 INFO:teuthology.orchestra.run.smithi097.stdout:5420 writesame 0x5559f02 thru 0x55646ec data_size 0x59(0xa7eb bytes) 2024-09-18T00:19:31.126 INFO:teuthology.orchestra.run.smithi097.stdout:5422 read 0x38503fd thru 0x385493e (0x4542 bytes) 2024-09-18T00:19:31.126 INFO:teuthology.orchestra.run.smithi097.stdout:5423 read 0x8020c22 thru 0x8027799 (0x6b78 bytes) 2024-09-18T00:19:31.126 INFO:teuthology.orchestra.run.smithi097.stdout:5424 read 0x83ba369 thru 0x83c9e70 (0xfb08 bytes) 2024-09-18T00:19:31.127 INFO:teuthology.orchestra.run.smithi097.stdout:5427 read 0x4c37925 thru 0x4c43eab (0xc587 bytes) 2024-09-18T00:19:31.127 INFO:teuthology.orchestra.run.smithi097.stdout:5429 write 0x65f444a thru 0x65f7cce (0x3885 bytes) 2024-09-18T00:19:31.128 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:31.147 INFO:teuthology.orchestra.run.smithi097.stdout:5432 trunc from 0x9f335ea to 0x2372388 2024-09-18T00:19:31.219 INFO:teuthology.orchestra.run.smithi097.stdout:5435 trunc from 0x2372388 to 0xa781f7 2024-09-18T00:19:31.225 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:31.221+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:31.300 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:31.300 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:31.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.300 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4394.0:55389 3.d e90/79 3:b12bf1e9:::rbd_data.112a859a01c9.0000000000000000:head v 90'660, mlcod=90'658)", 2024-09-18T00:19:31.301 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:31.269772+0000", 2024-09-18T00:19:31.301 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.030085509999999999, 2024-09-18T00:19:31.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030128293, 2024-09-18T00:19:31.301 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.301 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.301 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.301 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.269772+0000", 2024-09-18T00:19:31.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.269772+0000", 2024-09-18T00:19:31.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.269775+0000", 2024-09-18T00:19:31.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4229999999999999e-06 2024-09-18T00:19:31.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.273039+0000", 2024-09-18T00:19:31.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032642719999999999 2024-09-18T00:19:31.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.273076+0000", 2024-09-18T00:19:31.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7459999999999997e-05 2024-09-18T00:19:31.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.273093+0000", 2024-09-18T00:19:31.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6878999999999999e-05 2024-09-18T00:19:31.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.273230+0000", 2024-09-18T00:19:31.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013711600000000001 2024-09-18T00:19:31.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.273270+0000", 2024-09-18T00:19:31.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9301999999999997e-05 2024-09-18T00:19:31.304 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.304 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.305 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.305 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.305 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-18T00:19:31.305 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 1 2024-09-18T00:19:31.305 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:31.317 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:19:31.323 INFO:teuthology.orchestra.run.smithi097.stdout:5436 read 0xa5b879 thru 0xa67649 (0xbdd1 bytes) 2024-09-18T00:19:31.355 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:19:31.355 INFO:teuthology.orchestra.run.smithi097.stdout:5437 trunc from 0xa781f7 to 0xc85f401 2024-09-18T00:19:31.366 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:31.365+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:31.366 INFO:teuthology.orchestra.run.smithi097.stdout:5440 write 0x6be78bc thru 0x6bf60d1 (0xe816 bytes) 2024-09-18T00:19:31.373 INFO:teuthology.orchestra.run.smithi097.stdout:5441 read 0x6c85ad5 thru 0x6c883ac (0x28d8 bytes) 2024-09-18T00:19:31.373 INFO:teuthology.orchestra.run.smithi097.stdout:5442 write 0x2e20e88 thru 0x2e2a779 (0x98f2 bytes) 2024-09-18T00:19:31.382 INFO:teuthology.orchestra.run.smithi097.stdout:5443 write 0xc571567 thru 0xc57b453 (0x9eed bytes) 2024-09-18T00:19:31.389 INFO:teuthology.orchestra.run.smithi097.stdout:5445 write 0x5e4036a thru 0x5e4d784 (0xd41b bytes) 2024-09-18T00:19:31.408 INFO:teuthology.orchestra.run.smithi097.stdout:5447 write 0xcabdc04 thru 0xcac02a0 (0x269d bytes) 2024-09-18T00:19:31.413 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:31.413 INFO:teuthology.orchestra.run.smithi097.stdout:5449 write 0xaa2aacf thru 0xaa3457c (0x9aae bytes) 2024-09-18T00:19:31.423 INFO:teuthology.orchestra.run.smithi097.stdout:5450 trunc from 0xcac02a1 to 0xd7a1499 2024-09-18T00:19:31.426 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:31.421+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:31.441 INFO:teuthology.orchestra.run.smithi097.stdout:5451 trunc from 0xd7a1499 to 0x9ded367 2024-09-18T00:19:31.444 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:31.444 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:31.444 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:31.444 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:31.444 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:31.461 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is set 2024-09-18T00:19:31.462 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:19:31.476 INFO:teuthology.orchestra.run.smithi097.stdout:5452 punch from 0x45de79 to 0x4660a1, (0x8228 bytes) 2024-09-18T00:19:31.476 INFO:teuthology.orchestra.run.smithi097.stdout:5453 write 0x90ffdc8 thru 0x910556a (0x57a3 bytes) 2024-09-18T00:19:31.486 INFO:teuthology.orchestra.run.smithi097.stdout:5455 punch from 0x58dc6f5 to 0x58ead77, (0xe682 bytes) 2024-09-18T00:19:31.486 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:31.487 INFO:teuthology.orchestra.run.smithi097.stdout:5458 read 0x99235c6 thru 0x9926020 (0x2a5b bytes) 2024-09-18T00:19:31.487 INFO:teuthology.orchestra.run.smithi097.stdout:5459 punch from 0x690fcdb to 0x6912003, (0x2328 bytes) 2024-09-18T00:19:31.487 INFO:teuthology.orchestra.run.smithi097.stdout:5460 writesame 0x1785eb thru 0x17e1b6 data_size 0x5(0x5bcc bytes) 2024-09-18T00:19:31.497 INFO:teuthology.orchestra.run.smithi097.stdout:5461 writesame 0x6456754 thru 0x6458867 data_size 0x1d(0x2114 bytes) 2024-09-18T00:19:31.500 INFO:teuthology.orchestra.run.smithi097.stdout:5464 writesame 0x7494865 thru 0x749b379 data_size 0x1(0x6b15 bytes) 2024-09-18T00:19:31.516 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:31.516 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:31.516 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:31.526 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:31.521+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:31.533 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:31.534 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:31.558 INFO:teuthology.orchestra.run.smithi097.stdout:5465 writesame 0x609357f thru 0x60a09c4 data_size 0x9(0xd446 bytes) 2024-09-18T00:19:31.562 INFO:teuthology.orchestra.run.smithi097.stdout:5467 read 0x8a8b17d thru 0x8a956fa (0xa57e bytes) 2024-09-18T00:19:31.563 INFO:teuthology.orchestra.run.smithi097.stdout:5468 write 0x56792ee thru 0x567a161 (0xe74 bytes) 2024-09-18T00:19:31.566 INFO:teuthology.orchestra.run.smithi097.stdout:5471 read 0x7f1af6 thru 0x7f3f17 (0x2422 bytes) 2024-09-18T00:19:31.566 INFO:teuthology.orchestra.run.smithi097.stdout:5474 read 0x884167 thru 0x88dc8d (0x9b27 bytes) 2024-09-18T00:19:31.580 INFO:teuthology.orchestra.run.smithi097.stdout:5475 writesame 0xbf29086 thru 0xbf328f8 data_size 0x1(0x9873 bytes) 2024-09-18T00:19:31.580 INFO:teuthology.orchestra.run.smithi097.stdout:5476 punch from 0x72fc697 to 0x7308a39, (0xc3a2 bytes) 2024-09-18T00:19:31.580 INFO:teuthology.orchestra.run.smithi097.stdout:5479 punch from 0x9a67f94 to 0x9a6d3e8, (0x5454 bytes) 2024-09-18T00:19:31.581 INFO:teuthology.orchestra.run.smithi097.stdout:5480 write 0x6cbd7c2 thru 0x6cc034d (0x2b8c bytes) 2024-09-18T00:19:31.582 INFO:teuthology.orchestra.run.smithi097.stdout:5481 writesame 0x878082e thru 0x878b743 data_size 0x2(0xaf16 bytes) 2024-09-18T00:19:31.627 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:31.621+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:31.630 INFO:teuthology.orchestra.run.smithi097.stdout:5482 punch from 0x8dcdf32 to 0x8dd4999, (0x6a67 bytes) 2024-09-18T00:19:31.630 INFO:teuthology.orchestra.run.smithi097.stdout:5483 writesame 0x6d3a4db thru 0x6d453a7 data_size 0x1(0xaecd bytes) 2024-09-18T00:19:31.634 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34543 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:19:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.339701770000001, 2024-09-18T00:19:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023207593, 2024-09-18T00:19:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34543 2024-09-18T00:19:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:19:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:19:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290783+0000", 2024-09-18T00:19:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5240000000000001e-06 2024-09-18T00:19:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290799+0000", 2024-09-18T00:19:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:19:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290840+0000", 2024-09-18T00:19:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0926000000000001e-05 2024-09-18T00:19:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290868+0000", 2024-09-18T00:19:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8163000000000001e-05 2024-09-18T00:19:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291294+0000", 2024-09-18T00:19:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00042588700000000001 2024-09-18T00:19:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:19:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:19:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.986e-05 2024-09-18T00:19:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316617+0000", 2024-09-18T00:19:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025303583000000001 2024-09-18T00:19:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316637+0000", 2024-09-18T00:19:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9388000000000001e-05 2024-09-18T00:19:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313933+0000", 2024-09-18T00:19:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99729681000000003 2024-09-18T00:19:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313987+0000", 2024-09-18T00:19:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4069e-05 2024-09-18T00:19:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34544 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:19:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.339553415000001, 2024-09-18T00:19:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0232412120000001, 2024-09-18T00:19:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34544 2024-09-18T00:19:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:19:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:19:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290930+0000", 2024-09-18T00:19:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5030000000000001e-06 2024-09-18T00:19:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290940+0000", 2024-09-18T00:19:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9000000000000001e-06 2024-09-18T00:19:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290964+0000", 2024-09-18T00:19:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4273e-05 2024-09-18T00:19:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290982+0000", 2024-09-18T00:19:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8216e-05 2024-09-18T00:19:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291433+0000", 2024-09-18T00:19:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000451278 2024-09-18T00:19:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291446+0000", 2024-09-18T00:19:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3039e-05 2024-09-18T00:19:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316687+0000", 2024-09-18T00:19:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025240069 2024-09-18T00:19:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316701+0000", 2024-09-18T00:19:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4100000000000001e-05 2024-09-18T00:19:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314142+0000", 2024-09-18T00:19:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99744119799999997 2024-09-18T00:19:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314169+0000", 2024-09-18T00:19:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7636000000000001e-05 2024-09-18T00:19:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34545 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:19:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.339474892999998, 2024-09-18T00:19:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023257802, 2024-09-18T00:19:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34545 2024-09-18T00:19:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:19:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:19:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291008+0000", 2024-09-18T00:19:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.237e-06 2024-09-18T00:19:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291015+0000", 2024-09-18T00:19:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2030000000000003e-06 2024-09-18T00:19:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291035+0000", 2024-09-18T00:19:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.978e-05 2024-09-18T00:19:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291049+0000", 2024-09-18T00:19:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4429e-05 2024-09-18T00:19:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291630+0000", 2024-09-18T00:19:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00058103199999999999 2024-09-18T00:19:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291644+0000", 2024-09-18T00:19:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.342e-05 2024-09-18T00:19:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316753+0000", 2024-09-18T00:19:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025108703999999999 2024-09-18T00:19:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316766+0000", 2024-09-18T00:19:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3237e-05 2024-09-18T00:19:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314238+0000", 2024-09-18T00:19:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99747232699999999 2024-09-18T00:19:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314265+0000", 2024-09-18T00:19:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6432999999999999e-05 2024-09-18T00:19:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34546 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:19:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.339416554, 2024-09-18T00:19:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023324535, 2024-09-18T00:19:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34546 2024-09-18T00:19:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:19:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:19:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291066+0000", 2024-09-18T00:19:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.569999999999999e-07 2024-09-18T00:19:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291073+0000", 2024-09-18T00:19:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.063e-06 2024-09-18T00:19:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291090+0000", 2024-09-18T00:19:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6977e-05 2024-09-18T00:19:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291105+0000", 2024-09-18T00:19:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5166e-05 2024-09-18T00:19:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291771+0000", 2024-09-18T00:19:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00066599699999999999 2024-09-18T00:19:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291785+0000", 2024-09-18T00:19:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.327e-05 2024-09-18T00:19:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316826+0000", 2024-09-18T00:19:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025041955000000001 2024-09-18T00:19:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316840+0000", 2024-09-18T00:19:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3256e-05 2024-09-18T00:19:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314335+0000", 2024-09-18T00:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99749496100000001 2024-09-18T00:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314390+0000", 2024-09-18T00:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4932999999999997e-05 2024-09-18T00:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34547 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.339357254999999, 2024-09-18T00:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.028510467, 2024-09-18T00:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34547 2024-09-18T00:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:19:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:19:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291126+0000", 2024-09-18T00:19:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2699999999999999e-06 2024-09-18T00:19:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291150+0000", 2024-09-18T00:19:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4669999999999999e-05 2024-09-18T00:19:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291171+0000", 2024-09-18T00:19:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0706000000000001e-05 2024-09-18T00:19:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291186+0000", 2024-09-18T00:19:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5126999999999999e-05 2024-09-18T00:19:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291885+0000", 2024-09-18T00:19:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00069850299999999995 2024-09-18T00:19:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291897+0000", 2024-09-18T00:19:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-05 2024-09-18T00:19:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316887+0000", 2024-09-18T00:19:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024990155999999999 2024-09-18T00:19:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316900+0000", 2024-09-18T00:19:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.295e-05 2024-09-18T00:19:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319594+0000", 2024-09-18T00:19:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0026940719999999 2024-09-18T00:19:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319635+0000", 2024-09-18T00:19:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0583000000000001e-05 2024-09-18T00:19:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34548 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:19:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.339209796999999, 2024-09-18T00:19:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0285287400000001, 2024-09-18T00:19:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34548 2024-09-18T00:19:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:19:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:19:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291274+0000", 2024-09-18T00:19:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.12e-06 2024-09-18T00:19:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291288+0000", 2024-09-18T00:19:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4367e-05 2024-09-18T00:19:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:19:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5698999999999999e-05 2024-09-18T00:19:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291333+0000", 2024-09-18T00:19:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8783e-05 2024-09-18T00:19:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291987+0000", 2024-09-18T00:19:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00065417800000000005 2024-09-18T00:19:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.292000+0000", 2024-09-18T00:19:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2652e-05 2024-09-18T00:19:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316946+0000", 2024-09-18T00:19:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024946696000000001 2024-09-18T00:19:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316956+0000", 2024-09-18T00:19:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2760000000000006e-06 2024-09-18T00:19:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319752+0000", 2024-09-18T00:19:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002796083 2024-09-18T00:19:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319801+0000", 2024-09-18T00:19:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8885999999999999e-05 2024-09-18T00:19:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41948 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693756+0000", 2024-09-18T00:19:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.936725612, 2024-09-18T00:19:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053182506999999997, 2024-09-18T00:19:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41948 2024-09-18T00:19:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693756+0000", 2024-09-18T00:19:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693756+0000", 2024-09-18T00:19:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693757+0000", 2024-09-18T00:19:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.004e-06 2024-09-18T00:19:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693761+0000", 2024-09-18T00:19:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0670000000000002e-06 2024-09-18T00:19:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693789+0000", 2024-09-18T00:19:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7489999999999999e-05 2024-09-18T00:19:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693836+0000", 2024-09-18T00:19:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7021999999999998e-05 2024-09-18T00:19:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.746794+0000", 2024-09-18T00:19:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052958201000000003 2024-09-18T00:19:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.746858+0000", 2024-09-18T00:19:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4425999999999996e-05 2024-09-18T00:19:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.746939+0000", 2024-09-18T00:19:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0296999999999994e-05 2024-09-18T00:19:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41949 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693852+0000", 2024-09-18T00:19:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.936629991, 2024-09-18T00:19:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053245853000000003, 2024-09-18T00:19:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41949 2024-09-18T00:19:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693852+0000", 2024-09-18T00:19:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693852+0000", 2024-09-18T00:19:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693853+0000", 2024-09-18T00:19:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2999999999999999e-07 2024-09-18T00:19:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693866+0000", 2024-09-18T00:19:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3814e-05 2024-09-18T00:19:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693878+0000", 2024-09-18T00:19:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1384e-05 2024-09-18T00:19:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693893+0000", 2024-09-18T00:19:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4722000000000001e-05 2024-09-18T00:19:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.746958+0000", 2024-09-18T00:19:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053065734000000003 2024-09-18T00:19:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747015+0000", 2024-09-18T00:19:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6397000000000002e-05 2024-09-18T00:19:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747098+0000", 2024-09-18T00:19:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2872000000000002e-05 2024-09-18T00:19:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41950 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:19:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.936557088000001, 2024-09-18T00:19:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053368486, 2024-09-18T00:19:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41950 2024-09-18T00:19:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:19:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:19:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693926+0000", 2024-09-18T00:19:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4900000000000004e-07 2024-09-18T00:19:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693930+0000", 2024-09-18T00:19:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0130000000000004e-06 2024-09-18T00:19:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693940+0000", 2024-09-18T00:19:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0618000000000001e-05 2024-09-18T00:19:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693948+0000", 2024-09-18T00:19:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3140000000000004e-06 2024-09-18T00:19:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747133+0000", 2024-09-18T00:19:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053184689 2024-09-18T00:19:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747203+0000", 2024-09-18T00:19:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0339000000000005e-05 2024-09-18T00:19:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747293+0000", 2024-09-18T00:19:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9563999999999998e-05 2024-09-18T00:19:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41951 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:19:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.936522321, 2024-09-18T00:19:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053527695, 2024-09-18T00:19:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41951 2024-09-18T00:19:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:19:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:19:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693960+0000", 2024-09-18T00:19:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.16e-07 2024-09-18T00:19:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693971+0000", 2024-09-18T00:19:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1204e-05 2024-09-18T00:19:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693981+0000", 2024-09-18T00:19:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-18T00:19:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693989+0000", 2024-09-18T00:19:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0180000000000001e-06 2024-09-18T00:19:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747328+0000", 2024-09-18T00:19:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053338508 2024-09-18T00:19:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747400+0000", 2024-09-18T00:19:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2750999999999995e-05 2024-09-18T00:19:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747487+0000", 2024-09-18T00:19:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6527999999999996e-05 2024-09-18T00:19:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41952 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:19:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.936481914, 2024-09-18T00:19:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053728013999999998, 2024-09-18T00:19:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41952 2024-09-18T00:19:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:19:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:19:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694001+0000", 2024-09-18T00:19:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3799999999999996e-07 2024-09-18T00:19:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694004+0000", 2024-09-18T00:19:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.965e-06 2024-09-18T00:19:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694013+0000", 2024-09-18T00:19:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1689999999999998e-06 2024-09-18T00:19:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694022+0000", 2024-09-18T00:19:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7560000000000005e-06 2024-09-18T00:19:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747531+0000", 2024-09-18T00:19:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053508999000000002 2024-09-18T00:19:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747610+0000", 2024-09-18T00:19:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9212e-05 2024-09-18T00:19:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747728+0000", 2024-09-18T00:19:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011797499999999999 2024-09-18T00:19:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41953 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:19:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.936448285000001, 2024-09-18T00:19:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053936783000000002, 2024-09-18T00:19:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41953 2024-09-18T00:19:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:19:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:19:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694034+0000", 2024-09-18T00:19:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5199999999999996e-07 2024-09-18T00:19:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694037+0000", 2024-09-18T00:19:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.745e-06 2024-09-18T00:19:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694046+0000", 2024-09-18T00:19:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2490000000000007e-06 2024-09-18T00:19:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694054+0000", 2024-09-18T00:19:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1179999999999992e-06 2024-09-18T00:19:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747766+0000", 2024-09-18T00:19:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053711362999999998 2024-09-18T00:19:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747856+0000", 2024-09-18T00:19:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0779000000000006e-05 2024-09-18T00:19:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747970+0000", 2024-09-18T00:19:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011377700000000001 2024-09-18T00:19:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41954 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.936418132, 2024-09-18T00:19:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054104898999999998, 2024-09-18T00:19:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41954 2024-09-18T00:19:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4000000000000002e-07 2024-09-18T00:19:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694067+0000", 2024-09-18T00:19:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7329999999999999e-06 2024-09-18T00:19:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694076+0000", 2024-09-18T00:19:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2019999999999993e-06 2024-09-18T00:19:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694083+0000", 2024-09-18T00:19:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0029999999999997e-06 2024-09-18T00:19:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748003+0000", 2024-09-18T00:19:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053920463000000002 2024-09-18T00:19:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748077+0000", 2024-09-18T00:19:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4041000000000005e-05 2024-09-18T00:19:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748168+0000", 2024-09-18T00:19:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0916999999999997e-05 2024-09-18T00:19:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41955 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 231424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:19:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.936389294, 2024-09-18T00:19:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054272659000000001, 2024-09-18T00:19:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.704 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.704 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.704 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.704 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.704 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41955 2024-09-18T00:19:31.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.704 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:19:31.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:19:31.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694093+0000", 2024-09-18T00:19:31.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:19:31.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694109+0000", 2024-09-18T00:19:31.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6061000000000001e-05 2024-09-18T00:19:31.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694121+0000", 2024-09-18T00:19:31.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.216e-05 2024-09-18T00:19:31.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694129+0000", 2024-09-18T00:19:31.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0029999999999994e-06 2024-09-18T00:19:31.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748202+0000", 2024-09-18T00:19:31.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054072571 2024-09-18T00:19:31.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748274+0000", 2024-09-18T00:19:31.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2117000000000001e-05 2024-09-18T00:19:31.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748365+0000", 2024-09-18T00:19:31.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1139999999999998e-05 2024-09-18T00:19:31.707 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.707 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.707 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.708 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41956 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:31.708 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:31.708 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.936341086000001, 2024-09-18T00:19:31.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054420496999999998, 2024-09-18T00:19:31.708 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.708 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.708 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.708 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.708 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.708 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.708 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41956 2024-09-18T00:19:31.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.708 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:31.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:31.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:31.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8699999999999995e-07 2024-09-18T00:19:31.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694144+0000", 2024-09-18T00:19:31.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3510000000000002e-06 2024-09-18T00:19:31.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694154+0000", 2024-09-18T00:19:31.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0769999999999997e-06 2024-09-18T00:19:31.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694163+0000", 2024-09-18T00:19:31.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3300000000000005e-06 2024-09-18T00:19:31.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748397+0000", 2024-09-18T00:19:31.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054234497999999999 2024-09-18T00:19:31.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748471+0000", 2024-09-18T00:19:31.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3441000000000004e-05 2024-09-18T00:19:31.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748561+0000", 2024-09-18T00:19:31.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0212999999999998e-05 2024-09-18T00:19:31.713 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.713 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.713 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.713 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41957 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:31.713 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:19:31.713 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.936309833999999, 2024-09-18T00:19:31.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054590310000000003, 2024-09-18T00:19:31.713 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.713 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.713 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.714 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.714 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.714 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.714 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41957 2024-09-18T00:19:31.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.714 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:19:31.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:19:31.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694173+0000", 2024-09-18T00:19:31.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2500000000000005e-07 2024-09-18T00:19:31.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694175+0000", 2024-09-18T00:19:31.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.655e-06 2024-09-18T00:19:31.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694184+0000", 2024-09-18T00:19:31.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7630000000000006e-06 2024-09-18T00:19:31.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694191+0000", 2024-09-18T00:19:31.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8870000000000003e-06 2024-09-18T00:19:31.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748593+0000", 2024-09-18T00:19:31.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054402525 2024-09-18T00:19:31.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748669+0000", 2024-09-18T00:19:31.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5532999999999997e-05 2024-09-18T00:19:31.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748762+0000", 2024-09-18T00:19:31.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3221999999999996e-05 2024-09-18T00:19:31.718 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.718 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.718 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.718 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42391 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 337920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:31.719 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707180+0000", 2024-09-18T00:19:31.719 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.923302081999999, 2024-09-18T00:19:31.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053238496000000003, 2024-09-18T00:19:31.719 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.719 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.719 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.719 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.719 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.719 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.719 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42391 2024-09-18T00:19:31.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.719 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707180+0000", 2024-09-18T00:19:31.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707180+0000", 2024-09-18T00:19:31.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707180+0000", 2024-09-18T00:19:31.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6400000000000003e-07 2024-09-18T00:19:31.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707182+0000", 2024-09-18T00:19:31.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0870000000000001e-06 2024-09-18T00:19:31.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707189+0000", 2024-09-18T00:19:31.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7469999999999999e-06 2024-09-18T00:19:31.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707194+0000", 2024-09-18T00:19:31.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9459999999999997e-06 2024-09-18T00:19:31.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760268+0000", 2024-09-18T00:19:31.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053073899000000001 2024-09-18T00:19:31.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760351+0000", 2024-09-18T00:19:31.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3604999999999997e-05 2024-09-18T00:19:31.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760418+0000", 2024-09-18T00:19:31.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6748000000000005e-05 2024-09-18T00:19:31.723 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.723 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.723 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.723 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42392 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:31.723 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:19:31.723 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.923280395000001, 2024-09-18T00:19:31.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053378162, 2024-09-18T00:19:31.723 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.723 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.723 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.723 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.723 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.723 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.724 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42392 2024-09-18T00:19:31.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.724 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:19:31.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:19:31.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707430+0000", 2024-09-18T00:19:31.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022838599999999999 2024-09-18T00:19:31.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707434+0000", 2024-09-18T00:19:31.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5220000000000001e-06 2024-09-18T00:19:31.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707467+0000", 2024-09-18T00:19:31.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2570000000000002e-05 2024-09-18T00:19:31.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707475+0000", 2024-09-18T00:19:31.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:19:31.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760446+0000", 2024-09-18T00:19:31.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052971635000000003 2024-09-18T00:19:31.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760512+0000", 2024-09-18T00:19:31.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5679999999999995e-05 2024-09-18T00:19:31.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760579+0000", 2024-09-18T00:19:31.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7296000000000001e-05 2024-09-18T00:19:31.727 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.727 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.727 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.728 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42393 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 358400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:31.728 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:19:31.728 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.922991444000001, 2024-09-18T00:19:31.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053269618999999997, 2024-09-18T00:19:31.728 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.728 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.728 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.728 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.728 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.728 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.728 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42393 2024-09-18T00:19:31.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.729 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:19:31.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:19:31.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707491+0000", 2024-09-18T00:19:31.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1399999999999997e-07 2024-09-18T00:19:31.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707503+0000", 2024-09-18T00:19:31.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1912e-05 2024-09-18T00:19:31.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707512+0000", 2024-09-18T00:19:31.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1030000000000008e-06 2024-09-18T00:19:31.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707520+0000", 2024-09-18T00:19:31.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1249999999999993e-06 2024-09-18T00:19:31.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760605+0000", 2024-09-18T00:19:31.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053085495000000003 2024-09-18T00:19:31.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760680+0000", 2024-09-18T00:19:31.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5024000000000005e-05 2024-09-18T00:19:31.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760760+0000", 2024-09-18T00:19:31.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9345999999999996e-05 2024-09-18T00:19:31.732 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.732 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.733 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.733 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42394 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:31.733 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:19:31.733 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.922951456, 2024-09-18T00:19:31.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053403144999999999, 2024-09-18T00:19:31.733 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:31.733 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:31.733 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:31.733 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:31.734 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:31.734 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:31.734 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42394 2024-09-18T00:19:31.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.734 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:31.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:31.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:19:31.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:31.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:19:31.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:31.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:31.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711550+0000", 2024-09-18T00:19:31.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040200189999999997 2024-09-18T00:19:31.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:31.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711556+0000", 2024-09-18T00:19:31.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2339999999999999e-06 2024-09-18T00:19:31.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:31.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711569+0000", 2024-09-18T00:19:31.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2337999999999999e-05 2024-09-18T00:19:31.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:31.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711577+0000", 2024-09-18T00:19:31.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5660000000000007e-06 2024-09-18T00:19:31.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:31.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760790+0000", 2024-09-18T00:19:31.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049212196999999999 2024-09-18T00:19:31.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:31.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760857+0000", 2024-09-18T00:19:31.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7756000000000007e-05 2024-09-18T00:19:31.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:31.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:31.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:31.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760933+0000", 2024-09-18T00:19:31.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6035e-05 2024-09-18T00:19:31.738 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.738 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.738 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.738 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:31.738 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:31.738 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:31.739 INFO:teuthology.orchestra.run.smithi097.stdout:5484 trunc from 0xbf328f9 to 0x70c0ea8 2024-09-18T00:19:31.739 INFO:teuthology.orchestra.run.smithi097.stdout:5485 writesame 0x4cac211 thru 0x4cae502 data_size 0x12(0x22f2 bytes) 2024-09-18T00:19:31.740 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:31.733+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:31.740 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:19:31.741 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:31.741 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:31.751 INFO:teuthology.orchestra.run.smithi097.stdout:5489 trunc from 0x70c0ea8 to 0xc539285 2024-09-18T00:19:31.758 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:31.758 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:31.764 INFO:teuthology.orchestra.run.smithi097.stdout:5490 write 0x35af583 thru 0x35bf1ba (0xfc38 bytes) 2024-09-18T00:19:31.777 INFO:teuthology.orchestra.run.smithi097.stdout:5492 write 0x750b96c thru 0x750f3aa (0x3a3f bytes) 2024-09-18T00:19:31.782 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:31.782 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:31.784 INFO:teuthology.orchestra.run.smithi097.stdout:5494 punch from 0x5ebc60e to 0x5ec5b90, (0x9582 bytes) 2024-09-18T00:19:31.784 INFO:teuthology.orchestra.run.smithi097.stdout:5496 writesame 0x109be99 thru 0x10a3ac6 data_size 0x55(0x7c2e bytes) 2024-09-18T00:19:31.792 INFO:teuthology.orchestra.run.smithi097.stdout:5498 read 0xc27c3de thru 0xc28952b (0xd14e bytes) 2024-09-18T00:19:31.792 INFO:teuthology.orchestra.run.smithi097.stdout:5499 read 0x77d8b79 thru 0x77dd741 (0x4bc9 bytes) 2024-09-18T00:19:31.792 INFO:teuthology.orchestra.run.smithi097.stdout:5501 read 0x1571674 thru 0x1581077 (0xfa04 bytes) 2024-09-18T00:19:31.802 INFO:teuthology.orchestra.run.smithi097.stdout:5503 trunc from 0xc539285 to 0x6b1a752 2024-09-18T00:19:31.806 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:31.806 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:31.818 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:31.818 INFO:teuthology.orchestra.run.smithi097.stdout:5506 write 0x3392111 thru 0x339f117 (0xd007 bytes) 2024-09-18T00:19:31.828 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:31.828 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:31.839 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:31.833+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:31.844 INFO:teuthology.orchestra.run.smithi097.stdout:5507 write 0xb27de10 thru 0xb28d702 (0xf8f3 bytes) 2024-09-18T00:19:31.851 INFO:teuthology.orchestra.run.smithi097.stdout:5509 write 0x8f0b1cc thru 0x8f10897 (0x56cc bytes) 2024-09-18T00:19:31.853 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:31.853 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:31.857 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:31.857 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:31.858 INFO:teuthology.orchestra.run.smithi097.stdout:5510 read 0x10e5143 thru 0x10ee585 (0x9443 bytes) 2024-09-18T00:19:31.859 INFO:teuthology.orchestra.run.smithi097.stdout:5513 writesame 0x31bc7b2 thru 0x31c9159 data_size 0x6c(0xc9a8 bytes) 2024-09-18T00:19:31.862 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:31.863 INFO:teuthology.orchestra.run.smithi097.stdout:5516 read 0x1740a2e thru 0x1743bbd (0x3190 bytes) 2024-09-18T00:19:31.863 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:31.863 INFO:teuthology.orchestra.run.smithi097.stdout:5519 write 0x3944fda thru 0x394d019 (0x8040 bytes) 2024-09-18T00:19:31.868 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:31.868 INFO:teuthology.orchestra.run.smithi097.stdout:5523 read 0x57ff88d thru 0x580311d (0x3891 bytes) 2024-09-18T00:19:31.869 INFO:teuthology.orchestra.run.smithi097.stdout:5524 write 0x6a17252 thru 0x6a18822 (0x15d1 bytes) 2024-09-18T00:19:31.877 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:31.877 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:31.877 INFO:teuthology.orchestra.run.smithi097.stdout:5525 trunc from 0xb28d703 to 0x730dc58 2024-09-18T00:19:31.892 INFO:teuthology.orchestra.run.smithi097.stdout:5526 punch from 0x701c64c to 0x7021834, (0x51e8 bytes) 2024-09-18T00:19:31.893 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:31.893 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:31.893 INFO:teuthology.orchestra.run.smithi097.stdout:5531 trunc from 0x730dc58 to 0x6870c71 2024-09-18T00:19:31.894 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:19:31.921 INFO:teuthology.orchestra.run.smithi097.stdout:5535 trunc from 0x6870c71 to 0x1746c 2024-09-18T00:19:31.940 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:31.937+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:31.950 INFO:teuthology.orchestra.run.smithi097.stdout:5536 trunc from 0x1746c to 0x20dd8f8 2024-09-18T00:19:31.962 INFO:teuthology.orchestra.run.smithi097.stdout:5537 read 0x755973 thru 0x755dc9 (0x457 bytes) 2024-09-18T00:19:31.962 INFO:teuthology.orchestra.run.smithi097.stdout:5538 read 0xb6c3a2 thru 0xb724a9 (0x6108 bytes) 2024-09-18T00:19:31.995 INFO:teuthology.orchestra.run.smithi097.stdout:5539 write 0xa9721b3 thru 0xa977e2e (0x5c7c bytes) 2024-09-18T00:19:32.024 INFO:teuthology.orchestra.run.smithi097.stdout:5540 write 0xe62f60a thru 0xe63ba95 (0xc48c bytes) 2024-09-18T00:19:32.036 INFO:teuthology.orchestra.run.smithi097.stdout:5541 writesame 0x92a241d thru 0x92b1a79 data_size 0x1(0xf65d bytes) 2024-09-18T00:19:32.041 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:32.037+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:32.051 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:32.051 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:32.065 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:32.142 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:32.141+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:32.225 INFO:teuthology.orchestra.run.smithi097.stdout:5545 punch from 0x902764e to 0x902de7f, (0x6831 bytes) 2024-09-18T00:19:32.225 INFO:teuthology.orchestra.run.smithi097.stdout:5546 read 0x1e62b2f thru 0x1e6eb63 (0xc035 bytes) 2024-09-18T00:19:32.225 INFO:teuthology.orchestra.run.smithi097.stdout:5548 write 0x132836d thru 0x1333f06 (0xbb9a bytes) 2024-09-18T00:19:32.236 INFO:teuthology.orchestra.run.smithi097.stdout:5549 write 0x2988b6c thru 0x298dc74 (0x5109 bytes) 2024-09-18T00:19:32.239 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:32.239 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:32.242 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:32.241+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:32.243 INFO:teuthology.orchestra.run.smithi097.stdout:5551 read 0x4264208 thru 0x4265e19 (0x1c12 bytes) 2024-09-18T00:19:32.243 INFO:teuthology.orchestra.run.smithi097.stdout:5552 read 0x40ebfe5 thru 0x40f2a13 (0x6a2f bytes) 2024-09-18T00:19:32.243 INFO:teuthology.orchestra.run.smithi097.stdout:5554 write 0x8a07e85 thru 0x8a1203f (0xa1bb bytes) 2024-09-18T00:19:32.256 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:19:32.256 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:32.256 INFO:teuthology.orchestra.run.smithi097.stdout:5556 write 0x8116255 thru 0x8117d4b (0x1af7 bytes) 2024-09-18T00:19:32.259 INFO:teuthology.orchestra.run.smithi097.stdout:5557 write 0x4f02091 thru 0x4f0d76b (0xb6db bytes) 2024-09-18T00:19:32.269 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:32.269 INFO:teuthology.orchestra.run.smithi097.stdout:5559 read 0x15ab8a7 thru 0x15ac931 (0x108b bytes) 2024-09-18T00:19:32.270 INFO:teuthology.orchestra.run.smithi097.stdout:5560 punch from 0x85f93bf to 0x85fc973, (0x35b4 bytes) 2024-09-18T00:19:32.270 INFO:teuthology.orchestra.run.smithi097.stdout:5561 write 0x95268e1 thru 0x9528a99 (0x21b9 bytes) 2024-09-18T00:19:32.273 INFO:teuthology.orchestra.run.smithi097.stdout:5564 writesame 0x770e5e3 thru 0x7712c14 data_size 0xf(0x4632 bytes) 2024-09-18T00:19:32.292 INFO:teuthology.orchestra.run.smithi097.stdout:5565 writesame 0xed6b683 thru 0xed725e6 data_size 0x2(0x6f64 bytes) 2024-09-18T00:19:32.295 INFO:teuthology.orchestra.run.smithi097.stdout:5566 write 0x7786973 thru 0x778a72a (0x3db8 bytes) 2024-09-18T00:19:32.299 INFO:teuthology.orchestra.run.smithi097.stdout:5568 writesame 0xe2178df thru 0xe21f529 data_size 0x1(0x7c4b bytes) 2024-09-18T00:19:32.343 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:32.341+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:32.383 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:32.383 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:32.397 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:19:32.417 INFO:teuthology.orchestra.run.smithi097.stdout:5569 writesame 0x414c60b thru 0x4159420 data_size 0x1b(0xce16 bytes) 2024-09-18T00:19:32.425 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:32.425 INFO:teuthology.orchestra.run.smithi097.stdout:5574 writesame 0x4094142 thru 0x40a00e6 data_size 0x1(0xbfa5 bytes) 2024-09-18T00:19:32.444 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:32.437+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:32.541 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:32.541 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:32.545 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:32.541+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:32.555 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:19:32.586 INFO:teuthology.orchestra.run.smithi097.stdout:5575 read 0xfe309e thru 0xff118e (0xe0f1 bytes) 2024-09-18T00:19:32.588 INFO:teuthology.orchestra.run.smithi097.stdout:5576 write 0xb4ace70 thru 0xb4bbb71 (0xed02 bytes) 2024-09-18T00:19:32.598 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:32.606 INFO:teuthology.orchestra.run.smithi097.stdout:5579 trunc from 0xed725e7 to 0xa0fde5a 2024-09-18T00:19:32.626 INFO:teuthology.orchestra.run.smithi097.stdout:5580 punch from 0x5ae2b1e to 0x5ae804a, (0x552c bytes) 2024-09-18T00:19:32.626 INFO:teuthology.orchestra.run.smithi097.stdout:5581 write 0x6caaf46 thru 0x6cb3289 (0x8344 bytes) 2024-09-18T00:19:32.635 INFO:teuthology.orchestra.run.smithi097.stdout:5582 read 0x23af7b9 thru 0x23b07be (0x1006 bytes) 2024-09-18T00:19:32.635 INFO:teuthology.orchestra.run.smithi097.stdout:5583 punch from 0x3975dd to 0x397f2a, (0x94d bytes) 2024-09-18T00:19:32.646 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:32.645+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:32.655 INFO:teuthology.orchestra.run.smithi097.stdout:5584 trunc from 0xa0fde5a to 0x1ebb3b3 2024-09-18T00:19:32.683 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:32.695 INFO:teuthology.orchestra.run.smithi097.stdout:5587 write 0x20fa50f thru 0x20fb256 (0xd48 bytes) 2024-09-18T00:19:32.720 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:32.720 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:32.737 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:19:32.739 INFO:teuthology.orchestra.run.smithi097.stdout:5588 write 0xbe2ad87 thru 0xbe2be95 (0x110f bytes) 2024-09-18T00:19:32.746 INFO:teuthology.orchestra.run.smithi097.stdout:5589 read 0x1354e61 thru 0x135ddcd (0x8f6d bytes) 2024-09-18T00:19:32.746 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:32.741+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:32.747 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:32.760 INFO:teuthology.orchestra.run.smithi097.stdout:5591 write 0xda037e1 thru 0xda0f9a6 (0xc1c6 bytes) 2024-09-18T00:19:32.765 INFO:teuthology.orchestra.run.smithi097.stdout:5592 write 0x12e8671 thru 0x12f0dac (0x873c bytes) 2024-09-18T00:19:32.771 INFO:teuthology.orchestra.run.smithi097.stdout:5594 write 0xc7642b9 thru 0xc76aa0e (0x6756 bytes) 2024-09-18T00:19:32.778 INFO:teuthology.orchestra.run.smithi097.stdout:5595 write 0x37481b9 thru 0x374b52a (0x3372 bytes) 2024-09-18T00:19:32.782 INFO:teuthology.orchestra.run.smithi097.stdout:5597 punch from 0x1b3b9b5 to 0x1b3e762, (0x2dad bytes) 2024-09-18T00:19:32.783 INFO:teuthology.orchestra.run.smithi097.stdout:5600 flatten 2024-09-18T00:19:32.796 INFO:teuthology.orchestra.run.smithi097.stdout:5602 writesame 0xabde23e thru 0xabe1770 data_size 0x1(0x3533 bytes) 2024-09-18T00:19:32.809 INFO:teuthology.orchestra.run.smithi097.stdout:5603 punch from 0xb8f50ae to 0xb8fad20, (0x5c72 bytes) 2024-09-18T00:19:32.809 INFO:teuthology.orchestra.run.smithi097.stdout:5604 writesame 0x8c259e2 thru 0x8c315cd data_size 0x72(0xbbec bytes) 2024-09-18T00:19:32.817 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:32.835 INFO:teuthology.orchestra.run.smithi097.stdout:5607 writesame 0xeb4da85 thru 0xeb5624d data_size 0x1(0x87c9 bytes) 2024-09-18T00:19:32.835 INFO:teuthology.orchestra.run.smithi097.stdout:5608 writesame 0x12c0392 thru 0x12c5b32 data_size 0x1(0x57a1 bytes) 2024-09-18T00:19:32.847 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:32.841+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:32.871 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:32.871 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:32.881 INFO:teuthology.orchestra.run.smithi097.stdout:5611 read 0xe9d711d thru 0xe9da3fa (0x32de bytes) 2024-09-18T00:19:32.881 INFO:teuthology.orchestra.run.smithi097.stdout:5614 read 0x5ea1a9c thru 0x5eb1863 (0xfdc8 bytes) 2024-09-18T00:19:32.881 INFO:teuthology.orchestra.run.smithi097.stdout:5616 write 0x41583e9 thru 0x415ea24 (0x663c bytes) 2024-09-18T00:19:32.888 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:19:32.911 INFO:teuthology.orchestra.run.smithi097.stdout:5617 trunc from 0xeb5624e to 0x1e2065e 2024-09-18T00:19:32.949 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:32.945+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:32.958 INFO:teuthology.orchestra.run.smithi097.stdout:5618 write 0x5a05f7f thru 0x5a08a72 (0x2af4 bytes) 2024-09-18T00:19:32.972 INFO:teuthology.orchestra.run.smithi097.stdout:5619 trunc from 0x5a08a73 to 0x267ce 2024-09-18T00:19:33.023 INFO:teuthology.orchestra.run.smithi097.stdout:5620 write 0x80e82f2 thru 0x80f43e5 (0xc0f4 bytes) 2024-09-18T00:19:33.031 INFO:teuthology.orchestra.run.smithi097.stdout:5622 punch from 0x7dc02e0 to 0x7dc6b8d, (0x68ad bytes) 2024-09-18T00:19:33.031 INFO:teuthology.orchestra.run.smithi097.stdout:5623 clone 25 order 18 su 65536 sc 9 2024-09-18T00:19:33.038 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:33.038 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:33.050 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:33.049+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:33.052 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:19:33.151 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:33.149+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:33.173 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:33.174 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:33.188 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:19:33.252 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:33.249+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:33.322 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:33.322 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:33.336 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:19:33.353 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:33.353+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:33.453 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:33.449+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:33.485 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:19:33.507 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:33.507 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:33.522 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:19:33.566 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:33.565+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:33.655 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:33.653+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:33.692 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:33.692 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:33.706 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:19:33.745 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-18T00:19:33.747 INFO:teuthology.orchestra.run.smithi097.stdout:5625 punch from 0x7d4bfec to 0x7d592d7, (0xd2eb bytes) 2024-09-18T00:19:33.753 INFO:teuthology.orchestra.run.smithi097.stdout:5626 write 0x1b1fdd5 thru 0x1b2340d (0x3639 bytes) 2024-09-18T00:19:33.756 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:33.753+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:33.769 INFO:teuthology.orchestra.run.smithi097.stdout:5627 trunc from 0x80f43e6 to 0xd841173 2024-09-18T00:19:33.778 INFO:teuthology.orchestra.run.smithi097.stdout:5628 write 0xb8e6b56 thru 0xb8f4ef0 (0xe39b bytes) 2024-09-18T00:19:33.789 INFO:teuthology.orchestra.run.smithi097.stdout:5629 write 0x7616e18 thru 0x761f830 (0x8a19 bytes) 2024-09-18T00:19:33.798 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:33.798 INFO:teuthology.orchestra.run.smithi097.stdout:5631 writesame 0x683f215 thru 0x684e82f data_size 0x1(0xf61b bytes) 2024-09-18T00:19:33.849 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:33.849 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:33.857 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:33.857+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:33.858 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:19:33.876 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:19:33.958 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:33.957+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:33.967 INFO:teuthology.orchestra.run.smithi097.stdout:5633 writesame 0x88a93bf thru 0x88ac3e1 data_size 0x1(0x3023 bytes) 2024-09-18T00:19:33.999 INFO:teuthology.orchestra.run.smithi097.stdout:5634 write 0xddbfa3d thru 0xddcf1c0 (0xf784 bytes) 2024-09-18T00:19:34.003 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:34.003 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:34.004 INFO:teuthology.orchestra.run.smithi097.stdout:5637 punch from 0x95196d2 to 0x951d974, (0x42a2 bytes) 2024-09-18T00:19:34.004 INFO:teuthology.orchestra.run.smithi097.stdout:5638 write 0x4fbac71 thru 0x4fc0fd9 (0x6369 bytes) 2024-09-18T00:19:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:5639 read 0x9724f92 thru 0x9726c7e (0x1ced bytes) 2024-09-18T00:19:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:5640 write 0x92088b6 thru 0x920ef48 (0x6693 bytes) 2024-09-18T00:19:34.009 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:34.010 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:34.010 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:34.013 INFO:teuthology.orchestra.run.smithi097.stdout:5641 write 0xd6ca63c thru 0xd6d14d5 (0x6e9a bytes) 2024-09-18T00:19:34.018 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:19:34.019 INFO:teuthology.orchestra.run.smithi097.stdout:5642 read 0x5999fdf thru 0x599a041 (0x63 bytes) 2024-09-18T00:19:34.019 INFO:teuthology.orchestra.run.smithi097.stdout:5644 write 0x505fca8 thru 0x506f0a8 (0xf401 bytes) 2024-09-18T00:19:34.028 INFO:teuthology.orchestra.run.smithi097.stdout:5645 write 0x9e9d4a8 thru 0x9ea09f5 (0x354e bytes) 2024-09-18T00:19:34.029 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:34.045 INFO:teuthology.orchestra.run.smithi097.stdout:5648 writesame 0xe51d2c6 thru 0xe522285 data_size 0x58(0x4fc0 bytes) 2024-09-18T00:19:34.045 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:34.045 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:34.045 INFO:teuthology.orchestra.run.smithi097.stdout:5651 write 0x3b37f30 thru 0x3b3c3b8 (0x4489 bytes) 2024-09-18T00:19:34.048 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:34.048 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:34.049 INFO:teuthology.orchestra.run.smithi097.stdout:5655 write 0x5b00537 thru 0x5b04ba5 (0x466f bytes) 2024-09-18T00:19:34.053 INFO:teuthology.orchestra.run.smithi097.stdout:5656 writesame 0x43f4ff8 thru 0x43fbf91 data_size 0x5(0x6f9a bytes) 2024-09-18T00:19:34.060 INFO:teuthology.orchestra.run.smithi097.stdout:5657 writesame 0x1d07207 thru 0x1d0d53e data_size 0x64(0x6338 bytes) 2024-09-18T00:19:34.062 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:34.061+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:34.064 INFO:teuthology.orchestra.run.smithi097.stdout:5658 writesame 0x91cb58b thru 0x91cbdb3 data_size 0x1(0x829 bytes) 2024-09-18T00:19:34.069 INFO:teuthology.orchestra.run.smithi097.stdout:5659 punch from 0x3f11944 to 0x3f1d2e3, (0xb99f bytes) 2024-09-18T00:19:34.070 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:34.070 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:34.084 INFO:teuthology.orchestra.run.smithi097.stdout:5660 trunc from 0xe522286 to 0x3b49ffb 2024-09-18T00:19:34.090 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:34.090 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:34.114 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:34.114 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:34.114 INFO:teuthology.orchestra.run.smithi097.stdout:5661 read 0x209b04c thru 0x20a28df (0x7894 bytes) 2024-09-18T00:19:34.121 INFO:teuthology.orchestra.run.smithi097.stdout:5662 trunc from 0x3b49ffb to 0x8a5ba40 2024-09-18T00:19:34.128 INFO:teuthology.orchestra.run.smithi097.stdout:5663 read 0x87f3942 thru 0x8803445 (0xfb04 bytes) 2024-09-18T00:19:34.129 INFO:teuthology.orchestra.run.smithi097.stdout:5664 read 0x2e43e58 thru 0x2e4b15c (0x7305 bytes) 2024-09-18T00:19:34.134 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:34.134 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:34.138 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:34.138 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:34.146 INFO:teuthology.orchestra.run.smithi097.stdout:5666 writesame 0xe879d29 thru 0xe888a04 data_size 0xa3(0xecdc bytes) 2024-09-18T00:19:34.146 INFO:teuthology.orchestra.run.smithi097.stdout:5667 read 0xc4faee9 thru 0xc5017f4 (0x690c bytes) 2024-09-18T00:19:34.146 INFO:teuthology.orchestra.run.smithi097.stdout:5668 read 0x58ec8a0 thru 0x58f613a (0x989b bytes) 2024-09-18T00:19:34.155 INFO:teuthology.orchestra.run.smithi097.stdout:5669 trunc from 0xe888a05 to 0x8ea22fc 2024-09-18T00:19:34.160 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:34.157+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:34.162 INFO:teuthology.orchestra.run.smithi097.stdout:5670 punch from 0x3fe88c6 to 0x3feaf8f, (0x26c9 bytes) 2024-09-18T00:19:34.162 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:34.162 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:34.162 INFO:teuthology.orchestra.run.smithi097.stdout:5673 writesame 0x7a7d2ca thru 0x7a84f81 data_size 0x34(0x7cb8 bytes) 2024-09-18T00:19:34.167 INFO:teuthology.orchestra.run.smithi097.stdout:5674 write 0x7eaf212 thru 0x7ebbcd2 (0xcac1 bytes) 2024-09-18T00:19:34.175 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:34.175 INFO:teuthology.orchestra.run.smithi097.stdout:5677 read 0x443dfca thru 0x4442ab8 (0x4aef bytes) 2024-09-18T00:19:34.175 INFO:teuthology.orchestra.run.smithi097.stdout:5678 punch from 0x1241db to 0x128629, (0x444e bytes) 2024-09-18T00:19:34.182 INFO:teuthology.orchestra.run.smithi097.stdout:5679 trunc from 0x8ea22fc to 0x3f16131 2024-09-18T00:19:34.198 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:34.198 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:34.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:34.198 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:34.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.198 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2229 3.10 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:34.198 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:34.198 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.932796525000001, 2024-09-18T00:19:34.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048730086999999998, 2024-09-18T00:19:34.198 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.198 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.199 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:34.199 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.199 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.199 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.199 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2229 2024-09-18T00:19:34.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.199 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:34.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:34.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263362+0000", 2024-09-18T00:19:34.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3869999999999999e-06 2024-09-18T00:19:34.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263381+0000", 2024-09-18T00:19:34.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9153000000000001e-05 2024-09-18T00:19:34.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263418+0000", 2024-09-18T00:19:34.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6346000000000001e-05 2024-09-18T00:19:34.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263435+0000", 2024-09-18T00:19:34.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7309999999999999e-05 2024-09-18T00:19:34.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263589+0000", 2024-09-18T00:19:34.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015451199999999999 2024-09-18T00:19:34.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263902+0000", 2024-09-18T00:19:34.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031291799999999997 2024-09-18T00:19:34.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:34.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.265995+0000", 2024-09-18T00:19:34.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020931809999999999 2024-09-18T00:19:34.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:34.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311788+0000", 2024-09-18T00:19:34.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045792078 2024-09-18T00:19:34.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:34.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311903+0000", 2024-09-18T00:19:34.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011510699999999999 2024-09-18T00:19:34.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:34.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312009+0000", 2024-09-18T00:19:34.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010630599999999999 2024-09-18T00:19:34.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312089+0000", 2024-09-18T00:19:34.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9789000000000005e-05 2024-09-18T00:19:34.204 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.204 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.204 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.204 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2229 3.10 e36/23 ondisk, result = 0)", 2024-09-18T00:19:34.204 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:34.204 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.913650865999998, 2024-09-18T00:19:34.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029600955000000002, 2024-09-18T00:19:34.204 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.204 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.205 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.205 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:34.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:34.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282507+0000", 2024-09-18T00:19:34.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9160000000000001e-06 2024-09-18T00:19:34.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282522+0000", 2024-09-18T00:19:34.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5e-05 2024-09-18T00:19:34.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282558+0000", 2024-09-18T00:19:34.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5694000000000001e-05 2024-09-18T00:19:34.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282580+0000", 2024-09-18T00:19:34.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2323e-05 2024-09-18T00:19:34.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311873+0000", 2024-09-18T00:19:34.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029293013999999999 2024-09-18T00:19:34.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311892+0000", 2024-09-18T00:19:34.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8556e-05 2024-09-18T00:19:34.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312105+0000", 2024-09-18T00:19:34.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021345199999999999 2024-09-18T00:19:34.208 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.208 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.208 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.208 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5714 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:34.208 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:34.208 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.885948362999997, 2024-09-18T00:19:34.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027448727999999999, 2024-09-18T00:19:34.208 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.208 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.208 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:34.208 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.208 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.209 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.209 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5714 2024-09-18T00:19:34.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.209 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:34.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:34.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310211+0000", 2024-09-18T00:19:34.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5639999999999998e-06 2024-09-18T00:19:34.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312678+0000", 2024-09-18T00:19:34.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002466761 2024-09-18T00:19:34.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312746+0000", 2024-09-18T00:19:34.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8046000000000005e-05 2024-09-18T00:19:34.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312774+0000", 2024-09-18T00:19:34.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7813e-05 2024-09-18T00:19:34.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312927+0000", 2024-09-18T00:19:34.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015328499999999999 2024-09-18T00:19:34.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313214+0000", 2024-09-18T00:19:34.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028683099999999998 2024-09-18T00:19:34.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:34.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313708+0000", 2024-09-18T00:19:34.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049379100000000004 2024-09-18T00:19:34.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:34.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336895+0000", 2024-09-18T00:19:34.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023186801 2024-09-18T00:19:34.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:34.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337454+0000", 2024-09-18T00:19:34.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055979399999999996 2024-09-18T00:19:34.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:34.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337571+0000", 2024-09-18T00:19:34.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116821 2024-09-18T00:19:34.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337656+0000", 2024-09-18T00:19:34.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4221000000000006e-05 2024-09-18T00:19:34.213 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.214 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.214 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.214 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6801 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:34.214 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:34.214 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.159111914999997, 2024-09-18T00:19:34.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051953602000000002, 2024-09-18T00:19:34.214 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.214 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.214 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:34.214 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.214 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.215 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.215 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6801 2024-09-18T00:19:34.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.215 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:34.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:34.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037046+0000", 2024-09-18T00:19:34.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.19e-06 2024-09-18T00:19:34.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037062+0000", 2024-09-18T00:19:34.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586999999999998e-05 2024-09-18T00:19:34.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037096+0000", 2024-09-18T00:19:34.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4112000000000001e-05 2024-09-18T00:19:34.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037120+0000", 2024-09-18T00:19:34.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.355e-05 2024-09-18T00:19:34.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037246+0000", 2024-09-18T00:19:34.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012652200000000001 2024-09-18T00:19:34.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037385+0000", 2024-09-18T00:19:34.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013924599999999999 2024-09-18T00:19:34.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:34.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.040881+0000", 2024-09-18T00:19:34.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034954830000000002 2024-09-18T00:19:34.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:34.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088794+0000", 2024-09-18T00:19:34.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047912897000000003 2024-09-18T00:19:34.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:34.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088828+0000", 2024-09-18T00:19:34.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3880000000000001e-05 2024-09-18T00:19:34.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:34.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088916+0000", 2024-09-18T00:19:34.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.865e-05 2024-09-18T00:19:34.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088997+0000", 2024-09-18T00:19:34.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0484999999999999e-05 2024-09-18T00:19:34.219 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.219 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.220 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.220 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:6801 3.10 e42/23 ondisk, result = 0)", 2024-09-18T00:19:34.220 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:34.220 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.145304504999999, 2024-09-18T00:19:34.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037960394000000001, 2024-09-18T00:19:34.220 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.220 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.220 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.220 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:34.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:34.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050852+0000", 2024-09-18T00:19:34.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:19:34.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050860+0000", 2024-09-18T00:19:34.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7729999999999999e-06 2024-09-18T00:19:34.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050891+0000", 2024-09-18T00:19:34.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0617000000000002e-05 2024-09-18T00:19:34.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050910+0000", 2024-09-18T00:19:34.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.944e-05 2024-09-18T00:19:34.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088766+0000", 2024-09-18T00:19:34.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037855633999999999 2024-09-18T00:19:34.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088784+0000", 2024-09-18T00:19:34.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7767000000000001e-05 2024-09-18T00:19:34.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088811+0000", 2024-09-18T00:19:34.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7382999999999999e-05 2024-09-18T00:19:34.223 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.223 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.223 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.223 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12594 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:34.223 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:34.224 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.804527966999999, 2024-09-18T00:19:34.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026492600000000002, 2024-09-18T00:19:34.224 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.224 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.224 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.224 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.224 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.224 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.224 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12594 2024-09-18T00:19:34.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.224 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:34.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:34.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391629+0000", 2024-09-18T00:19:34.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.86e-06 2024-09-18T00:19:34.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391637+0000", 2024-09-18T00:19:34.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3999999999999992e-06 2024-09-18T00:19:34.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391664+0000", 2024-09-18T00:19:34.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6536000000000001e-05 2024-09-18T00:19:34.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391681+0000", 2024-09-18T00:19:34.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6563e-05 2024-09-18T00:19:34.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417791+0000", 2024-09-18T00:19:34.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026110936000000001 2024-09-18T00:19:34.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417933+0000", 2024-09-18T00:19:34.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014112099999999999 2024-09-18T00:19:34.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418120+0000", 2024-09-18T00:19:34.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018718399999999999 2024-09-18T00:19:34.228 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.228 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.228 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.228 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12595 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:34.228 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:34.228 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.804451769000003, 2024-09-18T00:19:34.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026819428999999999, 2024-09-18T00:19:34.228 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.228 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.228 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.228 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.228 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.229 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.229 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12595 2024-09-18T00:19:34.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.229 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:34.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:34.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391705+0000", 2024-09-18T00:19:34.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3999999999999999e-06 2024-09-18T00:19:34.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391711+0000", 2024-09-18T00:19:34.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7459999999999997e-06 2024-09-18T00:19:34.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391733+0000", 2024-09-18T00:19:34.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2923000000000001e-05 2024-09-18T00:19:34.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391748+0000", 2024-09-18T00:19:34.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4790000000000001e-05 2024-09-18T00:19:34.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418180+0000", 2024-09-18T00:19:34.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026431888000000001 2024-09-18T00:19:34.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418328+0000", 2024-09-18T00:19:34.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000147725 2024-09-18T00:19:34.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418523+0000", 2024-09-18T00:19:34.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000194957 2024-09-18T00:19:34.232 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.232 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.232 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.232 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12596 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:34.232 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:34.233 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.804388801000002, 2024-09-18T00:19:34.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027220296000000001, 2024-09-18T00:19:34.233 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.233 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.233 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.233 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.233 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.233 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.233 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12596 2024-09-18T00:19:34.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.233 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:34.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:34.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391768+0000", 2024-09-18T00:19:34.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.283e-06 2024-09-18T00:19:34.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391773+0000", 2024-09-18T00:19:34.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3970000000000002e-06 2024-09-18T00:19:34.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391795+0000", 2024-09-18T00:19:34.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1835e-05 2024-09-18T00:19:34.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391809+0000", 2024-09-18T00:19:34.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4107000000000001e-05 2024-09-18T00:19:34.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418582+0000", 2024-09-18T00:19:34.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026772918999999999 2024-09-18T00:19:34.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418780+0000", 2024-09-18T00:19:34.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000198042 2024-09-18T00:19:34.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418987+0000", 2024-09-18T00:19:34.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000206713 2024-09-18T00:19:34.236 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.237 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.237 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.237 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12597 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:34.237 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:34.237 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.804328282999997, 2024-09-18T00:19:34.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027612714, 2024-09-18T00:19:34.237 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.237 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.237 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.237 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.237 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.237 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.238 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12597 2024-09-18T00:19:34.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.238 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:34.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:34.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391828+0000", 2024-09-18T00:19:34.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.203e-06 2024-09-18T00:19:34.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391834+0000", 2024-09-18T00:19:34.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6529999999999999e-06 2024-09-18T00:19:34.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391856+0000", 2024-09-18T00:19:34.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2195999999999999e-05 2024-09-18T00:19:34.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391870+0000", 2024-09-18T00:19:34.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4063000000000001e-05 2024-09-18T00:19:34.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419049+0000", 2024-09-18T00:19:34.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027178751000000001 2024-09-18T00:19:34.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419239+0000", 2024-09-18T00:19:34.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019039400000000001 2024-09-18T00:19:34.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419440+0000", 2024-09-18T00:19:34.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000200454 2024-09-18T00:19:34.241 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.241 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.241 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.241 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12598 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:34.241 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:34.241 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.804265231999999, 2024-09-18T00:19:34.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028269056000000001, 2024-09-18T00:19:34.241 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.241 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.241 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.242 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.242 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.242 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.242 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12598 2024-09-18T00:19:34.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.242 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:34.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:34.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391891+0000", 2024-09-18T00:19:34.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.257e-06 2024-09-18T00:19:34.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391897+0000", 2024-09-18T00:19:34.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9560000000000002e-06 2024-09-18T00:19:34.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391920+0000", 2024-09-18T00:19:34.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3025999999999999e-05 2024-09-18T00:19:34.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391935+0000", 2024-09-18T00:19:34.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4853e-05 2024-09-18T00:19:34.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419527+0000", 2024-09-18T00:19:34.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027592378000000001 2024-09-18T00:19:34.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419681+0000", 2024-09-18T00:19:34.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015316 2024-09-18T00:19:34.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420159+0000", 2024-09-18T00:19:34.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000478426 2024-09-18T00:19:34.245 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.245 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.245 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12599 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.804201376999998, 2024-09-18T00:19:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028649857000000001, 2024-09-18T00:19:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12599 2024-09-18T00:19:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:34.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:34.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391955+0000", 2024-09-18T00:19:34.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-06 2024-09-18T00:19:34.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391961+0000", 2024-09-18T00:19:34.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7100000000000004e-06 2024-09-18T00:19:34.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391982+0000", 2024-09-18T00:19:34.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1492e-05 2024-09-18T00:19:34.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391996+0000", 2024-09-18T00:19:34.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4194e-05 2024-09-18T00:19:34.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420237+0000", 2024-09-18T00:19:34.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028240818000000001 2024-09-18T00:19:34.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420430+0000", 2024-09-18T00:19:34.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019228299999999999 2024-09-18T00:19:34.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420604+0000", 2024-09-18T00:19:34.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000174117 2024-09-18T00:19:34.250 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.250 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.250 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.250 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12600 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 270336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:34.250 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:34.250 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.804140383000004, 2024-09-18T00:19:34.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028964586, 2024-09-18T00:19:34.250 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.250 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.251 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.251 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.251 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.251 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.251 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12600 2024-09-18T00:19:34.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.251 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:34.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:34.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392016+0000", 2024-09-18T00:19:34.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.31e-06 2024-09-18T00:19:34.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392022+0000", 2024-09-18T00:19:34.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5600000000000001e-06 2024-09-18T00:19:34.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392045+0000", 2024-09-18T00:19:34.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3595999999999999e-05 2024-09-18T00:19:34.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392060+0000", 2024-09-18T00:19:34.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4727e-05 2024-09-18T00:19:34.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420669+0000", 2024-09-18T00:19:34.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028608855999999998 2024-09-18T00:19:34.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420813+0000", 2024-09-18T00:19:34.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014429500000000001 2024-09-18T00:19:34.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420979+0000", 2024-09-18T00:19:34.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016624200000000001 2024-09-18T00:19:34.255 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.255 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12601 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.804076477999999, 2024-09-18T00:19:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029292832000000001, 2024-09-18T00:19:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.256 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12601 2024-09-18T00:19:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392080+0000", 2024-09-18T00:19:34.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:34.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392086+0000", 2024-09-18T00:19:34.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7570000000000001e-06 2024-09-18T00:19:34.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392105+0000", 2024-09-18T00:19:34.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9415999999999998e-05 2024-09-18T00:19:34.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392132+0000", 2024-09-18T00:19:34.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7019e-05 2024-09-18T00:19:34.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421049+0000", 2024-09-18T00:19:34.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028916659000000001 2024-09-18T00:19:34.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421204+0000", 2024-09-18T00:19:34.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155224 2024-09-18T00:19:34.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421372+0000", 2024-09-18T00:19:34.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016754699999999999 2024-09-18T00:19:34.260 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.260 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.260 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.260 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12602 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:34.260 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:34.260 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.804003846999997, 2024-09-18T00:19:34.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029612833000000002, 2024-09-18T00:19:34.261 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.261 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.261 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.261 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.261 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.261 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.261 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12602 2024-09-18T00:19:34.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.261 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:34.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:34.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392152+0000", 2024-09-18T00:19:34.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0470000000000001e-06 2024-09-18T00:19:34.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392158+0000", 2024-09-18T00:19:34.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:19:34.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392181+0000", 2024-09-18T00:19:34.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2629e-05 2024-09-18T00:19:34.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392222+0000", 2024-09-18T00:19:34.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0772000000000002e-05 2024-09-18T00:19:34.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421441+0000", 2024-09-18T00:19:34.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029219327 2024-09-18T00:19:34.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421608+0000", 2024-09-18T00:19:34.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000166813 2024-09-18T00:19:34.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421764+0000", 2024-09-18T00:19:34.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015628500000000001 2024-09-18T00:19:34.264 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.265 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.265 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.265 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12603 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:34.265 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:34.265 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.803913453, 2024-09-18T00:19:34.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029902524, 2024-09-18T00:19:34.265 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.265 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.265 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.265 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.265 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.265 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.266 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12603 2024-09-18T00:19:34.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.266 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:34.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:34.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392243+0000", 2024-09-18T00:19:34.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:34.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392260+0000", 2024-09-18T00:19:34.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6860000000000001e-05 2024-09-18T00:19:34.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392282+0000", 2024-09-18T00:19:34.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2713000000000001e-05 2024-09-18T00:19:34.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392297+0000", 2024-09-18T00:19:34.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4375e-05 2024-09-18T00:19:34.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421830+0000", 2024-09-18T00:19:34.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029532668000000002 2024-09-18T00:19:34.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421985+0000", 2024-09-18T00:19:34.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155613 2024-09-18T00:19:34.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422144+0000", 2024-09-18T00:19:34.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000159102 2024-09-18T00:19:34.269 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.269 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.269 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.269 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12604 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:34.269 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:34.269 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.803838401999997, 2024-09-18T00:19:34.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030228695, 2024-09-18T00:19:34.270 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.270 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.270 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.270 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.270 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.270 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.270 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12604 2024-09-18T00:19:34.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.270 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:34.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:34.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392318+0000", 2024-09-18T00:19:34.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:34.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392337+0000", 2024-09-18T00:19:34.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:19:34.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392364+0000", 2024-09-18T00:19:34.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7288999999999999e-05 2024-09-18T00:19:34.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392379+0000", 2024-09-18T00:19:34.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:19:34.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422209+0000", 2024-09-18T00:19:34.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029829050999999999 2024-09-18T00:19:34.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422388+0000", 2024-09-18T00:19:34.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017988699999999999 2024-09-18T00:19:34.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422545+0000", 2024-09-18T00:19:34.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015701199999999999 2024-09-18T00:19:34.273 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.273 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.274 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.274 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12605 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:34.274 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:34.274 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.803753841000002, 2024-09-18T00:19:34.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030488674, 2024-09-18T00:19:34.274 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.274 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.274 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.274 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.274 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.274 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.275 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12605 2024-09-18T00:19:34.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.275 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:34.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:34.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392403+0000", 2024-09-18T00:19:34.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.883e-06 2024-09-18T00:19:34.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392421+0000", 2024-09-18T00:19:34.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7526000000000001e-05 2024-09-18T00:19:34.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392450+0000", 2024-09-18T00:19:34.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8972e-05 2024-09-18T00:19:34.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392488+0000", 2024-09-18T00:19:34.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8522000000000001e-05 2024-09-18T00:19:34.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422614+0000", 2024-09-18T00:19:34.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030125945000000001 2024-09-18T00:19:34.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422752+0000", 2024-09-18T00:19:34.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000138178 2024-09-18T00:19:34.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422890+0000", 2024-09-18T00:19:34.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013764800000000001 2024-09-18T00:19:34.278 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.278 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.278 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.278 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12606 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:34.278 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:34.278 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.803645181, 2024-09-18T00:19:34.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033727073000000003, 2024-09-18T00:19:34.279 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.279 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.279 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.279 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.279 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.279 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.279 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12606 2024-09-18T00:19:34.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.279 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:34.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:34.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392511+0000", 2024-09-18T00:19:34.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3090000000000001e-06 2024-09-18T00:19:34.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392517+0000", 2024-09-18T00:19:34.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.993e-06 2024-09-18T00:19:34.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392543+0000", 2024-09-18T00:19:34.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6152999999999998e-05 2024-09-18T00:19:34.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392558+0000", 2024-09-18T00:19:34.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4610000000000001e-05 2024-09-18T00:19:34.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.425959+0000", 2024-09-18T00:19:34.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033400484000000001 2024-09-18T00:19:34.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426106+0000", 2024-09-18T00:19:34.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014779800000000001 2024-09-18T00:19:34.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426237+0000", 2024-09-18T00:19:34.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013072600000000001 2024-09-18T00:19:34.282 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.282 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.283 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.283 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17562 3.a 3:5578626c:::rbd_data.112a6efb7564.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T00:19:34.283 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:34.283 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.238180020000001, 2024-09-18T00:19:34.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052733453999999999, 2024-09-18T00:19:34.283 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.283 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.283 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:34.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.284 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17562 2024-09-18T00:19:34.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.284 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:34.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:34.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957977+0000", 2024-09-18T00:19:34.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.119e-06 2024-09-18T00:19:34.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957988+0000", 2024-09-18T00:19:34.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0557000000000001e-05 2024-09-18T00:19:34.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958017+0000", 2024-09-18T00:19:34.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9156e-05 2024-09-18T00:19:34.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958036+0000", 2024-09-18T00:19:34.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8919e-05 2024-09-18T00:19:34.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958144+0000", 2024-09-18T00:19:34.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107905 2024-09-18T00:19:34.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958284+0000", 2024-09-18T00:19:34.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140216 2024-09-18T00:19:34.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:34.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.960572+0000", 2024-09-18T00:19:34.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022881860000000002 2024-09-18T00:19:34.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:34.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010537+0000", 2024-09-18T00:19:34.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049964664999999998 2024-09-18T00:19:34.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:34.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010590+0000", 2024-09-18T00:19:34.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3094999999999999e-05 2024-09-18T00:19:34.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:34.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010642+0000", 2024-09-18T00:19:34.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2073000000000001e-05 2024-09-18T00:19:34.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010709+0000", 2024-09-18T00:19:34.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6563000000000006e-05 2024-09-18T00:19:34.288 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.289 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.289 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.289 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17562 3.a e60/23 ondisk, result = 0)", 2024-09-18T00:19:34.289 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:34.289 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.227976648999999, 2024-09-18T00:19:34.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042539307999999998, 2024-09-18T00:19:34.289 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.289 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.289 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.289 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:34.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:34.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968180+0000", 2024-09-18T00:19:34.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-18T00:19:34.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968188+0000", 2024-09-18T00:19:34.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7419999999999997e-06 2024-09-18T00:19:34.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968217+0000", 2024-09-18T00:19:34.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8926999999999999e-05 2024-09-18T00:19:34.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968231+0000", 2024-09-18T00:19:34.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4168999999999999e-05 2024-09-18T00:19:34.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010575+0000", 2024-09-18T00:19:34.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042343923999999998 2024-09-18T00:19:34.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010584+0000", 2024-09-18T00:19:34.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-18T00:19:34.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010718+0000", 2024-09-18T00:19:34.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013344499999999999 2024-09-18T00:19:34.293 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.293 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.293 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.293 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.293 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.293 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:34.295 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:19:34.296 INFO:teuthology.orchestra.run.smithi097.stdout:5680 write 0x9aa7239 thru 0x9ab3442 (0xc20a bytes) 2024-09-18T00:19:34.296 INFO:teuthology.orchestra.run.smithi097.stdout:5681 write 0x70f6b3 thru 0x713ed4 (0x4822 bytes) 2024-09-18T00:19:34.296 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:34.296 INFO:teuthology.orchestra.run.smithi097.stdout:5683 punch from 0x6eacb66 to 0x6eb18b9, (0x4d53 bytes) 2024-09-18T00:19:34.296 INFO:teuthology.orchestra.run.smithi097.stdout:5685 read 0x8fdb2f1 thru 0x8fe78c6 (0xc5d6 bytes) 2024-09-18T00:19:34.296 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:34.296 INFO:teuthology.orchestra.run.smithi097.stdout:5687 read 0x70a573e thru 0x70ac44a (0x6d0d bytes) 2024-09-18T00:19:34.296 INFO:teuthology.orchestra.run.smithi097.stdout:5688 read 0x201702f thru 0x201cff3 (0x5fc5 bytes) 2024-09-18T00:19:34.296 INFO:teuthology.orchestra.run.smithi097.stdout:5689 punch from 0x75ce69e to 0x75ced49, (0x6ab bytes) 2024-09-18T00:19:34.297 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:34.297 INFO:teuthology.orchestra.run.smithi097.stdout:5692 read 0x66e39c thru 0x6722ae (0x3f13 bytes) 2024-09-18T00:19:34.297 INFO:teuthology.orchestra.run.smithi097.stdout:5693 trunc from 0x9ab3443 to 0x1febe5c 2024-09-18T00:19:34.297 INFO:teuthology.orchestra.run.smithi097.stdout:5694 trunc from 0x1febe5c to 0xa0fec10 2024-09-18T00:19:34.297 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:34.293+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:34.313 INFO:teuthology.orchestra.run.smithi097.stdout:5696 write 0xb738b0b thru 0xb745bc5 (0xd0bb bytes) 2024-09-18T00:19:34.323 INFO:teuthology.orchestra.run.smithi097.stdout:5698 trunc from 0xb745bc6 to 0xc4370ab 2024-09-18T00:19:34.333 INFO:teuthology.orchestra.run.smithi097.stdout:5699 read 0x5a1e675 thru 0x5a2435a (0x5ce6 bytes) 2024-09-18T00:19:34.333 INFO:teuthology.orchestra.run.smithi097.stdout:5702 writesame 0x8c8d836 thru 0x8c8f45c data_size 0x1(0x1c27 bytes) 2024-09-18T00:19:34.346 INFO:teuthology.orchestra.run.smithi097.stdout:5703 write 0x6e41bf2 thru 0x6e516a6 (0xfab5 bytes) 2024-09-18T00:19:34.356 INFO:teuthology.orchestra.run.smithi097.stdout:5704 read 0x5decafb thru 0x5df2ffd (0x6503 bytes) 2024-09-18T00:19:34.368 INFO:teuthology.orchestra.run.smithi097.stdout:5705 trunc from 0xc4370ab to 0x80a8802 2024-09-18T00:19:34.394 INFO:teuthology.orchestra.run.smithi097.stdout:5706 read 0x1e0d4e4 thru 0x1e1be82 (0xe99f bytes) 2024-09-18T00:19:34.394 INFO:teuthology.orchestra.run.smithi097.stdout:5707 write 0x61b61c7 thru 0x61ba8bf (0x46f9 bytes) 2024-09-18T00:19:34.395 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:34.393+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:34.408 INFO:teuthology.orchestra.run.smithi097.stdout:5709 trunc from 0x80a8802 to 0x9aaf62e 2024-09-18T00:19:34.422 INFO:teuthology.orchestra.run.smithi097.stdout:5713 read 0x18480d0 thru 0x18508fe (0x882f bytes) 2024-09-18T00:19:34.431 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:34.431 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:34.431 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:34.431 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:34.446 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:19:34.447 INFO:teuthology.orchestra.run.smithi097.stdout:5714 writesame 0xd9cce62 thru 0xd9d811f data_size 0x2(0xb2be bytes) 2024-09-18T00:19:34.447 INFO:teuthology.orchestra.run.smithi097.stdout:5715 write 0x5e7597d thru 0x5e7ca50 (0x70d4 bytes) 2024-09-18T00:19:34.482 INFO:teuthology.orchestra.run.smithi097.stdout:5716 trunc from 0xd9d8120 to 0x3dcf73d 2024-09-18T00:19:34.492 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is unset 2024-09-18T00:19:34.496 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:34.493+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:34.516 INFO:teuthology.orchestra.run.smithi097.stdout:5717 write 0x3a9e713 thru 0x3aa229a (0x3b88 bytes) 2024-09-18T00:19:34.526 INFO:teuthology.orchestra.run.smithi097.stdout:5718 trunc from 0x3dcf73d to 0x2455151 2024-09-18T00:19:34.534 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 7] out_osds: [6, 1] dead_osds: [1] live_osds: [0, 2, 3, 4, 6, 7, 5] 2024-09-18T00:19:34.534 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:19:34.534 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T00:19:34.534 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T00:19:34.540 INFO:teuthology.orchestra.run.smithi097.stdout:5719 punch from 0x1050fb5 to 0x105ed72, (0xddbd bytes) 2024-09-18T00:19:34.549 INFO:teuthology.orchestra.run.smithi097.stdout:5721 write 0x2fc1385 thru 0x2fcea2b (0xd6a7 bytes) 2024-09-18T00:19:34.554 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:34.554 INFO:teuthology.orchestra.run.smithi097.stdout:5724 read 0xb812c1 thru 0xb8a98f (0x96cf bytes) 2024-09-18T00:19:34.564 INFO:teuthology.orchestra.run.smithi097.stdout:5725 trunc from 0x2fcea2c to 0x80d21d2 2024-09-18T00:19:34.572 INFO:teuthology.orchestra.run.smithi097.stdout:5726 trunc from 0x80d21d2 to 0x79c132f 2024-09-18T00:19:34.596 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:34.597+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:34.598 INFO:teuthology.orchestra.run.smithi097.stdout:5727 writesame 0xd8984a1 thru 0xd8a7b75 data_size 0x77(0xf6d5 bytes) 2024-09-18T00:19:34.598 INFO:teuthology.orchestra.run.smithi097.stdout:5728 clone 26 order 25 su 16777216 sc 9 2024-09-18T00:19:34.613 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:34.614 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:34.614 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:34.614 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:34.614 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:34.628 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:19:34.698 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:34.697+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:34.756 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2228 3.15 3:aa2fc059:::rbd_data.112aa7ba4578.000000000000014a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 74.489190105999995, 2024-09-18T00:19:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.068966377999999995, 2024-09-18T00:19:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2228 2024-09-18T00:19:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263099+0000", 2024-09-18T00:19:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2359999999999999e-06 2024-09-18T00:19:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263115+0000", 2024-09-18T00:19:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5996999999999999e-05 2024-09-18T00:19:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263156+0000", 2024-09-18T00:19:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:19:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263183+0000", 2024-09-18T00:19:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7233000000000002e-05 2024-09-18T00:19:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263304+0000", 2024-09-18T00:19:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121209 2024-09-18T00:19:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263528+0000", 2024-09-18T00:19:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022401800000000001 2024-09-18T00:19:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:19:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.266458+0000", 2024-09-18T00:19:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029296980000000001 2024-09-18T00:19:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331628+0000", 2024-09-18T00:19:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.065170045999999995 2024-09-18T00:19:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331819+0000", 2024-09-18T00:19:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000190819 2024-09-18T00:19:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331971+0000", 2024-09-18T00:19:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151972 2024-09-18T00:19:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332063+0000", 2024-09-18T00:19:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2083999999999995e-05 2024-09-18T00:19:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2854 3.f 3:f02b0bca:::rbd_data.112aa7ba4578.0000000000000135:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:19:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 72.882217616000005, 2024-09-18T00:19:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.082582802999999996, 2024-09-18T00:19:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2854 2024-09-18T00:19:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870071+0000", 2024-09-18T00:19:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.227e-06 2024-09-18T00:19:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870083+0000", 2024-09-18T00:19:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1909999999999999e-05 2024-09-18T00:19:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870114+0000", 2024-09-18T00:19:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0576e-05 2024-09-18T00:19:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870129+0000", 2024-09-18T00:19:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.485e-05 2024-09-18T00:19:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870256+0000", 2024-09-18T00:19:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012691300000000001 2024-09-18T00:19:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870493+0000", 2024-09-18T00:19:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023737399999999999 2024-09-18T00:19:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.874933+0000", 2024-09-18T00:19:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044398299999999996 2024-09-18T00:19:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952452+0000", 2024-09-18T00:19:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.077518918000000006 2024-09-18T00:19:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952486+0000", 2024-09-18T00:19:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4035999999999997e-05 2024-09-18T00:19:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952575+0000", 2024-09-18T00:19:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.954e-05 2024-09-18T00:19:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952652+0000", 2024-09-18T00:19:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6629000000000001e-05 2024-09-18T00:19:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2854 3.f e38/32 ondisk, result = 0)", 2024-09-18T00:19:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 72.864686801000005, 2024-09-18T00:19:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064868625999999999, 2024-09-18T00:19:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887602+0000", 2024-09-18T00:19:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.677e-06 2024-09-18T00:19:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887609+0000", 2024-09-18T00:19:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6729999999999992e-06 2024-09-18T00:19:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887637+0000", 2024-09-18T00:19:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8206000000000001e-05 2024-09-18T00:19:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887651+0000", 2024-09-18T00:19:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3387000000000001e-05 2024-09-18T00:19:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952422+0000", 2024-09-18T00:19:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064771366999999996 2024-09-18T00:19:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952441+0000", 2024-09-18T00:19:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8737e-05 2024-09-18T00:19:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952469+0000", 2024-09-18T00:19:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7579000000000001e-05 2024-09-18T00:19:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15879 3.16 3:692b47b6:::rbd_object_map.112a13ec2d6c.000000000000000b:head [call] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:19:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.097522894000001, 2024-09-18T00:19:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.011184018, 2024-09-18T00:19:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15879 2024-09-18T00:19:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654768+0000", 2024-09-18T00:19:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9129999999999996e-06 2024-09-18T00:19:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654786+0000", 2024-09-18T00:19:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8695999999999999e-05 2024-09-18T00:19:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654832+0000", 2024-09-18T00:19:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5735000000000002e-05 2024-09-18T00:19:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654860+0000", 2024-09-18T00:19:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7789000000000001e-05 2024-09-18T00:19:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655048+0000", 2024-09-18T00:19:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018781900000000001 2024-09-18T00:19:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:19:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655070+0000", 2024-09-18T00:19:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1916000000000002e-05 2024-09-18T00:19:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663335+0000", 2024-09-18T00:19:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0082651210000000003 2024-09-18T00:19:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663358+0000", 2024-09-18T00:19:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2986e-05 2024-09-18T00:19:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663739+0000", 2024-09-18T00:19:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0003814820000001 2024-09-18T00:19:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663920+0000", 2024-09-18T00:19:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018116900000000001 2024-09-18T00:19:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.665948+0000", 2024-09-18T00:19:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020273919999999998 2024-09-18T00:19:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15881 3.16 3:6cc14920:::rbd_header.112a3285400b:head [watch reconnect cookie 140685950184608 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:19:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.097361898999999, 2024-09-18T00:19:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0138723599999999, 2024-09-18T00:19:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15881 2024-09-18T00:19:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654926+0000", 2024-09-18T00:19:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6470000000000001e-06 2024-09-18T00:19:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654935+0000", 2024-09-18T00:19:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8859999999999993e-06 2024-09-18T00:19:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654960+0000", 2024-09-18T00:19:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4689999999999999e-05 2024-09-18T00:19:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654976+0000", 2024-09-18T00:19:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6319000000000001e-05 2024-09-18T00:19:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655238+0000", 2024-09-18T00:19:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026145999999999998 2024-09-18T00:19:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655255+0000", 2024-09-18T00:19:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7249999999999999e-05 2024-09-18T00:19:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663416+0000", 2024-09-18T00:19:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081607959999999997 2024-09-18T00:19:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663436+0000", 2024-09-18T00:19:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0384e-05 2024-09-18T00:19:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666184+0000", 2024-09-18T00:19:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002748121 2024-09-18T00:19:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666452+0000", 2024-09-18T00:19:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026801700000000001 2024-09-18T00:19:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.668797+0000", 2024-09-18T00:19:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023447899999999998 2024-09-18T00:19:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17089 3.f 3:f7681386:::rbd_data.112a6efb7564.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:19:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.872434802999997, 2024-09-18T00:19:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.12573889399999999, 2024-09-18T00:19:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17089 2024-09-18T00:19:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879854+0000", 2024-09-18T00:19:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:19:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879867+0000", 2024-09-18T00:19:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3103e-05 2024-09-18T00:19:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879898+0000", 2024-09-18T00:19:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0103e-05 2024-09-18T00:19:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879912+0000", 2024-09-18T00:19:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4636e-05 2024-09-18T00:19:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879992+0000", 2024-09-18T00:19:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9814000000000005e-05 2024-09-18T00:19:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.880101+0000", 2024-09-18T00:19:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109172 2024-09-18T00:19:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.887269+0000", 2024-09-18T00:19:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071679300000000003 2024-09-18T00:19:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005269+0000", 2024-09-18T00:19:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.117999621 2024-09-18T00:19:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005317+0000", 2024-09-18T00:19:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8279000000000003e-05 2024-09-18T00:19:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005464+0000", 2024-09-18T00:19:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014661200000000001 2024-09-18T00:19:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005591+0000", 2024-09-18T00:19:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000127107 2024-09-18T00:19:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17089 3.f e59/32 ondisk, result = 0)", 2024-09-18T00:19:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.848327937999997, 2024-09-18T00:19:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101334403, 2024-09-18T00:19:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903960+0000", 2024-09-18T00:19:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.33e-06 2024-09-18T00:19:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903966+0000", 2024-09-18T00:19:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5929999999999996e-06 2024-09-18T00:19:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903986+0000", 2024-09-18T00:19:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.018e-05 2024-09-18T00:19:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903997+0000", 2024-09-18T00:19:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0988999999999999e-05 2024-09-18T00:19:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005171+0000", 2024-09-18T00:19:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101174267 2024-09-18T00:19:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005200+0000", 2024-09-18T00:19:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8540000000000001e-05 2024-09-18T00:19:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005293+0000", 2024-09-18T00:19:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504000000000005e-05 2024-09-18T00:19:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24529 3.16 3:6857ce1a:::rbd_data.112a990280ef.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.115516527, 2024-09-18T00:19:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10693805200000001, 2024-09-18T00:19:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24529 2024-09-18T00:19:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636772+0000", 2024-09-18T00:19:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1100000000000001e-06 2024-09-18T00:19:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636783+0000", 2024-09-18T00:19:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1005999999999999e-05 2024-09-18T00:19:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636814+0000", 2024-09-18T00:19:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1047000000000003e-05 2024-09-18T00:19:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636829+0000", 2024-09-18T00:19:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4363e-05 2024-09-18T00:19:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636980+0000", 2024-09-18T00:19:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015181099999999999 2024-09-18T00:19:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.637237+0000", 2024-09-18T00:19:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000256352 2024-09-18T00:19:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743536+0000", 2024-09-18T00:19:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.106298825 2024-09-18T00:19:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743599+0000", 2024-09-18T00:19:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3449999999999997e-05 2024-09-18T00:19:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743708+0000", 2024-09-18T00:19:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109088 2024-09-18T00:19:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24805 3.16 3:6de5e15d:::rbd_data.112a990280ef.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.529713295000001, 2024-09-18T00:19:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14775107700000001, 2024-09-18T00:19:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24805 2024-09-18T00:19:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222574+0000", 2024-09-18T00:19:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0210000000000001e-06 2024-09-18T00:19:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222581+0000", 2024-09-18T00:19:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.144e-06 2024-09-18T00:19:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222595+0000", 2024-09-18T00:19:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4567e-05 2024-09-18T00:19:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222603+0000", 2024-09-18T00:19:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6529999999999994e-06 2024-09-18T00:19:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222687+0000", 2024-09-18T00:19:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4587999999999997e-05 2024-09-18T00:19:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222828+0000", 2024-09-18T00:19:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140178 2024-09-18T00:19:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370231+0000", 2024-09-18T00:19:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14740378700000001 2024-09-18T00:19:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370269+0000", 2024-09-18T00:19:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7886999999999998e-05 2024-09-18T00:19:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370324+0000", 2024-09-18T00:19:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5251999999999999e-05 2024-09-18T00:19:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24852 3.13 3:cc64f0fd:::rbd_data.112a990280ef.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.296973824999998, 2024-09-18T00:19:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.066392943999999995, 2024-09-18T00:19:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24852 2024-09-18T00:19:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455315+0000", 2024-09-18T00:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.705e-06 2024-09-18T00:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455324+0000", 2024-09-18T00:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8120000000000002e-06 2024-09-18T00:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455349+0000", 2024-09-18T00:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4250999999999999e-05 2024-09-18T00:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455361+0000", 2024-09-18T00:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2805e-05 2024-09-18T00:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455425+0000", 2024-09-18T00:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3981999999999998e-05 2024-09-18T00:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455480+0000", 2024-09-18T00:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4792e-05 2024-09-18T00:19:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.460671+0000", 2024-09-18T00:19:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051907300000000002 2024-09-18T00:19:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521355+0000", 2024-09-18T00:19:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060684208000000003 2024-09-18T00:19:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521493+0000", 2024-09-18T00:19:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013803200000000001 2024-09-18T00:19:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521590+0000", 2024-09-18T00:19:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6339e-05 2024-09-18T00:19:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521706+0000", 2024-09-18T00:19:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116288 2024-09-18T00:19:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:25269 3.f 3:f6adfd48:::rbd_data.112aa34d8ec3.0000000000000145:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T00:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.147383863999998, 2024-09-18T00:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.075209085999999994, 2024-09-18T00:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 25269 2024-09-18T00:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604904+0000", 2024-09-18T00:19:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:19:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604910+0000", 2024-09-18T00:19:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.964e-06 2024-09-18T00:19:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604925+0000", 2024-09-18T00:19:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4894e-05 2024-09-18T00:19:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604932+0000", 2024-09-18T00:19:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-18T00:19:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604990+0000", 2024-09-18T00:19:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8143000000000002e-05 2024-09-18T00:19:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.605104+0000", 2024-09-18T00:19:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011404399999999999 2024-09-18T00:19:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.608755+0000", 2024-09-18T00:19:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003651081 2024-09-18T00:19:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679920+0000", 2024-09-18T00:19:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.071164543999999996 2024-09-18T00:19:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679978+0000", 2024-09-18T00:19:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8585000000000002e-05 2024-09-18T00:19:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680036+0000", 2024-09-18T00:19:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7978e-05 2024-09-18T00:19:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680112+0000", 2024-09-18T00:19:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5527999999999999e-05 2024-09-18T00:19:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:25269 3.f e72/32 ondisk, result = 0)", 2024-09-18T00:19:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.134173479000001, 2024-09-18T00:19:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062014761000000002, 2024-09-18T00:19:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.819 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618114+0000", 2024-09-18T00:19:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1650000000000001e-06 2024-09-18T00:19:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618119+0000", 2024-09-18T00:19:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-18T00:19:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618134+0000", 2024-09-18T00:19:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5770999999999999e-05 2024-09-18T00:19:34.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618143+0000", 2024-09-18T00:19:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.551e-06 2024-09-18T00:19:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679967+0000", 2024-09-18T00:19:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061823937000000002 2024-09-18T00:19:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679975+0000", 2024-09-18T00:19:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7340000000000008e-06 2024-09-18T00:19:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680128+0000", 2024-09-18T00:19:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015338299999999999 2024-09-18T00:19:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48483 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 15671296~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.367214+0000", 2024-09-18T00:19:34.823 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.3850727759999995, 2024-09-18T00:19:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058250745999999999, 2024-09-18T00:19:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48483 2024-09-18T00:19:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367214+0000", 2024-09-18T00:19:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367214+0000", 2024-09-18T00:19:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367280+0000", 2024-09-18T00:19:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6088999999999997e-05 2024-09-18T00:19:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367284+0000", 2024-09-18T00:19:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0489999999999997e-06 2024-09-18T00:19:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367295+0000", 2024-09-18T00:19:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0908e-05 2024-09-18T00:19:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367303+0000", 2024-09-18T00:19:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2199999999999992e-06 2024-09-18T00:19:34.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425271+0000", 2024-09-18T00:19:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.057968325000000001 2024-09-18T00:19:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425358+0000", 2024-09-18T00:19:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6157999999999997e-05 2024-09-18T00:19:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425465+0000", 2024-09-18T00:19:34.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010699700000000001 2024-09-18T00:19:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48484 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 15704064~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:19:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.3849713700000006, 2024-09-18T00:19:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058450876999999998, 2024-09-18T00:19:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.828 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48484 2024-09-18T00:19:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:19:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:19:34.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370429+0000", 2024-09-18T00:19:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031138530000000002 2024-09-18T00:19:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370435+0000", 2024-09-18T00:19:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6269999999999997e-06 2024-09-18T00:19:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370455+0000", 2024-09-18T00:19:34.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0426999999999999e-05 2024-09-18T00:19:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370465+0000", 2024-09-18T00:19:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0118999999999999e-05 2024-09-18T00:19:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425525+0000", 2024-09-18T00:19:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055059943 2024-09-18T00:19:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425645+0000", 2024-09-18T00:19:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119392 2024-09-18T00:19:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425766+0000", 2024-09-18T00:19:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012151599999999999 2024-09-18T00:19:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48502 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16293888~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:34.832 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371167+0000", 2024-09-18T00:19:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.3811193660000001, 2024-09-18T00:19:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058236384000000002, 2024-09-18T00:19:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48502 2024-09-18T00:19:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371167+0000", 2024-09-18T00:19:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371167+0000", 2024-09-18T00:19:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371168+0000", 2024-09-18T00:19:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8400000000000004e-07 2024-09-18T00:19:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371171+0000", 2024-09-18T00:19:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9189999999999999e-06 2024-09-18T00:19:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371180+0000", 2024-09-18T00:19:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7320000000000004e-06 2024-09-18T00:19:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371187+0000", 2024-09-18T00:19:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0700000000000001e-06 2024-09-18T00:19:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429295+0000", 2024-09-18T00:19:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058108805999999999 2024-09-18T00:19:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429353+0000", 2024-09-18T00:19:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7445000000000003e-05 2024-09-18T00:19:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429404+0000", 2024-09-18T00:19:34.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0828000000000002e-05 2024-09-18T00:19:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48503 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16326656~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.3810909690000006, 2024-09-18T00:19:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058310395000000001, 2024-09-18T00:19:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.837 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48503 2024-09-18T00:19:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:34.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0100000000000004e-07 2024-09-18T00:19:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371199+0000", 2024-09-18T00:19:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8899999999999999e-06 2024-09-18T00:19:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371211+0000", 2024-09-18T00:19:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1707e-05 2024-09-18T00:19:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371218+0000", 2024-09-18T00:19:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.131e-06 2024-09-18T00:19:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429426+0000", 2024-09-18T00:19:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058207771999999998 2024-09-18T00:19:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429467+0000", 2024-09-18T00:19:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1532999999999997e-05 2024-09-18T00:19:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429506+0000", 2024-09-18T00:19:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8661000000000001e-05 2024-09-18T00:19:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.841 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48504 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16359424~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.3810591500000005, 2024-09-18T00:19:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058412775, 2024-09-18T00:19:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48504 2024-09-18T00:19:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6899999999999996e-07 2024-09-18T00:19:34.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371231+0000", 2024-09-18T00:19:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6709999999999999e-06 2024-09-18T00:19:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371240+0000", 2024-09-18T00:19:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7309999999999999e-06 2024-09-18T00:19:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371249+0000", 2024-09-18T00:19:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4280000000000002e-06 2024-09-18T00:19:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429533+0000", 2024-09-18T00:19:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058283852999999997 2024-09-18T00:19:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429590+0000", 2024-09-18T00:19:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6824999999999999e-05 2024-09-18T00:19:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429640+0000", 2024-09-18T00:19:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0498000000000003e-05 2024-09-18T00:19:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48505 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16392192~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.3810261770000007, 2024-09-18T00:19:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058482777999999999, 2024-09-18T00:19:34.846 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48505 2024-09-18T00:19:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7800000000000001e-07 2024-09-18T00:19:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371264+0000", 2024-09-18T00:19:34.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-18T00:19:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371274+0000", 2024-09-18T00:19:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4690000000000003e-06 2024-09-18T00:19:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371281+0000", 2024-09-18T00:19:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4120000000000004e-06 2024-09-18T00:19:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429665+0000", 2024-09-18T00:19:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058384108999999997 2024-09-18T00:19:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429705+0000", 2024-09-18T00:19:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0299000000000002e-05 2024-09-18T00:19:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429743+0000", 2024-09-18T00:19:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7951000000000003e-05 2024-09-18T00:19:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.850 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48506 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16424960~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:19:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.3809969649999996, 2024-09-18T00:19:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058604485999999997, 2024-09-18T00:19:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48506 2024-09-18T00:19:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:19:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:19:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371355+0000", 2024-09-18T00:19:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5549000000000005e-05 2024-09-18T00:19:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371359+0000", 2024-09-18T00:19:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1049999999999997e-06 2024-09-18T00:19:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371380+0000", 2024-09-18T00:19:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0205000000000001e-05 2024-09-18T00:19:34.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371388+0000", 2024-09-18T00:19:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3229999999999998e-06 2024-09-18T00:19:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429771+0000", 2024-09-18T00:19:34.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058382878999999999 2024-09-18T00:19:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429842+0000", 2024-09-18T00:19:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1110000000000002e-05 2024-09-18T00:19:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429894+0000", 2024-09-18T00:19:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2315000000000003e-05 2024-09-18T00:19:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.855 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48507 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16457728~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:19:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.3808856049999996, 2024-09-18T00:19:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058597275999999997, 2024-09-18T00:19:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48507 2024-09-18T00:19:34.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:19:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:19:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:34.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374394+0000", 2024-09-18T00:19:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029924449999999998 2024-09-18T00:19:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374400+0000", 2024-09-18T00:19:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0229999999999998e-06 2024-09-18T00:19:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374419+0000", 2024-09-18T00:19:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9488e-05 2024-09-18T00:19:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374432+0000", 2024-09-18T00:19:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2492e-05 2024-09-18T00:19:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429917+0000", 2024-09-18T00:19:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055485087000000002 2024-09-18T00:19:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429958+0000", 2024-09-18T00:19:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1258999999999999e-05 2024-09-18T00:19:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:34.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429998+0000", 2024-09-18T00:19:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0482000000000003e-05 2024-09-18T00:19:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:34.860 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:34.860 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:34.862 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:19:34.863 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:34.857+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:19:34.863 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 93, 2024-09-18T00:19:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: "fsid": "b993f56a-c4ef-4602-bb41-f32ae406b45a", 2024-09-18T00:19:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: "created": "2024-09-18T00:17:35.658788+0000", 2024-09-18T00:19:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: "modified": "2024-09-18T00:19:34.483118+0000", 2024-09-18T00:19:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: "last_up_change": "2024-09-18T00:19:14.285211+0000", 2024-09-18T00:19:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: "last_in_change": "2024-09-18T00:19:23.376648+0000", 2024-09-18T00:19:34.863 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T00:19:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "flags_num": 5804032, 2024-09-18T00:19:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "flags_set": [ 2024-09-18T00:19:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "nodeep-scrub", 2024-09-18T00:19:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "pglog_hardlimit", 2024-09-18T00:19:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "purged_snapdirs", 2024-09-18T00:19:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_deletes", 2024-09-18T00:19:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "sortbitwise" 2024-09-18T00:19:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-18T00:19:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_version": 6, 2024-09-18T00:19:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T00:19:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T00:19:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T00:19:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_snapshot": "", 2024-09-18T00:19:34.864 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_max": 3, 2024-09-18T00:19:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "max_osd": 8, 2024-09-18T00:19:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "require_min_compat_client": "luminous", 2024-09-18T00:19:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "min_compat_client": "jewel", 2024-09-18T00:19:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "require_osd_release": "squid", 2024-09-18T00:19:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_crimson": false, 2024-09-18T00:19:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "pools": [ 2024-09-18T00:19:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "pool": 1, 2024-09-18T00:19:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_name": ".mgr", 2024-09-18T00:19:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "create_time": "2024-09-18T00:17:50.375367+0000", 2024-09-18T00:19:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-09-18T00:19:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "flags_names": "hashpspool", 2024-09-18T00:19:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "type": 1, 2024-09-18T00:19:34.865 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 2, 2024-09-18T00:19:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "min_size": 1, 2024-09-18T00:19:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_rule": 0, 2024-09-18T00:19:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:19:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:19:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:19:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:19:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "is_stretch_pool": false, 2024-09-18T00:19:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "object_hash": 2, 2024-09-18T00:19:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:19:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num": 1, 2024-09-18T00:19:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_placement_num": 1, 2024-09-18T00:19:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_placement_num_target": 1, 2024-09-18T00:19:34.866 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num_target": 1, 2024-09-18T00:19:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num_pending": 1, 2024-09-18T00:19:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "last_pg_merge_meta": { 2024-09-18T00:19:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "source_pgid": "0.0", 2024-09-18T00:19:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "ready_epoch": 0, 2024-09-18T00:19:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "last_epoch_started": 0, 2024-09-18T00:19:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "last_epoch_clean": 0, 2024-09-18T00:19:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "source_version": "0'0", 2024-09-18T00:19:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "target_version": "0'0" 2024-09-18T00:19:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "last_change": "18", 2024-09-18T00:19:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "last_force_op_resend": "0", 2024-09-18T00:19:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:19:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:19:34.867 INFO:teuthology.orchestra.run.smithi097.stdout: "auid": 0, 2024-09-18T00:19:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:19:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "snap_seq": 0, 2024-09-18T00:19:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "snap_epoch": 0, 2024-09-18T00:19:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_snaps": [], 2024-09-18T00:19:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "removed_snaps": "[]", 2024-09-18T00:19:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "quota_max_bytes": 0, 2024-09-18T00:19:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "quota_max_objects": 0, 2024-09-18T00:19:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "tiers": [], 2024-09-18T00:19:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "tier_of": -1, 2024-09-18T00:19:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "read_tier": -1, 2024-09-18T00:19:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "write_tier": -1, 2024-09-18T00:19:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_mode": "none", 2024-09-18T00:19:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "target_max_bytes": 0, 2024-09-18T00:19:34.868 INFO:teuthology.orchestra.run.smithi097.stdout: "target_max_objects": 0, 2024-09-18T00:19:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:19:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:19:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:19:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_min_flush_age": 0, 2024-09-18T00:19:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_min_evict_age": 0, 2024-09-18T00:19:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "erasure_code_profile": "", 2024-09-18T00:19:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "hit_set_params": { 2024-09-18T00:19:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "none" 2024-09-18T00:19:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "hit_set_period": 0, 2024-09-18T00:19:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "hit_set_count": 0, 2024-09-18T00:19:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "use_gmt_hitset": true, 2024-09-18T00:19:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:19:34.869 INFO:teuthology.orchestra.run.smithi097.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:19:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:19:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:19:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "grade_table": [], 2024-09-18T00:19:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "stripe_width": 0, 2024-09-18T00:19:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "expected_num_objects": 0, 2024-09-18T00:19:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "fast_read": false, 2024-09-18T00:19:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "options": { 2024-09-18T00:19:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num_max": 32, 2024-09-18T00:19:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num_min": 1 2024-09-18T00:19:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "application_metadata": { 2024-09-18T00:19:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": {} 2024-09-18T00:19:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.870 INFO:teuthology.orchestra.run.smithi097.stdout: "read_balance": { 2024-09-18T00:19:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "score_type": "Fair distribution", 2024-09-18T00:19:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "score_acting": 8, 2024-09-18T00:19:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "score_stable": 8, 2024-09-18T00:19:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "optimal_score": 0.25, 2024-09-18T00:19:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "raw_score_acting": 2, 2024-09-18T00:19:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "raw_score_stable": 2, 2024-09-18T00:19:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:19:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "average_primary_affinity": 1, 2024-09-18T00:19:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:19:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "pool": 2, 2024-09-18T00:19:34.871 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_name": "rbd", 2024-09-18T00:19:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "create_time": "2024-09-18T00:17:53.896718+0000", 2024-09-18T00:19:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 8193, 2024-09-18T00:19:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T00:19:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "type": 1, 2024-09-18T00:19:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 2, 2024-09-18T00:19:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "min_size": 1, 2024-09-18T00:19:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_rule": 0, 2024-09-18T00:19:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:19:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:19:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:19:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:19:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "is_stretch_pool": false, 2024-09-18T00:19:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: "object_hash": 2, 2024-09-18T00:19:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:19:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num": 8, 2024-09-18T00:19:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_placement_num": 8, 2024-09-18T00:19:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_placement_num_target": 8, 2024-09-18T00:19:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num_target": 8, 2024-09-18T00:19:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num_pending": 8, 2024-09-18T00:19:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "last_pg_merge_meta": { 2024-09-18T00:19:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "source_pgid": "0.0", 2024-09-18T00:19:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "ready_epoch": 0, 2024-09-18T00:19:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "last_epoch_started": 0, 2024-09-18T00:19:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "last_epoch_clean": 0, 2024-09-18T00:19:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "source_version": "0'0", 2024-09-18T00:19:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: "target_version": "0'0" 2024-09-18T00:19:34.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "last_change": "22", 2024-09-18T00:19:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "last_force_op_resend": "0", 2024-09-18T00:19:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:19:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:19:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "auid": 0, 2024-09-18T00:19:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:19:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "snap_seq": 2, 2024-09-18T00:19:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "snap_epoch": 22, 2024-09-18T00:19:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_snaps": [], 2024-09-18T00:19:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "removed_snaps": "[]", 2024-09-18T00:19:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "quota_max_bytes": 0, 2024-09-18T00:19:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "quota_max_objects": 0, 2024-09-18T00:19:34.874 INFO:teuthology.orchestra.run.smithi097.stdout: "tiers": [], 2024-09-18T00:19:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "tier_of": -1, 2024-09-18T00:19:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "read_tier": -1, 2024-09-18T00:19:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "write_tier": -1, 2024-09-18T00:19:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_mode": "none", 2024-09-18T00:19:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "target_max_bytes": 0, 2024-09-18T00:19:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "target_max_objects": 0, 2024-09-18T00:19:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:19:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:19:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:19:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_min_flush_age": 0, 2024-09-18T00:19:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_min_evict_age": 0, 2024-09-18T00:19:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "erasure_code_profile": "", 2024-09-18T00:19:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "hit_set_params": { 2024-09-18T00:19:34.875 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "none" 2024-09-18T00:19:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "hit_set_period": 0, 2024-09-18T00:19:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "hit_set_count": 0, 2024-09-18T00:19:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "use_gmt_hitset": true, 2024-09-18T00:19:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:19:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:19:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:19:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:19:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "grade_table": [], 2024-09-18T00:19:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "stripe_width": 0, 2024-09-18T00:19:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "expected_num_objects": 0, 2024-09-18T00:19:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "fast_read": false, 2024-09-18T00:19:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "options": {}, 2024-09-18T00:19:34.876 INFO:teuthology.orchestra.run.smithi097.stdout: "application_metadata": { 2024-09-18T00:19:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd": {} 2024-09-18T00:19:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "read_balance": { 2024-09-18T00:19:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "score_type": "Fair distribution", 2024-09-18T00:19:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "score_acting": 3, 2024-09-18T00:19:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "score_stable": 3, 2024-09-18T00:19:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "optimal_score": 0.75, 2024-09-18T00:19:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "raw_score_acting": 2.25, 2024-09-18T00:19:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "raw_score_stable": 2.25, 2024-09-18T00:19:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:19:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "average_primary_affinity": 1, 2024-09-18T00:19:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:19:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "pool": 3, 2024-09-18T00:19:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_name": "pool_client.0", 2024-09-18T00:19:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "create_time": "2024-09-18T00:18:03.715378+0000", 2024-09-18T00:19:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 8193, 2024-09-18T00:19:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T00:19:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "type": 1, 2024-09-18T00:19:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 2, 2024-09-18T00:19:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "min_size": 1, 2024-09-18T00:19:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_rule": 0, 2024-09-18T00:19:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "peering_crush_bucket_count": 0, 2024-09-18T00:19:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "peering_crush_bucket_target": 0, 2024-09-18T00:19:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T00:19:34.878 INFO:teuthology.orchestra.run.smithi097.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T00:19:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "is_stretch_pool": false, 2024-09-18T00:19:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "object_hash": 2, 2024-09-18T00:19:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscale_mode": "off", 2024-09-18T00:19:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num": 32, 2024-09-18T00:19:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_placement_num": 32, 2024-09-18T00:19:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_placement_num_target": 32, 2024-09-18T00:19:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num_target": 32, 2024-09-18T00:19:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num_pending": 32, 2024-09-18T00:19:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "last_pg_merge_meta": { 2024-09-18T00:19:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "source_pgid": "0.0", 2024-09-18T00:19:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "ready_epoch": 0, 2024-09-18T00:19:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "last_epoch_started": 0, 2024-09-18T00:19:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "last_epoch_clean": 0, 2024-09-18T00:19:34.879 INFO:teuthology.orchestra.run.smithi097.stdout: "source_version": "0'0", 2024-09-18T00:19:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "target_version": "0'0" 2024-09-18T00:19:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "last_change": "92", 2024-09-18T00:19:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "last_force_op_resend": "0", 2024-09-18T00:19:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T00:19:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T00:19:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "auid": 0, 2024-09-18T00:19:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "snap_mode": "selfmanaged", 2024-09-18T00:19:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "snap_seq": 27, 2024-09-18T00:19:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "snap_epoch": 92, 2024-09-18T00:19:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_snaps": [], 2024-09-18T00:19:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "removed_snaps": "[]", 2024-09-18T00:19:34.880 INFO:teuthology.orchestra.run.smithi097.stdout: "quota_max_bytes": 0, 2024-09-18T00:19:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "quota_max_objects": 0, 2024-09-18T00:19:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "tiers": [], 2024-09-18T00:19:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "tier_of": -1, 2024-09-18T00:19:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "read_tier": -1, 2024-09-18T00:19:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "write_tier": -1, 2024-09-18T00:19:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_mode": "none", 2024-09-18T00:19:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "target_max_bytes": 0, 2024-09-18T00:19:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "target_max_objects": 0, 2024-09-18T00:19:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T00:19:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T00:19:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T00:19:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_min_flush_age": 0, 2024-09-18T00:19:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "cache_min_evict_age": 0, 2024-09-18T00:19:34.881 INFO:teuthology.orchestra.run.smithi097.stdout: "erasure_code_profile": "", 2024-09-18T00:19:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "hit_set_params": { 2024-09-18T00:19:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "none" 2024-09-18T00:19:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "hit_set_period": 0, 2024-09-18T00:19:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "hit_set_count": 0, 2024-09-18T00:19:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "use_gmt_hitset": true, 2024-09-18T00:19:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "min_read_recency_for_promote": 0, 2024-09-18T00:19:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "min_write_recency_for_promote": 0, 2024-09-18T00:19:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T00:19:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "hit_set_search_last_n": 0, 2024-09-18T00:19:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "grade_table": [], 2024-09-18T00:19:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "stripe_width": 0, 2024-09-18T00:19:34.882 INFO:teuthology.orchestra.run.smithi097.stdout: "expected_num_objects": 0, 2024-09-18T00:19:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "fast_read": false, 2024-09-18T00:19:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "options": {}, 2024-09-18T00:19:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "application_metadata": { 2024-09-18T00:19:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd": {} 2024-09-18T00:19:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "read_balance": { 2024-09-18T00:19:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "score_type": "Fair distribution", 2024-09-18T00:19:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "score_acting": 1.75, 2024-09-18T00:19:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "score_stable": 1.75, 2024-09-18T00:19:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "optimal_score": 0.75, 2024-09-18T00:19:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "raw_score_acting": 1.309999942779541, 2024-09-18T00:19:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "raw_score_stable": 1.309999942779541, 2024-09-18T00:19:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "primary_affinity_weighted": 1, 2024-09-18T00:19:34.883 INFO:teuthology.orchestra.run.smithi097.stdout: "average_primary_affinity": 1, 2024-09-18T00:19:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "average_primary_affinity_weighted": 1 2024-09-18T00:19:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-18T00:19:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "osds": [ 2024-09-18T00:19:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": 0, 2024-09-18T00:19:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "uuid": "bfda7950-3d83-435d-8313-2f14345096ec", 2024-09-18T00:19:34.884 INFO:teuthology.orchestra.run.smithi097.stdout: "up": 1, 2024-09-18T00:19:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "in": 1, 2024-09-18T00:19:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "weight": 1, 2024-09-18T00:19:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "primary_affinity": 1, 2024-09-18T00:19:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "last_clean_begin": 0, 2024-09-18T00:19:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "last_clean_end": 0, 2024-09-18T00:19:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "up_from": 14, 2024-09-18T00:19:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "up_thru": 87, 2024-09-18T00:19:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "down_at": 0, 2024-09-18T00:19:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "lost_at": 0, 2024-09-18T00:19:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addrs": { 2024-09-18T00:19:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6800", 2024-09-18T00:19:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 40468291 2024-09-18T00:19:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6801", 2024-09-18T00:19:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 40468291 2024-09-18T00:19:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addrs": { 2024-09-18T00:19:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6802", 2024-09-18T00:19:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 40468291 2024-09-18T00:19:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6803", 2024-09-18T00:19:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 40468291 2024-09-18T00:19:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_back_addrs": { 2024-09-18T00:19:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6806", 2024-09-18T00:19:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 40468291 2024-09-18T00:19:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6807", 2024-09-18T00:19:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 40468291 2024-09-18T00:19:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.888 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_front_addrs": { 2024-09-18T00:19:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6804", 2024-09-18T00:19:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 40468291 2024-09-18T00:19:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6805", 2024-09-18T00:19:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 40468291 2024-09-18T00:19:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.889 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addr": "172.21.15.97:6801/40468291", 2024-09-18T00:19:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addr": "172.21.15.97:6803/40468291", 2024-09-18T00:19:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_back_addr": "172.21.15.97:6807/40468291", 2024-09-18T00:19:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_front_addr": "172.21.15.97:6805/40468291", 2024-09-18T00:19:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "state": [ 2024-09-18T00:19:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "exists", 2024-09-18T00:19:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "up" 2024-09-18T00:19:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": 1, 2024-09-18T00:19:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "uuid": "35949061-aa30-4aff-b39d-90aebeffb115", 2024-09-18T00:19:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "up": 0, 2024-09-18T00:19:34.890 INFO:teuthology.orchestra.run.smithi097.stdout: "in": 0, 2024-09-18T00:19:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "weight": 0, 2024-09-18T00:19:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "primary_affinity": 1, 2024-09-18T00:19:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "last_clean_begin": 0, 2024-09-18T00:19:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "last_clean_end": 0, 2024-09-18T00:19:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "up_from": 15, 2024-09-18T00:19:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "up_thru": 27, 2024-09-18T00:19:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "down_at": 46, 2024-09-18T00:19:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "lost_at": 0, 2024-09-18T00:19:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addrs": { 2024-09-18T00:19:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6824", 2024-09-18T00:19:34.891 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2443452537 2024-09-18T00:19:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6825", 2024-09-18T00:19:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2443452537 2024-09-18T00:19:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addrs": { 2024-09-18T00:19:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.892 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6826", 2024-09-18T00:19:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2443452537 2024-09-18T00:19:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6827", 2024-09-18T00:19:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2443452537 2024-09-18T00:19:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_back_addrs": { 2024-09-18T00:19:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.893 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6830", 2024-09-18T00:19:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2443452537 2024-09-18T00:19:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6831", 2024-09-18T00:19:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2443452537 2024-09-18T00:19:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_front_addrs": { 2024-09-18T00:19:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6828", 2024-09-18T00:19:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2443452537 2024-09-18T00:19:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6829", 2024-09-18T00:19:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2443452537 2024-09-18T00:19:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addr": "172.21.15.97:6825/2443452537", 2024-09-18T00:19:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addr": "172.21.15.97:6827/2443452537", 2024-09-18T00:19:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_back_addr": "172.21.15.97:6831/2443452537", 2024-09-18T00:19:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_front_addr": "172.21.15.97:6829/2443452537", 2024-09-18T00:19:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "state": [ 2024-09-18T00:19:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "exists" 2024-09-18T00:19:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": 2, 2024-09-18T00:19:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "uuid": "31b3e1c1-7648-44fe-9931-758ffe9a53ca", 2024-09-18T00:19:34.896 INFO:teuthology.orchestra.run.smithi097.stdout: "up": 1, 2024-09-18T00:19:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "in": 1, 2024-09-18T00:19:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "weight": 1, 2024-09-18T00:19:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "primary_affinity": 1, 2024-09-18T00:19:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "last_clean_begin": 0, 2024-09-18T00:19:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "last_clean_end": 0, 2024-09-18T00:19:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "up_from": 15, 2024-09-18T00:19:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "up_thru": 87, 2024-09-18T00:19:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "down_at": 0, 2024-09-18T00:19:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "lost_at": 0, 2024-09-18T00:19:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addrs": { 2024-09-18T00:19:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.897 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6816", 2024-09-18T00:19:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2425060925 2024-09-18T00:19:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6817", 2024-09-18T00:19:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2425060925 2024-09-18T00:19:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addrs": { 2024-09-18T00:19:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.898 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6818", 2024-09-18T00:19:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2425060925 2024-09-18T00:19:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6819", 2024-09-18T00:19:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2425060925 2024-09-18T00:19:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_back_addrs": { 2024-09-18T00:19:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6822", 2024-09-18T00:19:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2425060925 2024-09-18T00:19:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6823", 2024-09-18T00:19:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2425060925 2024-09-18T00:19:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_front_addrs": { 2024-09-18T00:19:34.900 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6820", 2024-09-18T00:19:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2425060925 2024-09-18T00:19:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6821", 2024-09-18T00:19:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2425060925 2024-09-18T00:19:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addr": "172.21.15.97:6817/2425060925", 2024-09-18T00:19:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addr": "172.21.15.97:6819/2425060925", 2024-09-18T00:19:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_back_addr": "172.21.15.97:6823/2425060925", 2024-09-18T00:19:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_front_addr": "172.21.15.97:6821/2425060925", 2024-09-18T00:19:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "state": [ 2024-09-18T00:19:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "exists", 2024-09-18T00:19:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "up" 2024-09-18T00:19:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": 3, 2024-09-18T00:19:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "uuid": "d7daebbc-0e33-42ab-97ee-a6c1b449ae8d", 2024-09-18T00:19:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "up": 1, 2024-09-18T00:19:34.902 INFO:teuthology.orchestra.run.smithi097.stdout: "in": 1, 2024-09-18T00:19:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "weight": 1, 2024-09-18T00:19:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "primary_affinity": 1, 2024-09-18T00:19:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "last_clean_begin": 0, 2024-09-18T00:19:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "last_clean_end": 0, 2024-09-18T00:19:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "up_from": 14, 2024-09-18T00:19:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "up_thru": 87, 2024-09-18T00:19:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "down_at": 0, 2024-09-18T00:19:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "lost_at": 0, 2024-09-18T00:19:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addrs": { 2024-09-18T00:19:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6808", 2024-09-18T00:19:34.903 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3512666164 2024-09-18T00:19:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6809", 2024-09-18T00:19:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3512666164 2024-09-18T00:19:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addrs": { 2024-09-18T00:19:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.904 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6810", 2024-09-18T00:19:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3512666164 2024-09-18T00:19:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6811", 2024-09-18T00:19:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3512666164 2024-09-18T00:19:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_back_addrs": { 2024-09-18T00:19:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.905 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6814", 2024-09-18T00:19:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3512666164 2024-09-18T00:19:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6815", 2024-09-18T00:19:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3512666164 2024-09-18T00:19:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_front_addrs": { 2024-09-18T00:19:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.906 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6812", 2024-09-18T00:19:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3512666164 2024-09-18T00:19:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6813", 2024-09-18T00:19:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3512666164 2024-09-18T00:19:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addr": "172.21.15.97:6809/3512666164", 2024-09-18T00:19:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addr": "172.21.15.97:6811/3512666164", 2024-09-18T00:19:34.907 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_back_addr": "172.21.15.97:6815/3512666164", 2024-09-18T00:19:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_front_addr": "172.21.15.97:6813/3512666164", 2024-09-18T00:19:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "state": [ 2024-09-18T00:19:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "exists", 2024-09-18T00:19:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "up" 2024-09-18T00:19:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": 4, 2024-09-18T00:19:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "uuid": "d55ca69b-5000-45ff-acc4-00f7c4763cad", 2024-09-18T00:19:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "up": 1, 2024-09-18T00:19:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "in": 1, 2024-09-18T00:19:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "weight": 1, 2024-09-18T00:19:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "primary_affinity": 1, 2024-09-18T00:19:34.908 INFO:teuthology.orchestra.run.smithi097.stdout: "last_clean_begin": 0, 2024-09-18T00:19:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "last_clean_end": 0, 2024-09-18T00:19:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "up_from": 15, 2024-09-18T00:19:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "up_thru": 87, 2024-09-18T00:19:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "down_at": 0, 2024-09-18T00:19:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "lost_at": 0, 2024-09-18T00:19:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addrs": { 2024-09-18T00:19:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6802", 2024-09-18T00:19:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3191744356 2024-09-18T00:19:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6804", 2024-09-18T00:19:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3191744356 2024-09-18T00:19:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addrs": { 2024-09-18T00:19:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6806", 2024-09-18T00:19:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3191744356 2024-09-18T00:19:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6808", 2024-09-18T00:19:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3191744356 2024-09-18T00:19:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_back_addrs": { 2024-09-18T00:19:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6814", 2024-09-18T00:19:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3191744356 2024-09-18T00:19:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6815", 2024-09-18T00:19:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3191744356 2024-09-18T00:19:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_front_addrs": { 2024-09-18T00:19:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6810", 2024-09-18T00:19:34.912 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3191744356 2024-09-18T00:19:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6812", 2024-09-18T00:19:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3191744356 2024-09-18T00:19:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addr": "172.21.15.114:6804/3191744356", 2024-09-18T00:19:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addr": "172.21.15.114:6808/3191744356", 2024-09-18T00:19:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_back_addr": "172.21.15.114:6815/3191744356", 2024-09-18T00:19:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_front_addr": "172.21.15.114:6812/3191744356", 2024-09-18T00:19:34.913 INFO:teuthology.orchestra.run.smithi097.stdout: "state": [ 2024-09-18T00:19:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "exists", 2024-09-18T00:19:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "up" 2024-09-18T00:19:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": 5, 2024-09-18T00:19:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "uuid": "196a329b-bd93-4cc2-85bb-64ed81d3918b", 2024-09-18T00:19:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "up": 1, 2024-09-18T00:19:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "in": 1, 2024-09-18T00:19:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "weight": 1, 2024-09-18T00:19:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "primary_affinity": 1, 2024-09-18T00:19:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "last_clean_begin": 15, 2024-09-18T00:19:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "last_clean_end": 55, 2024-09-18T00:19:34.914 INFO:teuthology.orchestra.run.smithi097.stdout: "up_from": 76, 2024-09-18T00:19:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "up_thru": 84, 2024-09-18T00:19:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "down_at": 56, 2024-09-18T00:19:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "lost_at": 0, 2024-09-18T00:19:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addrs": { 2024-09-18T00:19:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6800", 2024-09-18T00:19:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 407304039 2024-09-18T00:19:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6801", 2024-09-18T00:19:34.915 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 407304039 2024-09-18T00:19:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addrs": { 2024-09-18T00:19:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6803", 2024-09-18T00:19:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 407304039 2024-09-18T00:19:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.916 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6805", 2024-09-18T00:19:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 407304039 2024-09-18T00:19:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_back_addrs": { 2024-09-18T00:19:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6811", 2024-09-18T00:19:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 407304039 2024-09-18T00:19:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.917 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6813", 2024-09-18T00:19:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 407304039 2024-09-18T00:19:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_front_addrs": { 2024-09-18T00:19:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6807", 2024-09-18T00:19:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 407304039 2024-09-18T00:19:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6809", 2024-09-18T00:19:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 407304039 2024-09-18T00:19:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addr": "172.21.15.114:6801/407304039", 2024-09-18T00:19:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addr": "172.21.15.114:6805/407304039", 2024-09-18T00:19:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_back_addr": "172.21.15.114:6813/407304039", 2024-09-18T00:19:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_front_addr": "172.21.15.114:6809/407304039", 2024-09-18T00:19:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "state": [ 2024-09-18T00:19:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "exists", 2024-09-18T00:19:34.919 INFO:teuthology.orchestra.run.smithi097.stdout: "up" 2024-09-18T00:19:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": 6, 2024-09-18T00:19:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "uuid": "dc9712f0-8858-467a-8397-d5eb97187165", 2024-09-18T00:19:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "up": 1, 2024-09-18T00:19:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "in": 0, 2024-09-18T00:19:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "weight": 0, 2024-09-18T00:19:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "primary_affinity": 1, 2024-09-18T00:19:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "last_clean_begin": 0, 2024-09-18T00:19:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "last_clean_end": 0, 2024-09-18T00:19:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "up_from": 15, 2024-09-18T00:19:34.920 INFO:teuthology.orchestra.run.smithi097.stdout: "up_thru": 46, 2024-09-18T00:19:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "down_at": 0, 2024-09-18T00:19:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "lost_at": 0, 2024-09-18T00:19:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addrs": { 2024-09-18T00:19:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6824", 2024-09-18T00:19:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1148348470 2024-09-18T00:19:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6825", 2024-09-18T00:19:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1148348470 2024-09-18T00:19:34.921 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addrs": { 2024-09-18T00:19:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6826", 2024-09-18T00:19:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1148348470 2024-09-18T00:19:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6827", 2024-09-18T00:19:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1148348470 2024-09-18T00:19:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_back_addrs": { 2024-09-18T00:19:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6830", 2024-09-18T00:19:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1148348470 2024-09-18T00:19:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.923 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6831", 2024-09-18T00:19:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1148348470 2024-09-18T00:19:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_front_addrs": { 2024-09-18T00:19:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6828", 2024-09-18T00:19:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1148348470 2024-09-18T00:19:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6829", 2024-09-18T00:19:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1148348470 2024-09-18T00:19:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addr": "172.21.15.114:6825/1148348470", 2024-09-18T00:19:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addr": "172.21.15.114:6827/1148348470", 2024-09-18T00:19:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_back_addr": "172.21.15.114:6831/1148348470", 2024-09-18T00:19:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_front_addr": "172.21.15.114:6829/1148348470", 2024-09-18T00:19:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: "state": [ 2024-09-18T00:19:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: "exists", 2024-09-18T00:19:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: "up" 2024-09-18T00:19:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": 7, 2024-09-18T00:19:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "uuid": "b6d774ee-fe6d-462c-afa4-7ffd5d7343a9", 2024-09-18T00:19:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "up": 1, 2024-09-18T00:19:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "in": 1, 2024-09-18T00:19:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "weight": 1, 2024-09-18T00:19:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "primary_affinity": 1, 2024-09-18T00:19:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "last_clean_begin": 0, 2024-09-18T00:19:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "last_clean_end": 0, 2024-09-18T00:19:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "up_from": 15, 2024-09-18T00:19:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "up_thru": 84, 2024-09-18T00:19:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "down_at": 0, 2024-09-18T00:19:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "lost_at": 0, 2024-09-18T00:19:34.926 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addrs": { 2024-09-18T00:19:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6816", 2024-09-18T00:19:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2133497352 2024-09-18T00:19:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6817", 2024-09-18T00:19:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2133497352 2024-09-18T00:19:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.927 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addrs": { 2024-09-18T00:19:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6818", 2024-09-18T00:19:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2133497352 2024-09-18T00:19:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6819", 2024-09-18T00:19:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2133497352 2024-09-18T00:19:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.928 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_back_addrs": { 2024-09-18T00:19:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6822", 2024-09-18T00:19:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2133497352 2024-09-18T00:19:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6823", 2024-09-18T00:19:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2133497352 2024-09-18T00:19:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.929 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_front_addrs": { 2024-09-18T00:19:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-09-18T00:19:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-09-18T00:19:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6820", 2024-09-18T00:19:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2133497352 2024-09-18T00:19:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-09-18T00:19:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.114:6821", 2024-09-18T00:19:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2133497352 2024-09-18T00:19:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.930 INFO:teuthology.orchestra.run.smithi097.stdout: "public_addr": "172.21.15.114:6817/2133497352", 2024-09-18T00:19:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_addr": "172.21.15.114:6819/2133497352", 2024-09-18T00:19:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_back_addr": "172.21.15.114:6823/2133497352", 2024-09-18T00:19:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: "heartbeat_front_addr": "172.21.15.114:6821/2133497352", 2024-09-18T00:19:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: "state": [ 2024-09-18T00:19:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: "exists", 2024-09-18T00:19:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: "up" 2024-09-18T00:19:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-18T00:19:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_xinfo": [ 2024-09-18T00:19:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": 0, 2024-09-18T00:19:34.931 INFO:teuthology.orchestra.run.smithi097.stdout: "down_stamp": "0.000000", 2024-09-18T00:19:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "laggy_probability": 0, 2024-09-18T00:19:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "laggy_interval": 0, 2024-09-18T00:19:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "features": 4540701547738038271, 2024-09-18T00:19:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "old_weight": 0, 2024-09-18T00:19:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "last_purged_snaps_scrub": "2024-09-18T00:17:44.237241+0000", 2024-09-18T00:19:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "dead_epoch": 0 2024-09-18T00:19:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": 1, 2024-09-18T00:19:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "down_stamp": "0.000000", 2024-09-18T00:19:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "laggy_probability": 0, 2024-09-18T00:19:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "laggy_interval": 0, 2024-09-18T00:19:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "features": 4540701547738038271, 2024-09-18T00:19:34.932 INFO:teuthology.orchestra.run.smithi097.stdout: "old_weight": 65536, 2024-09-18T00:19:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: "last_purged_snaps_scrub": "2024-09-18T00:17:44.504107+0000", 2024-09-18T00:19:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: "dead_epoch": 46 2024-09-18T00:19:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": 2, 2024-09-18T00:19:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: "down_stamp": "0.000000", 2024-09-18T00:19:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: "laggy_probability": 0, 2024-09-18T00:19:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: "laggy_interval": 0, 2024-09-18T00:19:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: "features": 4540701547738038271, 2024-09-18T00:19:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: "old_weight": 0, 2024-09-18T00:19:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: "last_purged_snaps_scrub": "2024-09-18T00:17:44.690795+0000", 2024-09-18T00:19:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: "dead_epoch": 0 2024-09-18T00:19:34.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": 3, 2024-09-18T00:19:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "down_stamp": "0.000000", 2024-09-18T00:19:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "laggy_probability": 0, 2024-09-18T00:19:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "laggy_interval": 0, 2024-09-18T00:19:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "features": 4540701547738038271, 2024-09-18T00:19:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "old_weight": 0, 2024-09-18T00:19:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "last_purged_snaps_scrub": "2024-09-18T00:17:44.197056+0000", 2024-09-18T00:19:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "dead_epoch": 0 2024-09-18T00:19:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": 4, 2024-09-18T00:19:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "down_stamp": "0.000000", 2024-09-18T00:19:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "laggy_probability": 0, 2024-09-18T00:19:34.934 INFO:teuthology.orchestra.run.smithi097.stdout: "laggy_interval": 0, 2024-09-18T00:19:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "features": 4540701547738038271, 2024-09-18T00:19:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "old_weight": 0, 2024-09-18T00:19:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "last_purged_snaps_scrub": "2024-09-18T00:17:44.459497+0000", 2024-09-18T00:19:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "dead_epoch": 0 2024-09-18T00:19:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": 5, 2024-09-18T00:19:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "down_stamp": "0.000000", 2024-09-18T00:19:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "laggy_probability": 0, 2024-09-18T00:19:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "laggy_interval": 0, 2024-09-18T00:19:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "features": 4540701547738038271, 2024-09-18T00:19:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "old_weight": 0, 2024-09-18T00:19:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "last_purged_snaps_scrub": "2024-09-18T00:17:44.805108+0000", 2024-09-18T00:19:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: "dead_epoch": 56 2024-09-18T00:19:34.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": 6, 2024-09-18T00:19:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "down_stamp": "0.000000", 2024-09-18T00:19:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "laggy_probability": 0, 2024-09-18T00:19:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "laggy_interval": 0, 2024-09-18T00:19:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "features": 4540701547738038271, 2024-09-18T00:19:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "old_weight": 65536, 2024-09-18T00:19:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "last_purged_snaps_scrub": "2024-09-18T00:17:44.484317+0000", 2024-09-18T00:19:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "dead_epoch": 0 2024-09-18T00:19:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": 7, 2024-09-18T00:19:34.936 INFO:teuthology.orchestra.run.smithi097.stdout: "down_stamp": "0.000000", 2024-09-18T00:19:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "laggy_probability": 0, 2024-09-18T00:19:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "laggy_interval": 0, 2024-09-18T00:19:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "features": 4540701547738038271, 2024-09-18T00:19:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "old_weight": 0, 2024-09-18T00:19:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "last_purged_snaps_scrub": "2024-09-18T00:17:44.741853+0000", 2024-09-18T00:19:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "dead_epoch": 0 2024-09-18T00:19:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-09-18T00:19:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_upmap": [], 2024-09-18T00:19:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_upmap_items": [], 2024-09-18T00:19:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_upmap_primaries": [], 2024-09-18T00:19:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_temp": [], 2024-09-18T00:19:34.937 INFO:teuthology.orchestra.run.smithi097.stdout: "primary_temp": [], 2024-09-18T00:19:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklist": {}, 2024-09-18T00:19:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "range_blocklist": {}, 2024-09-18T00:19:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "erasure_code_profiles": { 2024-09-18T00:19:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "default": { 2024-09-18T00:19:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "crush-failure-domain": "osd", 2024-09-18T00:19:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "k": "2", 2024-09-18T00:19:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "m": "1", 2024-09-18T00:19:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "plugin": "jerasure", 2024-09-18T00:19:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "technique": "reed_sol_van" 2024-09-18T00:19:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "removed_snaps_queue": [], 2024-09-18T00:19:34.938 INFO:teuthology.orchestra.run.smithi097.stdout: "new_removed_snaps": [], 2024-09-18T00:19:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "new_purged_snaps": [], 2024-09-18T00:19:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_node_flags": {}, 2024-09-18T00:19:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "device_class_flags": {}, 2024-09-18T00:19:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "stretch_mode": { 2024-09-18T00:19:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "stretch_mode_enabled": false, 2024-09-18T00:19:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "stretch_bucket_count": 0, 2024-09-18T00:19:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "degraded_stretch_mode": 0, 2024-09-18T00:19:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "recovering_stretch_mode": 0, 2024-09-18T00:19:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: "stretch_mode_bucket": 0 2024-09-18T00:19:34.939 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:34.939 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:34.940 INFO:tasks.thrashosds.thrasher:j is {'epoch': 93, 'fsid': 'b993f56a-c4ef-4602-bb41-f32ae406b45a', 'created': '2024-09-18T00:17:35.658788+0000', 'modified': '2024-09-18T00:19:34.483118+0000', 'last_up_change': '2024-09-18T00:19:14.285211+0000', 'last_in_change': '2024-09-18T00:19:23.376648+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T00:17:50.375367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T00:17:53.896718+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T00:18:03.715378+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 92, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 0.75, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'bfda7950-3d83-435d-8313-2f14345096ec', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6800', 'nonce': 40468291}, {'type': 'v1', 'addr': '172.21.15.97:6801', 'nonce': 40468291}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6802', 'nonce': 40468291}, {'type': 'v1', 'addr': '172.21.15.97:6803', 'nonce': 40468291}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6806', 'nonce': 40468291}, {'type': 'v1', 'addr': '172.21.15.97:6807', 'nonce': 40468291}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6804', 'nonce': 40468291}, {'type': 'v1', 'addr': '172.21.15.97:6805', 'nonce': 40468291}]}, 'public_addr': '172.21.15.97:6801/40468291', 'cluster_addr': '172.21.15.97:6803/40468291', 'heartbeat_back_addr': '172.21.15.97:6807/40468291', 'heartbeat_front_addr': '172.21.15.97:6805/40468291', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '35949061-aa30-4aff-b39d-90aebeffb115', 'up': 0, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 27, 'down_at': 46, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6824', 'nonce': 2443452537}, {'type': 'v1', 'addr': '172.21.15.97:6825', 'nonce': 2443452537}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6826', 'nonce': 2443452537}, {'type': 'v1', 'addr': '172.21.15.97:6827', 'nonce': 2443452537}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6830', 'nonce': 2443452537}, {'type': 'v1', 'addr': '172.21.15.97:6831', 'nonce': 2443452537}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6828', 'nonce': 2443452537}, {'type': 'v1', 'addr': '172.21.15.97:6829', 'nonce': 2443452537}]}, 'public_addr': '172.21.15.97:6825/2443452537', 'cluster_addr': '172.21.15.97:6827/2443452537', 'heartbeat_back_addr': '172.21.15.97:6831/2443452537', 'heartbeat_front_addr': '172.21.15.97:6829/2443452537', 'state': ['exists']}, {'osd': 2, 'uuid': '31b3e1c1-7648-44fe-9931-758ffe9a53ca', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6816', 'nonce': 2425060925}, {'type': 'v1', 'addr': '172.21.15.97:6817', 'nonce': 2425060925}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6818', 'nonce': 2425060925}, {'type': 'v1', 'addr': '172.21.15.97:6819', 'nonce': 2425060925}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6822', 'nonce': 2425060925}, {'type': 'v1', 'addr': '172.21.15.97:6823', 'nonce': 2425060925}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6820', 'nonce': 2425060925}, {'type': 'v1', 'addr': '172.21.15.97:6821', 'nonce': 2425060925}]}, 'public_addr': '172.21.15.97:6817/2425060925', 'cluster_addr': '172.21.15.97:6819/2425060925', 'heartbeat_back_addr': '172.21.15.97:6823/2425060925', 'heartbeat_front_addr': '172.21.15.97:6821/2425060925', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd7daebbc-0e33-42ab-97ee-a6c1b449ae8d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6808', 'nonce': 3512666164}, {'type': 'v1', 'addr': '172.21.15.97:6809', 'nonce': 3512666164}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6810', 'nonce': 3512666164}, {'type': 'v1', 'addr': '172.21.15.97:6811', 'nonce': 3512666164}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6814', 'nonce': 3512666164}, {'type': 'v1', 'addr': '172.21.15.97:6815', 'nonce': 3512666164}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6812', 'nonce': 3512666164}, {'type': 'v1', 'addr': '172.21.15.97:6813', 'nonce': 3512666164}]}, 'public_addr': '172.21.15.97:6809/3512666164', 'cluster_addr': '172.21.15.97:6811/3512666164', 'heartbeat_back_addr': '172.21.15.97:6815/3512666164', 'heartbeat_front_addr': '172.21.15.97:6813/3512666164', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'd55ca69b-5000-45ff-acc4-00f7c4763cad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6802', 'nonce': 3191744356}, {'type': 'v1', 'addr': '172.21.15.114:6804', 'nonce': 3191744356}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6806', 'nonce': 3191744356}, {'type': 'v1', 'addr': '172.21.15.114:6808', 'nonce': 3191744356}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6814', 'nonce': 3191744356}, {'type': 'v1', 'addr': '172.21.15.114:6815', 'nonce': 3191744356}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6810', 'nonce': 3191744356}, {'type': 'v1', 'addr': '172.21.15.114:6812', 'nonce': 3191744356}]}, 'public_addr': '172.21.15.114:6804/3191744356', 'cluster_addr': '172.21.15.114:6808/3191744356', 'heartbeat_back_addr': '172.21.15.114:6815/3191744356', 'heartbeat_front_addr': '172.21.15.114:6812/3191744356', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '196a329b-bd93-4cc2-85bb-64ed81d3918b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 55, 'up_from': 76, 'up_thru': 84, 'down_at': 56, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6800', 'nonce': 407304039}, {'type': 'v1', 'addr': '172.21.15.114:6801', 'nonce': 407304039}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6803', 'nonce': 407304039}, {'type': 'v1', 'addr': '172.21.15.114:6805', 'nonce': 407304039}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6811', 'nonce': 407304039}, {'type': 'v1', 'addr': '172.21.15.114:6813', 'nonce': 407304039}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6807', 'nonce': 407304039}, {'type': 'v1', 'addr': '172.21.15.114:6809', 'nonce': 407304039}]}, 'public_addr': '172.21.15.114:6801/407304039', 'cluster_addr': '172.21.15.114:6805/407304039', 'heartbeat_back_addr': '172.21.15.114:6813/407304039', 'heartbeat_front_addr': '172.21.15.114:6809/407304039', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'dc9712f0-8858-467a-8397-d5eb97187165', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6824', 'nonce': 1148348470}, {'type': 'v1', 'addr': '172.21.15.114:6825', 'nonce': 1148348470}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6826', 'nonce': 1148348470}, {'type': 'v1', 'addr': '172.21.15.114:6827', 'nonce': 1148348470}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6830', 'nonce': 1148348470}, {'type': 'v1', 'addr': '172.21.15.114:6831', 'nonce': 1148348470}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6828', 'nonce': 1148348470}, {'type': 'v1', 'addr': '172.21.15.114:6829', 'nonce': 1148348470}]}, 'public_addr': '172.21.15.114:6825/1148348470', 'cluster_addr': '172.21.15.114:6827/1148348470', 'heartbeat_back_addr': '172.21.15.114:6831/1148348470', 'heartbeat_front_addr': '172.21.15.114:6829/1148348470', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'b6d774ee-fe6d-462c-afa4-7ffd5d7343a9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6816', 'nonce': 2133497352}, {'type': 'v1', 'addr': '172.21.15.114:6817', 'nonce': 2133497352}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6818', 'nonce': 2133497352}, {'type': 'v1', 'addr': '172.21.15.114:6819', 'nonce': 2133497352}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6822', 'nonce': 2133497352}, {'type': 'v1', 'addr': '172.21.15.114:6823', 'nonce': 2133497352}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6820', 'nonce': 2133497352}, {'type': 'v1', 'addr': '172.21.15.114:6821', 'nonce': 2133497352}]}, 'public_addr': '172.21.15.114:6817/2133497352', 'cluster_addr': '172.21.15.114:6819/2133497352', 'heartbeat_back_addr': '172.21.15.114:6823/2133497352', 'heartbeat_front_addr': '172.21.15.114:6821/2133497352', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:17:44.237241+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:17:44.504107+0000', 'dead_epoch': 46}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:17:44.690795+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:17:44.197056+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:17:44.459497+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:17:44.805108+0000', 'dead_epoch': 56}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T00:17:44.484317+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T00:17:44.741853+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T00:19:34.941 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:19:34.985 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:34.985+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:35.064 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:35.061+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:35.072 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:35.072 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:35.073 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:35.073 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:35.088 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:19:35.174 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:35.173+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:35.250 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:19:35.251 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-18T00:19:35.256 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:35.256 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:35.256 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:35.256 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:35.256 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:35.265 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:35.265+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:35.269 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-18T00:19:34.486956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":637876211,"num_objects":871,"num_object_clones":127,"num_object_copies":1742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":871,"num_whiteouts":164,"num_read":83988,"num_read_kb":258812,"num_write":16993,"num_write_kb":177567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":939,"num_bytes_recovered":21783373,"num_keys_recovered":189,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13606,"ondisk_log_size":13606,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":271980,"kb_used_data":33800,"kb_used_omap":16,"kb_used_meta":237999,"kb_avail":565959060,"statfs":{"total":579820584960,"available":579542077440,"internally_reserved":0,"allocated":34611200,"data_stored":179909218,"data_compressed":12326904,"data_compressed_allocated":20385792,"data_compressed_original":172194132,"omap_allocated":17314,"internal_metadata":243711070},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30835380,"num_objects":6,"num_object_clones":12,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":12,"num_read":8861,"num_read_kb":40526,"num_write":1465,"num_write_kb":23015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":298,"num_bytes_recovered":7201624,"num_keys_recovered":25,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.161706"},"pg_stats":[{"pgid":"3.1f","version":"89'638","reported_seq":2118,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T00:19:33.490391+0000","last_change":"2024-09-18T00:19:18.330008+0000","last_active":"2024-09-18T00:19:33.490391+0000","last_peered":"2024-09-18T00:19:33.490391+0000","last_clean":"2024-09-18T00:19:33.490391+0000","last_became_active":"2024-09-18T00:19:18.327350+0000","last_became_peered":"2024-09-18T00:19:18.327350+0000","last_unstale":"2024-09-18T00:19:33.490391+0000","last_undegraded":"2024-09-18T00:19:33.490391+0000","last_fullsized":"2024-09-18T00:19:33.490391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:32.135339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001658056,"stat_sum":{"num_bytes":16876151,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":5,"num_read":3279,"num_read_kb":12798,"num_write":899,"num_write_kb":7018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":2252743,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"92'387","reported_seq":1032,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T00:19:34.319660+0000","last_change":"2024-09-18T00:19:05.841262+0000","last_active":"2024-09-18T00:19:34.319660+0000","last_peered":"2024-09-18T00:19:34.319660+0000","last_clean":"2024-09-18T00:19:34.319660+0000","last_became_active":"2024-09-18T00:19:05.838946+0000","last_became_peered":"2024-09-18T00:19:05.838946+0000","last_unstale":"2024-09-18T00:19:34.319660+0000","last_undegraded":"2024-09-18T00:19:34.319660+0000","last_fullsized":"2024-09-18T00:19:34.319660+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:43.141603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019357549999999999,"stat_sum":{"num_bytes":6441278,"num_objects":23,"num_object_clones":2,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":5,"num_read":1210,"num_read_kb":1705,"num_write":449,"num_write_kb":2784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":46741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"89'712","reported_seq":1207,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-18T00:19:31.464125+0000","last_change":"2024-09-18T00:19:18.336992+0000","last_active":"2024-09-18T00:19:31.464125+0000","last_peered":"2024-09-18T00:19:31.464125+0000","last_clean":"2024-09-18T00:19:31.464125+0000","last_became_active":"2024-09-18T00:19:18.332394+0000","last_became_peered":"2024-09-18T00:19:18.332394+0000","last_unstale":"2024-09-18T00:19:31.464125+0000","last_undegraded":"2024-09-18T00:19:31.464125+0000","last_fullsized":"2024-09-18T00:19:31.464125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:29.569108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117322,"stat_sum":{"num_bytes":23239637,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":3117,"num_read_kb":4775,"num_write":1120,"num_write_kb":4172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":510608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"91'346","reported_seq":1321,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-18T00:19:32.922567+0000","last_change":"2024-09-18T00:19:15.380185+0000","last_active":"2024-09-18T00:19:32.922567+0000","last_peered":"2024-09-18T00:19:32.922567+0000","last_clean":"2024-09-18T00:19:32.922567+0000","last_became_active":"2024-09-18T00:19:15.320316+0000","last_became_peered":"2024-09-18T00:19:15.320316+0000","last_unstale":"2024-09-18T00:19:32.922567+0000","last_undegraded":"2024-09-18T00:19:32.922567+0000","last_fullsized":"2024-09-18T00:19:32.922567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:13:04.557441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055723800000000005,"stat_sum":{"num_bytes":9809671,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":1,"num_read":2888,"num_read_kb":9409,"num_write":348,"num_write_kb":3649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":72221,"num_keys_recovered":54,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"89'236","reported_seq":1265,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T00:19:33.490831+0000","last_change":"2024-09-18T00:18:07.409072+0000","last_active":"2024-09-18T00:19:33.490831+0000","last_peered":"2024-09-18T00:19:33.490831+0000","last_clean":"2024-09-18T00:19:33.490831+0000","last_became_active":"2024-09-18T00:18:05.415810+0000","last_became_peered":"2024-09-18T00:18:05.415810+0000","last_unstale":"2024-09-18T00:19:33.490831+0000","last_undegraded":"2024-09-18T00:19:33.490831+0000","last_fullsized":"2024-09-18T00:19:33.490831+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:54:00.226837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059152399999999996,"stat_sum":{"num_bytes":6653227,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":4,"num_read":1321,"num_read_kb":2495,"num_write":241,"num_write_kb":3656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"89'181","reported_seq":2423,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:30.341698+0000","last_change":"2024-09-18T00:19:27.424539+0000","last_active":"2024-09-18T00:19:30.341698+0000","last_peered":"2024-09-18T00:19:30.341698+0000","last_clean":"2024-09-18T00:19:30.341698+0000","last_became_active":"2024-09-18T00:19:27.423029+0000","last_became_peered":"2024-09-18T00:19:27.423029+0000","last_unstale":"2024-09-18T00:19:30.341698+0000","last_undegraded":"2024-09-18T00:19:30.341698+0000","last_fullsized":"2024-09-18T00:19:30.341698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:58:32.265603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020525439999999999,"stat_sum":{"num_bytes":27686084,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":6,"num_read":1937,"num_read_kb":4137,"num_write":178,"num_write_kb":1959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":420783,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"92'302","reported_seq":1084,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T00:19:33.738117+0000","last_change":"2024-09-18T00:19:27.427821+0000","last_active":"2024-09-18T00:19:33.738117+0000","last_peered":"2024-09-18T00:19:33.738117+0000","last_clean":"2024-09-18T00:19:33.738117+0000","last_became_active":"2024-09-18T00:19:27.425348+0000","last_became_peered":"2024-09-18T00:19:27.425348+0000","last_unstale":"2024-09-18T00:19:33.738117+0000","last_undegraded":"2024-09-18T00:19:33.738117+0000","last_fullsized":"2024-09-18T00:19:33.738117+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:31:08.988502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077375,"stat_sum":{"num_bytes":9150992,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":5,"num_read":913,"num_read_kb":3951,"num_write":326,"num_write_kb":2264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":933934,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"91'248","reported_seq":3065,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T00:19:33.499287+0000","last_change":"2024-09-18T00:18:07.500927+0000","last_active":"2024-09-18T00:19:33.499287+0000","last_peered":"2024-09-18T00:19:33.499287+0000","last_clean":"2024-09-18T00:19:33.499287+0000","last_became_active":"2024-09-18T00:18:05.406821+0000","last_became_peered":"2024-09-18T00:18:05.406821+0000","last_unstale":"2024-09-18T00:19:33.499287+0000","last_undegraded":"2024-09-18T00:19:33.499287+0000","last_fullsized":"2024-09-18T00:19:33.499287+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:24:26.925008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074352700000000001,"stat_sum":{"num_bytes":53780385,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":6,"num_read":2748,"num_read_kb":20318,"num_write":234,"num_write_kb":8126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"92'326","reported_seq":979,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T00:19:34.322540+0000","last_change":"2024-09-18T00:19:18.330061+0000","last_active":"2024-09-18T00:19:34.322540+0000","last_peered":"2024-09-18T00:19:34.322540+0000","last_clean":"2024-09-18T00:19:34.322540+0000","last_became_active":"2024-09-18T00:19:18.328043+0000","last_became_peered":"2024-09-18T00:19:18.328043+0000","last_unstale":"2024-09-18T00:19:34.322540+0000","last_undegraded":"2024-09-18T00:19:34.322540+0000","last_fullsized":"2024-09-18T00:19:34.322540+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:29:04.794845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012546969999999999,"stat_sum":{"num_bytes":14379162,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":9,"num_read":1047,"num_read_kb":5991,"num_write":295,"num_write_kb":4809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":491875,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"90'612","reported_seq":2022,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-18T00:19:31.464300+0000","last_change":"2024-09-18T00:19:18.332702+0000","last_active":"2024-09-18T00:19:31.464300+0000","last_peered":"2024-09-18T00:19:31.464300+0000","last_clean":"2024-09-18T00:19:31.464300+0000","last_became_active":"2024-09-18T00:19:18.330981+0000","last_became_peered":"2024-09-18T00:19:18.330981+0000","last_unstale":"2024-09-18T00:19:31.464300+0000","last_undegraded":"2024-09-18T00:19:31.464300+0000","last_fullsized":"2024-09-18T00:19:31.464300+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:34:08.235969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015027420000000001,"stat_sum":{"num_bytes":14988382,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":5,"num_read":4114,"num_read_kb":5656,"num_write":870,"num_write_kb":5677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1385456,"num_keys_recovered":29,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'300","reported_seq":544,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:29.705526+0000","last_change":"2024-09-18T00:18:16.491563+0000","last_active":"2024-09-18T00:19:29.705526+0000","last_peered":"2024-09-18T00:19:29.705526+0000","last_clean":"2024-09-18T00:19:29.705526+0000","last_became_active":"2024-09-18T00:18:16.438164+0000","last_became_peered":"2024-09-18T00:18:16.438164+0000","last_unstale":"2024-09-18T00:19:29.705526+0000","last_undegraded":"2024-09-18T00:19:29.705526+0000","last_fullsized":"2024-09-18T00:19:29.705526+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:43:53.711765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12003964,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":7,"num_read":390,"num_read_kb":647,"num_write":303,"num_write_kb":3714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"89'430","reported_seq":1173,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:30.262740+0000","last_change":"2024-09-18T00:18:16.544097+0000","last_active":"2024-09-18T00:19:30.262740+0000","last_peered":"2024-09-18T00:19:30.262740+0000","last_clean":"2024-09-18T00:19:30.262740+0000","last_became_active":"2024-09-18T00:18:16.441758+0000","last_became_peered":"2024-09-18T00:18:16.441758+0000","last_unstale":"2024-09-18T00:19:30.262740+0000","last_undegraded":"2024-09-18T00:19:30.262740+0000","last_fullsized":"2024-09-18T00:19:30.262740+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:47:29.941986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000991729,"stat_sum":{"num_bytes":9815973,"num_objects":26,"num_object_clones":1,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":4,"num_read":2132,"num_read_kb":2921,"num_write":471,"num_write_kb":4388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":103027,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:28.464783+0000","last_change":"2024-09-18T00:19:02.801535+0000","last_active":"2024-09-18T00:19:28.464783+0000","last_peered":"2024-09-18T00:19:28.464783+0000","last_clean":"2024-09-18T00:19:28.464783+0000","last_became_active":"2024-09-18T00:19:02.801178+0000","last_became_peered":"2024-09-18T00:19:02.801178+0000","last_unstale":"2024-09-18T00:19:28.464783+0000","last_undegraded":"2024-09-18T00:19:28.464783+0000","last_fullsized":"2024-09-18T00:19:28.464783+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:06:24.440364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089554000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"92'249","reported_seq":1760,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T00:19:34.060926+0000","last_change":"2024-09-18T00:19:27.439082+0000","last_active":"2024-09-18T00:19:34.060926+0000","last_peered":"2024-09-18T00:19:34.060926+0000","last_clean":"2024-09-18T00:19:34.060926+0000","last_became_active":"2024-09-18T00:19:27.430228+0000","last_became_peered":"2024-09-18T00:19:27.430228+0000","last_unstale":"2024-09-18T00:19:34.060926+0000","last_undegraded":"2024-09-18T00:19:34.060926+0000","last_fullsized":"2024-09-18T00:19:34.060926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:45:13.130780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010971830000000001,"stat_sum":{"num_bytes":21660034,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":6,"num_read":1727,"num_read_kb":5003,"num_write":271,"num_write_kb":5710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1054709,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:28.473548+0000","last_change":"2024-09-18T00:19:25.406762+0000","last_active":"2024-09-18T00:19:28.473548+0000","last_peered":"2024-09-18T00:19:28.473548+0000","last_clean":"2024-09-18T00:19:28.473548+0000","last_became_active":"2024-09-18T00:19:25.405458+0000","last_became_peered":"2024-09-18T00:19:25.405458+0000","last_unstale":"2024-09-18T00:19:28.473548+0000","last_undegraded":"2024-09-18T00:19:28.473548+0000","last_fullsized":"2024-09-18T00:19:28.473548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:55:54.576260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"92'601","reported_seq":2798,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T00:19:33.500142+0000","last_change":"2024-09-18T00:18:16.504734+0000","last_active":"2024-09-18T00:19:33.500142+0000","last_peered":"2024-09-18T00:19:33.500142+0000","last_clean":"2024-09-18T00:19:33.500142+0000","last_became_active":"2024-09-18T00:18:16.440536+0000","last_became_peered":"2024-09-18T00:18:16.440536+0000","last_unstale":"2024-09-18T00:19:33.500142+0000","last_undegraded":"2024-09-18T00:19:33.500142+0000","last_fullsized":"2024-09-18T00:19:33.500142+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:24:32.658924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066608800000000001,"stat_sum":{"num_bytes":39666843,"num_objects":53,"num_object_clones":11,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":13,"num_read":3824,"num_read_kb":13540,"num_write":871,"num_write_kb":9586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":73512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":122,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:28.464807+0000","last_change":"2024-09-18T00:18:16.439964+0000","last_active":"2024-09-18T00:19:28.464807+0000","last_peered":"2024-09-18T00:19:28.464807+0000","last_clean":"2024-09-18T00:19:28.464807+0000","last_became_active":"2024-09-18T00:18:16.437150+0000","last_became_peered":"2024-09-18T00:18:16.437150+0000","last_unstale":"2024-09-18T00:19:28.464807+0000","last_undegraded":"2024-09-18T00:19:28.464807+0000","last_fullsized":"2024-09-18T00:19:28.464807+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:25.772294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"89'431","reported_seq":2003,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:29.344971+0000","last_change":"2024-09-18T00:19:27.427585+0000","last_active":"2024-09-18T00:19:29.344971+0000","last_peered":"2024-09-18T00:19:29.344971+0000","last_clean":"2024-09-18T00:19:29.344971+0000","last_became_active":"2024-09-18T00:19:27.425218+0000","last_became_peered":"2024-09-18T00:19:27.425218+0000","last_unstale":"2024-09-18T00:19:29.344971+0000","last_undegraded":"2024-09-18T00:19:29.344971+0000","last_fullsized":"2024-09-18T00:19:29.344971+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:21:12.522710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015579680000000001,"stat_sum":{"num_bytes":27139906,"num_objects":31,"num_object_clones":3,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":5,"num_read":2244,"num_read_kb":4770,"num_write":374,"num_write_kb":4844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":1345766,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T00:19:33.490068+0000","last_change":"2024-09-18T00:19:25.430613+0000","last_active":"2024-09-18T00:19:33.490068+0000","last_peered":"2024-09-18T00:19:33.490068+0000","last_clean":"2024-09-18T00:19:33.490068+0000","last_became_active":"2024-09-18T00:19:25.430109+0000","last_became_peered":"2024-09-18T00:19:25.430109+0000","last_unstale":"2024-09-18T00:19:33.490068+0000","last_undegraded":"2024-09-18T00:19:33.490068+0000","last_fullsized":"2024-09-18T00:19:33.490068+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:23:35.398358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"91'311","reported_seq":2412,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-18T00:19:32.635665+0000","last_change":"2024-09-18T00:19:18.332377+0000","last_active":"2024-09-18T00:19:32.635665+0000","last_peered":"2024-09-18T00:19:32.635665+0000","last_clean":"2024-09-18T00:19:32.635665+0000","last_became_active":"2024-09-18T00:19:18.328924+0000","last_became_peered":"2024-09-18T00:19:18.328924+0000","last_unstale":"2024-09-18T00:19:32.635665+0000","last_undegraded":"2024-09-18T00:19:32.635665+0000","last_fullsized":"2024-09-18T00:19:32.635665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:02:37.336124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011486619999999999,"stat_sum":{"num_bytes":25058234,"num_objects":31,"num_object_clones":4,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":3,"num_read":2154,"num_read_kb":12251,"num_write":321,"num_write_kb":5298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":675014,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"89'318","reported_seq":1500,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:29.946519+0000","last_change":"2024-09-18T00:18:07.398743+0000","last_active":"2024-09-18T00:19:29.946519+0000","last_peered":"2024-09-18T00:19:29.946519+0000","last_clean":"2024-09-18T00:19:29.946519+0000","last_became_active":"2024-09-18T00:18:05.407889+0000","last_became_peered":"2024-09-18T00:18:05.407889+0000","last_unstale":"2024-09-18T00:19:29.946519+0000","last_undegraded":"2024-09-18T00:19:29.946519+0000","last_fullsized":"2024-09-18T00:19:29.946519+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:20.367096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097981000000000001,"stat_sum":{"num_bytes":21479493,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":1414,"num_read_kb":5084,"num_write":290,"num_write_kb":5180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":121,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-18T00:19:31.464100+0000","last_change":"2024-09-18T00:19:24.410776+0000","last_active":"2024-09-18T00:19:31.464100+0000","last_peered":"2024-09-18T00:19:31.464100+0000","last_clean":"2024-09-18T00:19:31.464100+0000","last_became_active":"2024-09-18T00:19:24.405834+0000","last_became_peered":"2024-09-18T00:19:24.405834+0000","last_unstale":"2024-09-18T00:19:31.464100+0000","last_undegraded":"2024-09-18T00:19:31.464100+0000","last_fullsized":"2024-09-18T00:19:31.464100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:22:09.235337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088189300000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"89'325","reported_seq":782,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:30.169128+0000","last_change":"2024-09-18T00:19:27.431511+0000","last_active":"2024-09-18T00:19:30.169128+0000","last_peered":"2024-09-18T00:19:30.169128+0000","last_clean":"2024-09-18T00:19:30.169128+0000","last_became_active":"2024-09-18T00:19:27.429291+0000","last_became_peered":"2024-09-18T00:19:27.429291+0000","last_unstale":"2024-09-18T00:19:30.169128+0000","last_undegraded":"2024-09-18T00:19:30.169128+0000","last_fullsized":"2024-09-18T00:19:30.169128+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:59:59.319718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101741,"stat_sum":{"num_bytes":5352008,"num_objects":23,"num_object_clones":1,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":701,"num_read_kb":1853,"num_write":403,"num_write_kb":2332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":1002237,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":153,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:28.446265+0000","last_change":"2024-09-18T00:17:57.477633+0000","last_active":"2024-09-18T00:19:28.446265+0000","last_peered":"2024-09-18T00:19:28.446265+0000","last_clean":"2024-09-18T00:19:28.446265+0000","last_became_active":"2024-09-18T00:17:55.631277+0000","last_became_peered":"2024-09-18T00:17:55.631277+0000","last_unstale":"2024-09-18T00:19:28.446265+0000","last_undegraded":"2024-09-18T00:19:28.446265+0000","last_fullsized":"2024-09-18T00:19:28.446265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:55.421684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001141204,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"92'433","reported_seq":2151,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T00:19:34.321217+0000","last_change":"2024-09-18T00:18:07.408962+0000","last_active":"2024-09-18T00:19:34.321217+0000","last_peered":"2024-09-18T00:19:34.321217+0000","last_clean":"2024-09-18T00:19:34.321217+0000","last_became_active":"2024-09-18T00:18:05.410952+0000","last_became_peered":"2024-09-18T00:18:05.410952+0000","last_unstale":"2024-09-18T00:19:34.321217+0000","last_undegraded":"2024-09-18T00:19:34.321217+0000","last_fullsized":"2024-09-18T00:19:34.321217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:09:28.052285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057326299999999996,"stat_sum":{"num_bytes":18285665,"num_objects":33,"num_object_clones":3,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":5,"num_read":2957,"num_read_kb":9787,"num_write":538,"num_write_kb":8623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":129,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:28.464598+0000","last_change":"2024-09-18T00:19:24.407788+0000","last_active":"2024-09-18T00:19:28.464598+0000","last_peered":"2024-09-18T00:19:28.464598+0000","last_clean":"2024-09-18T00:19:28.464598+0000","last_became_active":"2024-09-18T00:19:24.407417+0000","last_became_peered":"2024-09-18T00:19:24.407417+0000","last_unstale":"2024-09-18T00:19:28.464598+0000","last_undegraded":"2024-09-18T00:19:28.464598+0000","last_fullsized":"2024-09-18T00:19:28.464598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:19:51.083231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'432","reported_seq":2078,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T00:19:34.187568+0000","last_change":"2024-09-18T00:19:18.332219+0000","last_active":"2024-09-18T00:19:34.187568+0000","last_peered":"2024-09-18T00:19:34.187568+0000","last_clean":"2024-09-18T00:19:34.187568+0000","last_became_active":"2024-09-18T00:19:18.330155+0000","last_became_peered":"2024-09-18T00:19:18.330155+0000","last_unstale":"2024-09-18T00:19:34.187568+0000","last_undegraded":"2024-09-18T00:19:34.187568+0000","last_fullsized":"2024-09-18T00:19:34.187568+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:31:06.400323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010779679999999999,"stat_sum":{"num_bytes":21399732,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":4,"num_read":2286,"num_read_kb":7536,"num_write":424,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":493176,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":65,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-18T00:19:31.464043+0000","last_change":"2024-09-18T00:19:15.316359+0000","last_active":"2024-09-18T00:19:31.464043+0000","last_peered":"2024-09-18T00:19:31.464043+0000","last_clean":"2024-09-18T00:19:31.464043+0000","last_became_active":"2024-09-18T00:19:15.313552+0000","last_became_peered":"2024-09-18T00:19:15.313552+0000","last_unstale":"2024-09-18T00:19:31.464043+0000","last_undegraded":"2024-09-18T00:19:31.464043+0000","last_fullsized":"2024-09-18T00:19:31.464043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:09:31.101299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":303,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:28.464848+0000","last_change":"2024-09-18T00:18:19.466612+0000","last_active":"2024-09-18T00:19:28.464848+0000","last_peered":"2024-09-18T00:19:28.464848+0000","last_clean":"2024-09-18T00:19:28.464848+0000","last_became_active":"2024-09-18T00:18:19.465557+0000","last_became_peered":"2024-09-18T00:18:19.465557+0000","last_unstale":"2024-09-18T00:19:28.464848+0000","last_undegraded":"2024-09-18T00:19:28.464848+0000","last_fullsized":"2024-09-18T00:19:28.464848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_clean_scrub_stamp":"2024-09-18T00:17:50.414591+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:08:40.389972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"90'575","reported_seq":1703,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T00:19:33.499381+0000","last_change":"2024-09-18T00:19:27.430847+0000","last_active":"2024-09-18T00:19:33.499381+0000","last_peered":"2024-09-18T00:19:33.499381+0000","last_clean":"2024-09-18T00:19:33.499381+0000","last_became_active":"2024-09-18T00:19:27.428409+0000","last_became_peered":"2024-09-18T00:19:27.428409+0000","last_unstale":"2024-09-18T00:19:33.499381+0000","last_undegraded":"2024-09-18T00:19:33.499381+0000","last_fullsized":"2024-09-18T00:19:33.499381+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:07:04.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001352535,"stat_sum":{"num_bytes":7267513,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":4,"num_read":3030,"num_read_kb":6361,"num_write":796,"num_write_kb":5489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":1056031,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"89'295","reported_seq":1534,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:30.356612+0000","last_change":"2024-09-18T00:18:07.398132+0000","last_active":"2024-09-18T00:19:30.356612+0000","last_peered":"2024-09-18T00:19:30.356612+0000","last_clean":"2024-09-18T00:19:30.356612+0000","last_became_active":"2024-09-18T00:18:05.401097+0000","last_became_peered":"2024-09-18T00:18:05.401097+0000","last_unstale":"2024-09-18T00:19:30.356612+0000","last_undegraded":"2024-09-18T00:19:30.356612+0000","last_fullsized":"2024-09-18T00:19:30.356612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:06:19.421973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061791899999999998,"stat_sum":{"num_bytes":9844072,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":1,"num_read":2519,"num_read_kb":8646,"num_write":301,"num_write_kb":5881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'253","reported_seq":1090,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:30.054727+0000","last_change":"2024-09-18T00:19:27.425674+0000","last_active":"2024-09-18T00:19:30.054727+0000","last_peered":"2024-09-18T00:19:30.054727+0000","last_clean":"2024-09-18T00:19:30.054727+0000","last_became_active":"2024-09-18T00:19:27.424630+0000","last_became_peered":"2024-09-18T00:19:27.424630+0000","last_unstale":"2024-09-18T00:19:30.054727+0000","last_undegraded":"2024-09-18T00:19:30.054727+0000","last_fullsized":"2024-09-18T00:19:30.054727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:39:49.800597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010106729999999999,"stat_sum":{"num_bytes":15813020,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":7,"num_read":687,"num_read_kb":5401,"num_write":244,"num_write_kb":8145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":4432838,"num_keys_recovered":14,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"92'369","reported_seq":689,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T00:19:34.228902+0000","last_change":"2024-09-18T00:18:07.502018+0000","last_active":"2024-09-18T00:19:34.228902+0000","last_peered":"2024-09-18T00:19:34.228902+0000","last_clean":"2024-09-18T00:19:34.228902+0000","last_became_active":"2024-09-18T00:18:05.406977+0000","last_became_peered":"2024-09-18T00:18:05.406977+0000","last_unstale":"2024-09-18T00:19:34.228902+0000","last_undegraded":"2024-09-18T00:19:34.228902+0000","last_fullsized":"2024-09-18T00:19:34.228902+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:02.587338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001533539,"stat_sum":{"num_bytes":12552160,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":2,"num_read":1024,"num_read_kb":3002,"num_write":576,"num_write_kb":6190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"91'1325","reported_seq":3089,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-18T00:19:33.032285+0000","last_change":"2024-09-18T00:19:19.361838+0000","last_active":"2024-09-18T00:19:33.032285+0000","last_peered":"2024-09-18T00:19:33.032285+0000","last_clean":"2024-09-18T00:19:33.032285+0000","last_became_active":"2024-09-18T00:19:19.358241+0000","last_became_peered":"2024-09-18T00:19:19.358241+0000","last_unstale":"2024-09-18T00:19:33.032285+0000","last_undegraded":"2024-09-18T00:19:33.032285+0000","last_fullsized":"2024-09-18T00:19:33.032285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":1325,"log_dups_size":0,"ondisk_log_size":1325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:44:29.731918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073075799999999997,"stat_sum":{"num_bytes":10303528,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":5,"num_read":8831,"num_read_kb":8844,"num_write":2102,"num_write_kb":6708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":392734,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"89'651","reported_seq":2428,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:30.326882+0000","last_change":"2024-09-18T00:19:18.337502+0000","last_active":"2024-09-18T00:19:30.326882+0000","last_peered":"2024-09-18T00:19:30.326882+0000","last_clean":"2024-09-18T00:19:30.326882+0000","last_became_active":"2024-09-18T00:19:18.333730+0000","last_became_peered":"2024-09-18T00:19:18.333730+0000","last_unstale":"2024-09-18T00:19:30.326882+0000","last_undegraded":"2024-09-18T00:19:30.326882+0000","last_fullsized":"2024-09-18T00:19:30.326882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":651,"log_dups_size":0,"ondisk_log_size":651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:41:34.062137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7531919,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":5,"num_read":4962,"num_read_kb":6681,"num_write":888,"num_write_kb":3437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":697208,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"89'305","reported_seq":1084,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:28.939370+0000","last_change":"2024-09-18T00:18:16.512669+0000","last_active":"2024-09-18T00:19:28.939370+0000","last_peered":"2024-09-18T00:19:28.939370+0000","last_clean":"2024-09-18T00:19:28.939370+0000","last_became_active":"2024-09-18T00:18:16.440223+0000","last_became_peered":"2024-09-18T00:18:16.440223+0000","last_unstale":"2024-09-18T00:19:28.939370+0000","last_undegraded":"2024-09-18T00:19:28.939370+0000","last_fullsized":"2024-09-18T00:19:28.939370+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:50:41.667324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34358888,"num_objects":29,"num_object_clones":2,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":3,"num_read":1082,"num_read_kb":6680,"num_write":346,"num_write_kb":4313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":7258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"89'326","reported_seq":3047,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:29.570196+0000","last_change":"2024-09-18T00:18:16.528331+0000","last_active":"2024-09-18T00:19:29.570196+0000","last_peered":"2024-09-18T00:19:29.570196+0000","last_clean":"2024-09-18T00:19:29.570196+0000","last_became_active":"2024-09-18T00:18:16.441020+0000","last_became_peered":"2024-09-18T00:18:16.441020+0000","last_unstale":"2024-09-18T00:19:29.570196+0000","last_undegraded":"2024-09-18T00:19:29.570196+0000","last_fullsized":"2024-09-18T00:19:29.570196+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:42:53.859059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37103629,"num_objects":23,"num_object_clones":2,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":3920,"num_read_kb":10788,"num_write":341,"num_write_kb":8777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"92'329","reported_seq":4398,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T00:19:34.053969+0000","last_change":"2024-09-18T00:19:18.329403+0000","last_active":"2024-09-18T00:19:34.053969+0000","last_peered":"2024-09-18T00:19:34.053969+0000","last_clean":"2024-09-18T00:19:34.053969+0000","last_became_active":"2024-09-18T00:19:18.327346+0000","last_became_peered":"2024-09-18T00:19:18.327346+0000","last_unstale":"2024-09-18T00:19:34.053969+0000","last_undegraded":"2024-09-18T00:19:34.053969+0000","last_fullsized":"2024-09-18T00:19:34.053969+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:57:47.478567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091276599999999997,"stat_sum":{"num_bytes":28950701,"num_objects":27,"num_object_clones":3,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":4,"num_read":4892,"num_read_kb":32567,"num_write":367,"num_write_kb":13831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":3037482,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"89'300","reported_seq":4222,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:29.664908+0000","last_change":"2024-09-18T00:18:16.458821+0000","last_active":"2024-09-18T00:19:29.664908+0000","last_peered":"2024-09-18T00:19:29.664908+0000","last_clean":"2024-09-18T00:19:29.664908+0000","last_became_active":"2024-09-18T00:18:16.438286+0000","last_became_peered":"2024-09-18T00:18:16.438286+0000","last_unstale":"2024-09-18T00:19:29.664908+0000","last_undegraded":"2024-09-18T00:19:29.664908+0000","last_fullsized":"2024-09-18T00:19:29.664908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:57:43.994196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063423600000000004,"stat_sum":{"num_bytes":25782046,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":4,"num_read":4694,"num_read_kb":13833,"num_write":309,"num_write_kb":3770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":70145,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"92'231","reported_seq":1570,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T00:19:34.228201+0000","last_change":"2024-09-18T00:18:16.482158+0000","last_active":"2024-09-18T00:19:34.228201+0000","last_peered":"2024-09-18T00:19:34.228201+0000","last_clean":"2024-09-18T00:19:34.228201+0000","last_became_active":"2024-09-18T00:18:16.438556+0000","last_became_peered":"2024-09-18T00:18:16.438556+0000","last_unstale":"2024-09-18T00:19:34.228201+0000","last_undegraded":"2024-09-18T00:19:34.228201+0000","last_fullsized":"2024-09-18T00:19:34.228201+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:36:34.631105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010146669999999999,"stat_sum":{"num_bytes":17814552,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":9,"num_read":1221,"num_read_kb":8601,"num_write":221,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":47120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"89'709","reported_seq":2810,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-18T00:19:30.257162+0000","last_change":"2024-09-18T00:18:16.524316+0000","last_active":"2024-09-18T00:19:30.257162+0000","last_peered":"2024-09-18T00:19:30.257162+0000","last_clean":"2024-09-18T00:19:30.257162+0000","last_became_active":"2024-09-18T00:18:16.440910+0000","last_became_peered":"2024-09-18T00:18:16.440910+0000","last_unstale":"2024-09-18T00:19:30.257162+0000","last_undegraded":"2024-09-18T00:19:30.257162+0000","last_fullsized":"2024-09-18T00:19:30.257162+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:53.675765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45096975,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":8,"num_read":4607,"num_read_kb":12681,"num_write":1115,"num_write_kb":3129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":637285824,"num_objects":867,"num_object_clones":127,"num_object_copies":1734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":867,"num_whiteouts":164,"num_read":83872,"num_read_kb":258712,"num_write":16832,"num_write_kb":175149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":934,"num_bytes_recovered":20602618,"num_keys_recovered":189,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28852224,"data_stored":176788251,"data_compressed":12703023,"data_compressed_allocated":20705280,"data_compressed_original":171829182,"omap_allocated":6178,"internal_metadata":0},"log_size":13486,"ondisk_log_size":13486,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20247,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51480,"kb_used_data":6396,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94320360,"statfs":{"total":96636764160,"available":96584048640,"internally_reserved":0,"allocated":6549504,"data_stored":32218263,"data_compressed":2375677,"data_compressed_allocated":3985408,"data_compressed_original":30791156,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":64424509463,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":76,"seq":326417514501,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38372,"kb_used_data":6116,"kb_used_omap":9,"kb_used_meta":32182,"kb_avail":94333468,"statfs":{"total":96636764160,"available":96597471232,"internally_reserved":0,"allocated":6262784,"data_stored":38526309,"data_compressed":2498600,"data_compressed_allocated":3911680,"data_compressed_original":37250930,"omap_allocated":9346,"internal_metadata":32955262},"hb_peers":[0,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55900,"kb_used_data":6668,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94315940,"statfs":{"total":96636764160,"available":96579522560,"internally_reserved":0,"allocated":6828032,"data_stored":29654085,"data_compressed":2392481,"data_compressed_allocated":4034560,"data_compressed_original":28131461,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37860,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333980,"statfs":{"total":96636764160,"available":96597995520,"internally_reserved":0,"allocated":2572288,"data_stored":6121397,"data_compressed":535022,"data_compressed_allocated":847872,"data_compressed_original":5208580,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54728,"kb_used_data":5496,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317112,"statfs":{"total":96636764160,"available":96580722688,"internally_reserved":0,"allocated":5627904,"data_stored":27748169,"data_compressed":1849183,"data_compressed_allocated":3190784,"data_compressed_original":26439767,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542169,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33640,"kb_used_data":6612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338200,"statfs":{"total":96636764160,"available":96602316800,"internally_reserved":0,"allocated":6770688,"data_stored":45640995,"data_compressed":2675941,"data_compressed_allocated":4415488,"data_compressed_original":44372238,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5640192,"data_stored":44570328,"data_compressed":2677879,"data_compressed_allocated":4386816,"data_compressed_original":43811086,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":617485,"data_compressed":281533,"data_compressed_allocated":319488,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1495040,"data_stored":5552855,"data_compressed":534837,"data_compressed_allocated":835584,"data_compressed_original":5134852,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4538368,"data_stored":27215551,"data_compressed":1849120,"data_compressed_allocated":3186688,"data_compressed_original":26406999,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5799936,"data_stored":29192099,"data_compressed":2392481,"data_compressed_allocated":4034560,"data_compressed_original":28131461,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5189632,"data_stored":38036036,"data_compressed":2498600,"data_compressed_allocated":3911680,"data_compressed_original":37250930,"omap_allocated":6178,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":491520,"data_stored":4760518,"data_compressed":157821,"data_compressed_allocated":393216,"data_compressed_original":4710400,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5177344,"data_stored":26843379,"data_compressed":2310752,"data_compressed_allocated":3637248,"data_compressed_original":25871454,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:19:35.272 INFO:tasks.thrashosds.thrasher:Setting 1.0 to [2, 0, 6, 1] 2024-09-18T00:19:35.272 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '1.0', '2', '0', '6', '1'] 2024-09-18T00:19:35.272 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 1.0 2 0 6 1 2024-09-18T00:19:35.273 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:19:35.366 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:35.365+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:35.444 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34543 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:19:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 21.149477568000002, 2024-09-18T00:19:35.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023207593, 2024-09-18T00:19:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34543 2024-09-18T00:19:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:19:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:19:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290783+0000", 2024-09-18T00:19:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5240000000000001e-06 2024-09-18T00:19:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290799+0000", 2024-09-18T00:19:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:19:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290840+0000", 2024-09-18T00:19:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0926000000000001e-05 2024-09-18T00:19:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290868+0000", 2024-09-18T00:19:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8163000000000001e-05 2024-09-18T00:19:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291294+0000", 2024-09-18T00:19:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00042588700000000001 2024-09-18T00:19:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:19:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:19:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.986e-05 2024-09-18T00:19:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316617+0000", 2024-09-18T00:19:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025303583000000001 2024-09-18T00:19:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316637+0000", 2024-09-18T00:19:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9388000000000001e-05 2024-09-18T00:19:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313933+0000", 2024-09-18T00:19:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99729681000000003 2024-09-18T00:19:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313987+0000", 2024-09-18T00:19:35.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4069e-05 2024-09-18T00:19:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34544 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:19:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 21.149329213000001, 2024-09-18T00:19:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0232412120000001, 2024-09-18T00:19:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.450 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34544 2024-09-18T00:19:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:19:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:19:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290930+0000", 2024-09-18T00:19:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5030000000000001e-06 2024-09-18T00:19:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290940+0000", 2024-09-18T00:19:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9000000000000001e-06 2024-09-18T00:19:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290964+0000", 2024-09-18T00:19:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4273e-05 2024-09-18T00:19:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290982+0000", 2024-09-18T00:19:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8216e-05 2024-09-18T00:19:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291433+0000", 2024-09-18T00:19:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000451278 2024-09-18T00:19:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:35.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291446+0000", 2024-09-18T00:19:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3039e-05 2024-09-18T00:19:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316687+0000", 2024-09-18T00:19:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025240069 2024-09-18T00:19:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316701+0000", 2024-09-18T00:19:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4100000000000001e-05 2024-09-18T00:19:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314142+0000", 2024-09-18T00:19:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99744119799999997 2024-09-18T00:19:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314169+0000", 2024-09-18T00:19:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7636000000000001e-05 2024-09-18T00:19:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34545 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:35.455 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:19:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 21.149250690999999, 2024-09-18T00:19:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023257802, 2024-09-18T00:19:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34545 2024-09-18T00:19:35.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:19:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:19:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291008+0000", 2024-09-18T00:19:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.237e-06 2024-09-18T00:19:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291015+0000", 2024-09-18T00:19:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2030000000000003e-06 2024-09-18T00:19:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291035+0000", 2024-09-18T00:19:35.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.978e-05 2024-09-18T00:19:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291049+0000", 2024-09-18T00:19:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4429e-05 2024-09-18T00:19:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291630+0000", 2024-09-18T00:19:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00058103199999999999 2024-09-18T00:19:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291644+0000", 2024-09-18T00:19:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.342e-05 2024-09-18T00:19:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316753+0000", 2024-09-18T00:19:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025108703999999999 2024-09-18T00:19:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316766+0000", 2024-09-18T00:19:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3237e-05 2024-09-18T00:19:35.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314238+0000", 2024-09-18T00:19:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99747232699999999 2024-09-18T00:19:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314265+0000", 2024-09-18T00:19:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6432999999999999e-05 2024-09-18T00:19:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34546 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:19:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 21.149192352, 2024-09-18T00:19:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023324535, 2024-09-18T00:19:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34546 2024-09-18T00:19:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.462 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:19:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:19:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291066+0000", 2024-09-18T00:19:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.569999999999999e-07 2024-09-18T00:19:35.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291073+0000", 2024-09-18T00:19:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.063e-06 2024-09-18T00:19:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291090+0000", 2024-09-18T00:19:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6977e-05 2024-09-18T00:19:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291105+0000", 2024-09-18T00:19:35.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5166e-05 2024-09-18T00:19:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291771+0000", 2024-09-18T00:19:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00066599699999999999 2024-09-18T00:19:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291785+0000", 2024-09-18T00:19:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.327e-05 2024-09-18T00:19:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316826+0000", 2024-09-18T00:19:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025041955000000001 2024-09-18T00:19:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316840+0000", 2024-09-18T00:19:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3256e-05 2024-09-18T00:19:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314335+0000", 2024-09-18T00:19:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99749496100000001 2024-09-18T00:19:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314390+0000", 2024-09-18T00:19:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4932999999999997e-05 2024-09-18T00:19:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34547 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:19:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 21.149133053, 2024-09-18T00:19:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.028510467, 2024-09-18T00:19:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.467 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34547 2024-09-18T00:19:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:19:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:19:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291126+0000", 2024-09-18T00:19:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2699999999999999e-06 2024-09-18T00:19:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291150+0000", 2024-09-18T00:19:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4669999999999999e-05 2024-09-18T00:19:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291171+0000", 2024-09-18T00:19:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0706000000000001e-05 2024-09-18T00:19:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291186+0000", 2024-09-18T00:19:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5126999999999999e-05 2024-09-18T00:19:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291885+0000", 2024-09-18T00:19:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00069850299999999995 2024-09-18T00:19:35.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291897+0000", 2024-09-18T00:19:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-05 2024-09-18T00:19:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316887+0000", 2024-09-18T00:19:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024990155999999999 2024-09-18T00:19:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316900+0000", 2024-09-18T00:19:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.295e-05 2024-09-18T00:19:35.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319594+0000", 2024-09-18T00:19:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0026940719999999 2024-09-18T00:19:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319635+0000", 2024-09-18T00:19:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0583000000000001e-05 2024-09-18T00:19:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34548 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:19:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 21.148985594999999, 2024-09-18T00:19:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0285287400000001, 2024-09-18T00:19:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34548 2024-09-18T00:19:35.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:19:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:19:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291274+0000", 2024-09-18T00:19:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.12e-06 2024-09-18T00:19:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291288+0000", 2024-09-18T00:19:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4367e-05 2024-09-18T00:19:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:19:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5698999999999999e-05 2024-09-18T00:19:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291333+0000", 2024-09-18T00:19:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8783e-05 2024-09-18T00:19:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291987+0000", 2024-09-18T00:19:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00065417800000000005 2024-09-18T00:19:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.292000+0000", 2024-09-18T00:19:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2652e-05 2024-09-18T00:19:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316946+0000", 2024-09-18T00:19:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024946696000000001 2024-09-18T00:19:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316956+0000", 2024-09-18T00:19:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2760000000000006e-06 2024-09-18T00:19:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319752+0000", 2024-09-18T00:19:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002796083 2024-09-18T00:19:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319801+0000", 2024-09-18T00:19:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8885999999999999e-05 2024-09-18T00:19:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41949 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693852+0000", 2024-09-18T00:19:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.746405789000001, 2024-09-18T00:19:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053245853000000003, 2024-09-18T00:19:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.478 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41949 2024-09-18T00:19:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693852+0000", 2024-09-18T00:19:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693852+0000", 2024-09-18T00:19:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693853+0000", 2024-09-18T00:19:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2999999999999999e-07 2024-09-18T00:19:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693866+0000", 2024-09-18T00:19:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3814e-05 2024-09-18T00:19:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693878+0000", 2024-09-18T00:19:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1384e-05 2024-09-18T00:19:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693893+0000", 2024-09-18T00:19:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4722000000000001e-05 2024-09-18T00:19:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.746958+0000", 2024-09-18T00:19:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053065734000000003 2024-09-18T00:19:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747015+0000", 2024-09-18T00:19:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6397000000000002e-05 2024-09-18T00:19:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747098+0000", 2024-09-18T00:19:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2872000000000002e-05 2024-09-18T00:19:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41950 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:35.482 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:19:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.746332885999999, 2024-09-18T00:19:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053368486, 2024-09-18T00:19:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41950 2024-09-18T00:19:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:19:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:19:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693926+0000", 2024-09-18T00:19:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4900000000000004e-07 2024-09-18T00:19:35.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693930+0000", 2024-09-18T00:19:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0130000000000004e-06 2024-09-18T00:19:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693940+0000", 2024-09-18T00:19:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0618000000000001e-05 2024-09-18T00:19:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693948+0000", 2024-09-18T00:19:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3140000000000004e-06 2024-09-18T00:19:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747133+0000", 2024-09-18T00:19:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053184689 2024-09-18T00:19:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747203+0000", 2024-09-18T00:19:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0339000000000005e-05 2024-09-18T00:19:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747293+0000", 2024-09-18T00:19:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9563999999999998e-05 2024-09-18T00:19:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41951 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:19:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.746298119, 2024-09-18T00:19:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053527695, 2024-09-18T00:19:35.487 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41951 2024-09-18T00:19:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:19:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:19:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693960+0000", 2024-09-18T00:19:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.16e-07 2024-09-18T00:19:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693971+0000", 2024-09-18T00:19:35.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1204e-05 2024-09-18T00:19:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693981+0000", 2024-09-18T00:19:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-18T00:19:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693989+0000", 2024-09-18T00:19:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0180000000000001e-06 2024-09-18T00:19:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747328+0000", 2024-09-18T00:19:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053338508 2024-09-18T00:19:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747400+0000", 2024-09-18T00:19:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2750999999999995e-05 2024-09-18T00:19:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747487+0000", 2024-09-18T00:19:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6527999999999996e-05 2024-09-18T00:19:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.491 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41952 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:19:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.746257712, 2024-09-18T00:19:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053728013999999998, 2024-09-18T00:19:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41952 2024-09-18T00:19:35.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:19:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:19:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694001+0000", 2024-09-18T00:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3799999999999996e-07 2024-09-18T00:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694004+0000", 2024-09-18T00:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.965e-06 2024-09-18T00:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694013+0000", 2024-09-18T00:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1689999999999998e-06 2024-09-18T00:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694022+0000", 2024-09-18T00:19:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7560000000000005e-06 2024-09-18T00:19:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747531+0000", 2024-09-18T00:19:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053508999000000002 2024-09-18T00:19:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747610+0000", 2024-09-18T00:19:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9212e-05 2024-09-18T00:19:35.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747728+0000", 2024-09-18T00:19:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011797499999999999 2024-09-18T00:19:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.496 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41953 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:19:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.746224083, 2024-09-18T00:19:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053936783000000002, 2024-09-18T00:19:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41953 2024-09-18T00:19:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:19:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:19:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694034+0000", 2024-09-18T00:19:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5199999999999996e-07 2024-09-18T00:19:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694037+0000", 2024-09-18T00:19:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.745e-06 2024-09-18T00:19:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694046+0000", 2024-09-18T00:19:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2490000000000007e-06 2024-09-18T00:19:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694054+0000", 2024-09-18T00:19:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1179999999999992e-06 2024-09-18T00:19:35.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747766+0000", 2024-09-18T00:19:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053711362999999998 2024-09-18T00:19:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747856+0000", 2024-09-18T00:19:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0779000000000006e-05 2024-09-18T00:19:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747970+0000", 2024-09-18T00:19:35.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011377700000000001 2024-09-18T00:19:35.501 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.501 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.501 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.501 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41954 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:35.501 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:35.501 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.74619393, 2024-09-18T00:19:35.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054104898999999998, 2024-09-18T00:19:35.501 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.501 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.501 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:35.501 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.502 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41954 2024-09-18T00:19:35.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.502 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:35.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:35.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:35.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4000000000000002e-07 2024-09-18T00:19:35.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694067+0000", 2024-09-18T00:19:35.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7329999999999999e-06 2024-09-18T00:19:35.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694076+0000", 2024-09-18T00:19:35.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2019999999999993e-06 2024-09-18T00:19:35.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694083+0000", 2024-09-18T00:19:35.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0029999999999997e-06 2024-09-18T00:19:35.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748003+0000", 2024-09-18T00:19:35.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053920463000000002 2024-09-18T00:19:35.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:35.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748077+0000", 2024-09-18T00:19:35.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4041000000000005e-05 2024-09-18T00:19:35.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748168+0000", 2024-09-18T00:19:35.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0916999999999997e-05 2024-09-18T00:19:35.505 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.505 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.505 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.505 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41955 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 231424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:35.505 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:19:35.506 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.746165092, 2024-09-18T00:19:35.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054272659000000001, 2024-09-18T00:19:35.506 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.506 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.506 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:35.506 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.506 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.506 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.506 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41955 2024-09-18T00:19:35.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.506 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:19:35.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:19:35.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694093+0000", 2024-09-18T00:19:35.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:19:35.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694109+0000", 2024-09-18T00:19:35.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6061000000000001e-05 2024-09-18T00:19:35.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694121+0000", 2024-09-18T00:19:35.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.216e-05 2024-09-18T00:19:35.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694129+0000", 2024-09-18T00:19:35.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0029999999999994e-06 2024-09-18T00:19:35.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748202+0000", 2024-09-18T00:19:35.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054072571 2024-09-18T00:19:35.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:35.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748274+0000", 2024-09-18T00:19:35.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2117000000000001e-05 2024-09-18T00:19:35.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748365+0000", 2024-09-18T00:19:35.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1139999999999998e-05 2024-09-18T00:19:35.509 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.510 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.510 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.510 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41956 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:35.510 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:35.510 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.746116883999999, 2024-09-18T00:19:35.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054420496999999998, 2024-09-18T00:19:35.510 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.510 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.510 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:35.510 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.510 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.511 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.511 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41956 2024-09-18T00:19:35.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.511 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:35.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:35.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:35.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8699999999999995e-07 2024-09-18T00:19:35.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694144+0000", 2024-09-18T00:19:35.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3510000000000002e-06 2024-09-18T00:19:35.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694154+0000", 2024-09-18T00:19:35.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0769999999999997e-06 2024-09-18T00:19:35.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694163+0000", 2024-09-18T00:19:35.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3300000000000005e-06 2024-09-18T00:19:35.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748397+0000", 2024-09-18T00:19:35.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054234497999999999 2024-09-18T00:19:35.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:35.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748471+0000", 2024-09-18T00:19:35.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3441000000000004e-05 2024-09-18T00:19:35.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748561+0000", 2024-09-18T00:19:35.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0212999999999998e-05 2024-09-18T00:19:35.514 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.514 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.514 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.514 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41957 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:35.514 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:19:35.514 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.746085632, 2024-09-18T00:19:35.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054590310000000003, 2024-09-18T00:19:35.515 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.515 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.515 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:35.515 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.515 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.515 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.515 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41957 2024-09-18T00:19:35.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.515 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:19:35.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:19:35.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694173+0000", 2024-09-18T00:19:35.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2500000000000005e-07 2024-09-18T00:19:35.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694175+0000", 2024-09-18T00:19:35.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.655e-06 2024-09-18T00:19:35.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694184+0000", 2024-09-18T00:19:35.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7630000000000006e-06 2024-09-18T00:19:35.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694191+0000", 2024-09-18T00:19:35.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8870000000000003e-06 2024-09-18T00:19:35.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748593+0000", 2024-09-18T00:19:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054402525 2024-09-18T00:19:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748669+0000", 2024-09-18T00:19:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5532999999999997e-05 2024-09-18T00:19:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748762+0000", 2024-09-18T00:19:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3221999999999996e-05 2024-09-18T00:19:35.519 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.519 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.519 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.519 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42391 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 337920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:35.519 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707180+0000", 2024-09-18T00:19:35.519 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.73307788, 2024-09-18T00:19:35.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053238496000000003, 2024-09-18T00:19:35.519 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.519 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.519 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:35.519 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.519 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.520 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.520 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42391 2024-09-18T00:19:35.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.520 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707180+0000", 2024-09-18T00:19:35.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707180+0000", 2024-09-18T00:19:35.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707180+0000", 2024-09-18T00:19:35.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6400000000000003e-07 2024-09-18T00:19:35.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707182+0000", 2024-09-18T00:19:35.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0870000000000001e-06 2024-09-18T00:19:35.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707189+0000", 2024-09-18T00:19:35.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7469999999999999e-06 2024-09-18T00:19:35.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707194+0000", 2024-09-18T00:19:35.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9459999999999997e-06 2024-09-18T00:19:35.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760268+0000", 2024-09-18T00:19:35.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053073899000000001 2024-09-18T00:19:35.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:35.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760351+0000", 2024-09-18T00:19:35.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3604999999999997e-05 2024-09-18T00:19:35.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760418+0000", 2024-09-18T00:19:35.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6748000000000005e-05 2024-09-18T00:19:35.523 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.523 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.523 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.523 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42392 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:35.523 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:19:35.523 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.733056192999999, 2024-09-18T00:19:35.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053378162, 2024-09-18T00:19:35.524 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.524 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.524 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:35.524 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.524 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.524 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.524 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42392 2024-09-18T00:19:35.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.524 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:19:35.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:19:35.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707430+0000", 2024-09-18T00:19:35.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022838599999999999 2024-09-18T00:19:35.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707434+0000", 2024-09-18T00:19:35.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5220000000000001e-06 2024-09-18T00:19:35.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707467+0000", 2024-09-18T00:19:35.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2570000000000002e-05 2024-09-18T00:19:35.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707475+0000", 2024-09-18T00:19:35.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:19:35.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760446+0000", 2024-09-18T00:19:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052971635000000003 2024-09-18T00:19:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760512+0000", 2024-09-18T00:19:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5679999999999995e-05 2024-09-18T00:19:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760579+0000", 2024-09-18T00:19:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7296000000000001e-05 2024-09-18T00:19:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42393 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 358400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:19:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.732767242, 2024-09-18T00:19:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053269618999999997, 2024-09-18T00:19:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42393 2024-09-18T00:19:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:19:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:19:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707491+0000", 2024-09-18T00:19:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1399999999999997e-07 2024-09-18T00:19:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707503+0000", 2024-09-18T00:19:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1912e-05 2024-09-18T00:19:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707512+0000", 2024-09-18T00:19:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1030000000000008e-06 2024-09-18T00:19:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707520+0000", 2024-09-18T00:19:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1249999999999993e-06 2024-09-18T00:19:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760605+0000", 2024-09-18T00:19:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053085495000000003 2024-09-18T00:19:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760680+0000", 2024-09-18T00:19:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5024000000000005e-05 2024-09-18T00:19:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760760+0000", 2024-09-18T00:19:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9345999999999996e-05 2024-09-18T00:19:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42394 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:19:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.732727254, 2024-09-18T00:19:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053403144999999999, 2024-09-18T00:19:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42394 2024-09-18T00:19:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:19:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:19:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711550+0000", 2024-09-18T00:19:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040200189999999997 2024-09-18T00:19:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711556+0000", 2024-09-18T00:19:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2339999999999999e-06 2024-09-18T00:19:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711569+0000", 2024-09-18T00:19:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2337999999999999e-05 2024-09-18T00:19:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711577+0000", 2024-09-18T00:19:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5660000000000007e-06 2024-09-18T00:19:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760790+0000", 2024-09-18T00:19:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049212196999999999 2024-09-18T00:19:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760857+0000", 2024-09-18T00:19:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7756000000000007e-05 2024-09-18T00:19:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760933+0000", 2024-09-18T00:19:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6035e-05 2024-09-18T00:19:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:55489 3.5 3:a5f1e468:::rbd_data.112a859a01c9.000000000000000c:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T00:19:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:19:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.8088731280000001, 2024-09-18T00:19:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.05748731, 2024-09-18T00:19:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 55489 2024-09-18T00:19:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:19:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:19:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631388+0000", 2024-09-18T00:19:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6169999999999998e-06 2024-09-18T00:19:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631405+0000", 2024-09-18T00:19:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6643e-05 2024-09-18T00:19:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631444+0000", 2024-09-18T00:19:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9226e-05 2024-09-18T00:19:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631465+0000", 2024-09-18T00:19:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0826999999999999e-05 2024-09-18T00:19:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631578+0000", 2024-09-18T00:19:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011291899999999999 2024-09-18T00:19:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631651+0000", 2024-09-18T00:19:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3264999999999997e-05 2024-09-18T00:19:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:19:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.634073+0000", 2024-09-18T00:19:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0024221350000000002 2024-09-18T00:19:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688732+0000", 2024-09-18T00:19:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054658610000000003 2024-09-18T00:19:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688794+0000", 2024-09-18T00:19:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2255999999999995e-05 2024-09-18T00:19:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688834+0000", 2024-09-18T00:19:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9916999999999997e-05 2024-09-18T00:19:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688872+0000", 2024-09-18T00:19:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7895000000000002e-05 2024-09-18T00:19:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:35.543 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:35.544 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:35.541+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:35.544 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:19:35.637 INFO:teuthology.orchestra.run.smithi097.stdout:flattening image image_client.0-clone25 2024-09-18T00:19:35.645 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:35.645+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:35.685 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:35.685 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:35.686 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:35.686 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:35.699 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:19:35.746 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:35.745+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:35.847 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:35.845+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:35.871 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:35.871 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:35.871 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:35.871 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:35.871 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:35.884 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:35.898 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-18T00:19:35.947 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:35.945+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:36.009 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2230 3.17 3:ef492888:::rbd_data.112aa7ba4578.000000000000014f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 75.739986075999994, 2024-09-18T00:19:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075065283999999996, 2024-09-18T00:19:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.009 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2230 2024-09-18T00:19:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.010 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266014+0000", 2024-09-18T00:19:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:19:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266029+0000", 2024-09-18T00:19:36.011 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4616e-05 2024-09-18T00:19:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266053+0000", 2024-09-18T00:19:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4369999999999999e-05 2024-09-18T00:19:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266067+0000", 2024-09-18T00:19:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4243999999999999e-05 2024-09-18T00:19:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266157+0000", 2024-09-18T00:19:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9909999999999998e-05 2024-09-18T00:19:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266327+0000", 2024-09-18T00:19:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016981200000000001 2024-09-18T00:19:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:19:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269180+0000", 2024-09-18T00:19:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028526559999999999 2024-09-18T00:19:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.013 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340955+0000", 2024-09-18T00:19:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.071774904 2024-09-18T00:19:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340974+0000", 2024-09-18T00:19:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8824000000000001e-05 2024-09-18T00:19:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341028+0000", 2024-09-18T00:19:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4456999999999997e-05 2024-09-18T00:19:36.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341076+0000", 2024-09-18T00:19:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8284000000000001e-05 2024-09-18T00:19:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2230 3.17 e36/23 ondisk, result = 0)", 2024-09-18T00:19:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 75.722642495000002, 2024-09-18T00:19:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057609645000000001, 2024-09-18T00:19:36.015 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.016 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283356+0000", 2024-09-18T00:19:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.44e-06 2024-09-18T00:19:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283362+0000", 2024-09-18T00:19:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2539999999999998e-06 2024-09-18T00:19:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283382+0000", 2024-09-18T00:19:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9896000000000001e-05 2024-09-18T00:19:36.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283398+0000", 2024-09-18T00:19:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5489999999999999e-05 2024-09-18T00:19:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340936+0000", 2024-09-18T00:19:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057538649999999997 2024-09-18T00:19:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340947+0000", 2024-09-18T00:19:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0981e-05 2024-09-18T00:19:36.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340964+0000", 2024-09-18T00:19:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6934000000000001e-05 2024-09-18T00:19:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2419 3.17 3:ed94b133:::rbd_data.112aa7ba4578.00000000000001b5:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:19:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 75.268521022000002, 2024-09-18T00:19:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.064761812000000002, 2024-09-18T00:19:36.019 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2419 2024-09-18T00:19:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737479+0000", 2024-09-18T00:19:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6089999999999999e-06 2024-09-18T00:19:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737487+0000", 2024-09-18T00:19:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7900000000000005e-06 2024-09-18T00:19:36.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737507+0000", 2024-09-18T00:19:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9936e-05 2024-09-18T00:19:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737518+0000", 2024-09-18T00:19:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1046999999999999e-05 2024-09-18T00:19:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737599+0000", 2024-09-18T00:19:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0697000000000003e-05 2024-09-18T00:19:36.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737697+0000", 2024-09-18T00:19:36.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8058000000000002e-05 2024-09-18T00:19:36.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:19:36.023 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.740445+0000", 2024-09-18T00:19:36.023 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027481279999999999 2024-09-18T00:19:36.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.023 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802023+0000", 2024-09-18T00:19:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.061577321999999997 2024-09-18T00:19:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802060+0000", 2024-09-18T00:19:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7586000000000002e-05 2024-09-18T00:19:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802158+0000", 2024-09-18T00:19:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7583000000000004e-05 2024-09-18T00:19:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.024 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.025 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.025 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802238+0000", 2024-09-18T00:19:36.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0055999999999994e-05 2024-09-18T00:19:36.025 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.025 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.025 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.025 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.025 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2419 3.17 e37/23 ondisk, result = 0)", 2024-09-18T00:19:36.025 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:36.025 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 75.256296075999998, 2024-09-18T00:19:36.025 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052342948, 2024-09-18T00:19:36.025 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.026 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.026 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:36.026 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:36.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.026 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.026 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:36.026 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.026 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749702+0000", 2024-09-18T00:19:36.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:19:36.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749710+0000", 2024-09-18T00:19:36.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7279999999999995e-06 2024-09-18T00:19:36.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.027 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.027 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749736+0000", 2024-09-18T00:19:36.027 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5797999999999999e-05 2024-09-18T00:19:36.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.027 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749751+0000", 2024-09-18T00:19:36.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5238999999999999e-05 2024-09-18T00:19:36.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801745+0000", 2024-09-18T00:19:36.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051994022000000001 2024-09-18T00:19:36.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.028 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.028 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.028 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801770+0000", 2024-09-18T00:19:36.028 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5103e-05 2024-09-18T00:19:36.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.029 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.029 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802044+0000", 2024-09-18T00:19:36.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027395799999999998 2024-09-18T00:19:36.029 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.029 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.029 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.029 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6503 3.1b 3:d9387a1f:::rbd_data.112a3a39fc45.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:36.029 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:36.029 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 70.441623105999994, 2024-09-18T00:19:36.029 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050510547000000003, 2024-09-18T00:19:36.030 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.030 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.030 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:36.030 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:36.030 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:36.030 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:36.030 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6503 2024-09-18T00:19:36.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.030 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.030 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.030 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:36.030 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.030 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:36.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564376+0000", 2024-09-18T00:19:36.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-06 2024-09-18T00:19:36.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.031 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.031 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.031 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564386+0000", 2024-09-18T00:19:36.031 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0312999999999999e-05 2024-09-18T00:19:36.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564408+0000", 2024-09-18T00:19:36.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2001000000000002e-05 2024-09-18T00:19:36.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564421+0000", 2024-09-18T00:19:36.032 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2271e-05 2024-09-18T00:19:36.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.032 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.032 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.032 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564491+0000", 2024-09-18T00:19:36.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0630000000000006e-05 2024-09-18T00:19:36.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564606+0000", 2024-09-18T00:19:36.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011449 2024-09-18T00:19:36.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:36.033 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.566743+0000", 2024-09-18T00:19:36.033 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021373830000000001 2024-09-18T00:19:36.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.033 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.033 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:36.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614738+0000", 2024-09-18T00:19:36.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047995359000000001 2024-09-18T00:19:36.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:36.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614759+0000", 2024-09-18T00:19:36.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1010999999999999e-05 2024-09-18T00:19:36.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.034 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:36.034 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614826+0000", 2024-09-18T00:19:36.034 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6654000000000003e-05 2024-09-18T00:19:36.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.034 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.035 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.035 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614884+0000", 2024-09-18T00:19:36.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8332999999999999e-05 2024-09-18T00:19:36.035 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.035 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.035 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.035 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.035 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6518 3.1 3:8061bf9f:::rbd_data.112a3a39fc45.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:36.035 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:36.035 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 70.355184832000006, 2024-09-18T00:19:36.035 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062303123000000002, 2024-09-18T00:19:36.035 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.035 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.035 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:36.035 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:36.036 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:36.036 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:36.036 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6518 2024-09-18T00:19:36.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.036 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:36.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.036 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.036 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.036 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:36.036 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650814+0000", 2024-09-18T00:19:36.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.629e-06 2024-09-18T00:19:36.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650823+0000", 2024-09-18T00:19:36.037 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1800000000000002e-06 2024-09-18T00:19:36.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.037 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.037 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.037 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650843+0000", 2024-09-18T00:19:36.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9836000000000001e-05 2024-09-18T00:19:36.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650854+0000", 2024-09-18T00:19:36.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1564e-05 2024-09-18T00:19:36.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650917+0000", 2024-09-18T00:19:36.038 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3076000000000004e-05 2024-09-18T00:19:36.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.038 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.038 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.038 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.651029+0000", 2024-09-18T00:19:36.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111154 2024-09-18T00:19:36.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:36.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.655531+0000", 2024-09-18T00:19:36.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045027629999999999 2024-09-18T00:19:36.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.039 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:36.039 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712924+0000", 2024-09-18T00:19:36.039 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057393006000000003 2024-09-18T00:19:36.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.039 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:36.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712988+0000", 2024-09-18T00:19:36.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000005e-05 2024-09-18T00:19:36.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:36.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713056+0000", 2024-09-18T00:19:36.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7904999999999994e-05 2024-09-18T00:19:36.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.040 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.040 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713115+0000", 2024-09-18T00:19:36.040 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9339999999999998e-05 2024-09-18T00:19:36.040 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.041 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.041 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.041 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.041 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6518 3.1 e42/23 ondisk, result = 0)", 2024-09-18T00:19:36.041 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:36.041 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 70.342759748999995, 2024-09-18T00:19:36.041 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049887304, 2024-09-18T00:19:36.041 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.041 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.041 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:36.041 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.041 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.041 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:36.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:36.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.042 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.042 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663239+0000", 2024-09-18T00:19:36.042 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:19:36.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.042 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663246+0000", 2024-09-18T00:19:36.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4189999999999996e-06 2024-09-18T00:19:36.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663265+0000", 2024-09-18T00:19:36.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8323000000000001e-05 2024-09-18T00:19:36.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.043 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.043 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.043 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663277+0000", 2024-09-18T00:19:36.043 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2065999999999999e-05 2024-09-18T00:19:36.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712965+0000", 2024-09-18T00:19:36.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049688317000000003 2024-09-18T00:19:36.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.044 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712977+0000", 2024-09-18T00:19:36.044 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.206e-05 2024-09-18T00:19:36.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.044 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.044 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.045 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713125+0000", 2024-09-18T00:19:36.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014749899999999999 2024-09-18T00:19:36.045 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.045 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.045 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.045 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.045 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:8831 3.12 3:49cac8cf:::rbd_data.112a9f4259df.0000000000000130:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T00:19:36.045 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:36.045 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 63.724899499999999, 2024-09-18T00:19:36.045 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104125047, 2024-09-18T00:19:36.045 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.045 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.045 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:36.046 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:36.046 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:36.046 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:36.046 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 8831 2024-09-18T00:19:36.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.046 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:36.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.046 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.046 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:36.046 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281099+0000", 2024-09-18T00:19:36.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.029e-06 2024-09-18T00:19:36.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281103+0000", 2024-09-18T00:19:36.047 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7759999999999997e-06 2024-09-18T00:19:36.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.047 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.047 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.047 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281114+0000", 2024-09-18T00:19:36.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1090000000000001e-05 2024-09-18T00:19:36.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281120+0000", 2024-09-18T00:19:36.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.604e-06 2024-09-18T00:19:36.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.048 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281164+0000", 2024-09-18T00:19:36.048 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3696999999999999e-05 2024-09-18T00:19:36.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.048 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281270+0000", 2024-09-18T00:19:36.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106041 2024-09-18T00:19:36.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:36.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.285596+0000", 2024-09-18T00:19:36.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043266659999999998 2024-09-18T00:19:36.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.049 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:36.049 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385028+0000", 2024-09-18T00:19:36.049 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.099431808999999996 2024-09-18T00:19:36.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.049 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:36.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385094+0000", 2024-09-18T00:19:36.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5740000000000004e-05 2024-09-18T00:19:36.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:36.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385166+0000", 2024-09-18T00:19:36.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1637000000000005e-05 2024-09-18T00:19:36.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.050 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.050 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.050 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385223+0000", 2024-09-18T00:19:36.050 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6957999999999999e-05 2024-09-18T00:19:36.050 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.051 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.051 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.051 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:8831 3.12 e48/32 ondisk, result = 0)", 2024-09-18T00:19:36.051 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:36.051 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 63.708385389999997, 2024-09-18T00:19:36.051 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087621558000000002, 2024-09-18T00:19:36.051 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.051 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.051 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:36.051 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.051 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.051 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.051 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:36.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:36.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.052 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297613+0000", 2024-09-18T00:19:36.052 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3400000000000001e-06 2024-09-18T00:19:36.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.052 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.052 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297620+0000", 2024-09-18T00:19:36.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5869999999999998e-06 2024-09-18T00:19:36.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297639+0000", 2024-09-18T00:19:36.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9888999999999999e-05 2024-09-18T00:19:36.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.053 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.053 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.053 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297661+0000", 2024-09-18T00:19:36.053 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1772000000000001e-05 2024-09-18T00:19:36.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385073+0000", 2024-09-18T00:19:36.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087411303999999995 2024-09-18T00:19:36.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385086+0000", 2024-09-18T00:19:36.054 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3040000000000001e-05 2024-09-18T00:19:36.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.054 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.054 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.054 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385233+0000", 2024-09-18T00:19:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014762600000000001 2024-09-18T00:19:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22269 3.17 3:e9a1eb35:::rbd_data.112a948d5956.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.096853564, 2024-09-18T00:19:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.063462693000000001, 2024-09-18T00:19:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:36.055 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:36.056 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:36.056 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:36.056 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22269 2024-09-18T00:19:36.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.056 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:36.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.056 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.056 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.056 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:36.056 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909146+0000", 2024-09-18T00:19:36.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3700000000000002e-06 2024-09-18T00:19:36.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909160+0000", 2024-09-18T00:19:36.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3825000000000001e-05 2024-09-18T00:19:36.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.057 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.057 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.057 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909178+0000", 2024-09-18T00:19:36.057 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8567999999999999e-05 2024-09-18T00:19:36.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909189+0000", 2024-09-18T00:19:36.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0881e-05 2024-09-18T00:19:36.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.058 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.058 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.058 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909270+0000", 2024-09-18T00:19:36.058 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0846999999999994e-05 2024-09-18T00:19:36.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.059 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909383+0000", 2024-09-18T00:19:36.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112704 2024-09-18T00:19:36.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.060 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.060 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:36.060 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972310+0000", 2024-09-18T00:19:36.060 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062927081999999995 2024-09-18T00:19:36.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:36.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972433+0000", 2024-09-18T00:19:36.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-18T00:19:36.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.061 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.061 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972606+0000", 2024-09-18T00:19:36.061 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017317799999999999 2024-09-18T00:19:36.061 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.061 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.061 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.062 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.062 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22379 3.17 3:e9a1eb35:::rbd_data.112a948d5956.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:36.062 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:36.062 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 37.602778266999998, 2024-09-18T00:19:36.062 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.12560279099999999, 2024-09-18T00:19:36.062 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.062 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.062 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:36.062 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:36.062 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:36.062 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:36.062 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22379 2024-09-18T00:19:36.062 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.062 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:36.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:36.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.063 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.063 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.063 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403221+0000", 2024-09-18T00:19:36.063 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3269999999999999e-06 2024-09-18T00:19:36.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403232+0000", 2024-09-18T00:19:36.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1392e-05 2024-09-18T00:19:36.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403264+0000", 2024-09-18T00:19:36.064 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1146999999999998e-05 2024-09-18T00:19:36.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.064 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.064 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.064 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403279+0000", 2024-09-18T00:19:36.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5109000000000001e-05 2024-09-18T00:19:36.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403424+0000", 2024-09-18T00:19:36.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014516999999999999 2024-09-18T00:19:36.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.065 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403567+0000", 2024-09-18T00:19:36.065 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142824 2024-09-18T00:19:36.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.065 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.065 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:36.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528723+0000", 2024-09-18T00:19:36.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.12515647299999999 2024-09-18T00:19:36.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:36.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528764+0000", 2024-09-18T00:19:36.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0624000000000003e-05 2024-09-18T00:19:36.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.066 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.066 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528822+0000", 2024-09-18T00:19:36.066 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7725e-05 2024-09-18T00:19:36.066 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.066 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.067 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.067 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.067 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.067 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39267 3.1f 3:fdf3a436:::rbd_data.112a3496573c.0000000000000001:17 [read 342016~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:36.067 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.576201+0000", 2024-09-18T00:19:36.067 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.429795797000001, 2024-09-18T00:19:36.067 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047103260000000001, 2024-09-18T00:19:36.067 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.067 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.067 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:36.067 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:36.067 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:36.067 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:36.067 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39267 2024-09-18T00:19:36.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.068 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576201+0000", 2024-09-18T00:19:36.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.068 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.068 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576201+0000", 2024-09-18T00:19:36.068 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.068 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576202+0000", 2024-09-18T00:19:36.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5799999999999999e-07 2024-09-18T00:19:36.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576204+0000", 2024-09-18T00:19:36.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1320000000000001e-06 2024-09-18T00:19:36.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.069 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.069 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.069 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576210+0000", 2024-09-18T00:19:36.069 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8939999999999998e-06 2024-09-18T00:19:36.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576215+0000", 2024-09-18T00:19:36.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9409999999999998e-06 2024-09-18T00:19:36.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623140+0000", 2024-09-18T00:19:36.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.046925253 2024-09-18T00:19:36.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.070 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.070 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.070 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623223+0000", 2024-09-18T00:19:36.070 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2976e-05 2024-09-18T00:19:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623305+0000", 2024-09-18T00:19:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1506000000000002e-05 2024-09-18T00:19:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39268 3.1f 3:fdf3a436:::rbd_data.112a3496573c.0000000000000001:17 [read 352256~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.576221+0000", 2024-09-18T00:19:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.429776223999999, 2024-09-18T00:19:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047293515000000001, 2024-09-18T00:19:36.071 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39268 2024-09-18T00:19:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576221+0000", 2024-09-18T00:19:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576221+0000", 2024-09-18T00:19:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576221+0000", 2024-09-18T00:19:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-07 2024-09-18T00:19:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.073 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576229+0000", 2024-09-18T00:19:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9950000000000005e-06 2024-09-18T00:19:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576237+0000", 2024-09-18T00:19:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0010000000000001e-06 2024-09-18T00:19:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576243+0000", 2024-09-18T00:19:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.592e-06 2024-09-18T00:19:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.074 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623336+0000", 2024-09-18T00:19:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047092698000000002 2024-09-18T00:19:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623418+0000", 2024-09-18T00:19:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1977000000000005e-05 2024-09-18T00:19:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623514+0000", 2024-09-18T00:19:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6676999999999996e-05 2024-09-18T00:19:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.075 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39269 3.1f 3:fdf3a436:::rbd_data.112a3496573c.0000000000000001:17 [read 362496~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.576250+0000", 2024-09-18T00:19:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.429747313, 2024-09-18T00:19:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047454169999999997, 2024-09-18T00:19:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:36.076 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39269 2024-09-18T00:19:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576250+0000", 2024-09-18T00:19:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576250+0000", 2024-09-18T00:19:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.077 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576250+0000", 2024-09-18T00:19:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:19:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576252+0000", 2024-09-18T00:19:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1739999999999999e-06 2024-09-18T00:19:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576258+0000", 2024-09-18T00:19:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6670000000000001e-06 2024-09-18T00:19:36.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576264+0000", 2024-09-18T00:19:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1360000000000002e-06 2024-09-18T00:19:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623543+0000", 2024-09-18T00:19:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047278496000000003 2024-09-18T00:19:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623609+0000", 2024-09-18T00:19:36.079 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5901000000000005e-05 2024-09-18T00:19:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623704+0000", 2024-09-18T00:19:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5291999999999995e-05 2024-09-18T00:19:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39270 3.1f 3:fdf3a436:::rbd_data.112a3496573c.0000000000000001:17 [read 372736~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.576271+0000", 2024-09-18T00:19:36.080 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.429725983999999, 2024-09-18T00:19:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047626546999999998, 2024-09-18T00:19:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39270 2024-09-18T00:19:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576271+0000", 2024-09-18T00:19:36.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576271+0000", 2024-09-18T00:19:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576271+0000", 2024-09-18T00:19:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.01e-07 2024-09-18T00:19:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576274+0000", 2024-09-18T00:19:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1150000000000001e-06 2024-09-18T00:19:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576280+0000", 2024-09-18T00:19:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9399999999999999e-06 2024-09-18T00:19:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576285+0000", 2024-09-18T00:19:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0719999999999999e-06 2024-09-18T00:19:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623731+0000", 2024-09-18T00:19:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047446629999999997 2024-09-18T00:19:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623809+0000", 2024-09-18T00:19:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8141999999999998e-05 2024-09-18T00:19:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623898+0000", 2024-09-18T00:19:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8246999999999997e-05 2024-09-18T00:19:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.084 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39360 3.1f 3:fec8fa94:::rbd_data.112a3496573c.0000000000000006:17 [read 354304~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.579954+0000", 2024-09-18T00:19:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.426043136000001, 2024-09-18T00:19:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047250546999999997, 2024-09-18T00:19:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39360 2024-09-18T00:19:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579954+0000", 2024-09-18T00:19:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579954+0000", 2024-09-18T00:19:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579954+0000", 2024-09-18T00:19:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7800000000000002e-07 2024-09-18T00:19:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579956+0000", 2024-09-18T00:19:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.114e-06 2024-09-18T00:19:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579962+0000", 2024-09-18T00:19:36.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7400000000000001e-06 2024-09-18T00:19:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579967+0000", 2024-09-18T00:19:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.792e-06 2024-09-18T00:19:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627056+0000", 2024-09-18T00:19:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047089131999999999 2024-09-18T00:19:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627122+0000", 2024-09-18T00:19:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6229000000000006e-05 2024-09-18T00:19:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627204+0000", 2024-09-18T00:19:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2062000000000001e-05 2024-09-18T00:19:36.089 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39361 3.1f 3:fec8fa94:::rbd_data.112a3496573c.0000000000000006:17 [read 364544~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.579975+0000", 2024-09-18T00:19:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.42602198, 2024-09-18T00:19:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047405364999999998, 2024-09-18T00:19:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:36.090 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39361 2024-09-18T00:19:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579975+0000", 2024-09-18T00:19:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579975+0000", 2024-09-18T00:19:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579976+0000", 2024-09-18T00:19:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7399999999999998e-07 2024-09-18T00:19:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579978+0000", 2024-09-18T00:19:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5359999999999999e-06 2024-09-18T00:19:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579984+0000", 2024-09-18T00:19:36.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2469999999999997e-06 2024-09-18T00:19:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579989+0000", 2024-09-18T00:19:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6700000000000002e-06 2024-09-18T00:19:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627232+0000", 2024-09-18T00:19:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047243016999999998 2024-09-18T00:19:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627299+0000", 2024-09-18T00:19:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7145999999999997e-05 2024-09-18T00:19:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627380+0000", 2024-09-18T00:19:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1274999999999997e-05 2024-09-18T00:19:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39472 3.1f 3:fd448eba:::rbd_data.112a3496573c.000000000000000c:17 [read 356352~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:36.094 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.583279+0000", 2024-09-18T00:19:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.42271807, 2024-09-18T00:19:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047226974999999997, 2024-09-18T00:19:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39472 2024-09-18T00:19:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.095 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583279+0000", 2024-09-18T00:19:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583279+0000", 2024-09-18T00:19:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583279+0000", 2024-09-18T00:19:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7199999999999999e-07 2024-09-18T00:19:36.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583281+0000", 2024-09-18T00:19:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9920000000000002e-06 2024-09-18T00:19:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583288+0000", 2024-09-18T00:19:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.849e-06 2024-09-18T00:19:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583293+0000", 2024-09-18T00:19:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0329999999999999e-06 2024-09-18T00:19:36.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.630334+0000", 2024-09-18T00:19:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047041038 2024-09-18T00:19:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.630403+0000", 2024-09-18T00:19:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8468999999999998e-05 2024-09-18T00:19:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.630506+0000", 2024-09-18T00:19:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010312200000000001 2024-09-18T00:19:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39473 3.1f 3:fd448eba:::rbd_data.112a3496573c.000000000000000c:17 [read 366592~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.583299+0000", 2024-09-18T00:19:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 15.422697659000001, 2024-09-18T00:19:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047411813999999997, 2024-09-18T00:19:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.099 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39473 2024-09-18T00:19:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583299+0000", 2024-09-18T00:19:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583299+0000", 2024-09-18T00:19:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583300+0000", 2024-09-18T00:19:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6699999999999999e-07 2024-09-18T00:19:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583302+0000", 2024-09-18T00:19:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.979e-06 2024-09-18T00:19:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583308+0000", 2024-09-18T00:19:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.835e-06 2024-09-18T00:19:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583313+0000", 2024-09-18T00:19:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9409999999999998e-06 2024-09-18T00:19:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.630537+0000", 2024-09-18T00:19:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047224592000000003 2024-09-18T00:19:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.630623+0000", 2024-09-18T00:19:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5473e-05 2024-09-18T00:19:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.630711+0000", 2024-09-18T00:19:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8527000000000001e-05 2024-09-18T00:19:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.103 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:54153 3.b e90/23 3:d3f7b2f3:::rbd_data.112a859a01c9.0000000000000001:head v 90'238, mlcod=90'236)", 2024-09-18T00:19:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:30.858396+0000", 2024-09-18T00:19:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 5.147601291, 2024-09-18T00:19:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056676272999999999, 2024-09-18T00:19:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858396+0000", 2024-09-18T00:19:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:36.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858396+0000", 2024-09-18T00:19:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858398+0000", 2024-09-18T00:19:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.838e-06 2024-09-18T00:19:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860747+0000", 2024-09-18T00:19:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0023496160000000001 2024-09-18T00:19:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:36.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860777+0000", 2024-09-18T00:19:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9754e-05 2024-09-18T00:19:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860791+0000", 2024-09-18T00:19:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.377e-05 2024-09-18T00:19:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860879+0000", 2024-09-18T00:19:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7818999999999994e-05 2024-09-18T00:19:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:36.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860917+0000", 2024-09-18T00:19:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8124999999999998e-05 2024-09-18T00:19:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915020+0000", 2024-09-18T00:19:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.054103126000000001 2024-09-18T00:19:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915072+0000", 2024-09-18T00:19:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2225000000000002e-05 2024-09-18T00:19:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:36.107 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:36.108 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:36.109 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:19:36.109 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:36.109+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:36.125 INFO:teuthology.orchestra.run.smithi097.stdout:5729 trunc from 0xd8a7b76 to 0x2eb3e55 2024-09-18T00:19:36.139 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:19:36.163 INFO:teuthology.orchestra.run.smithi097.stdout:5730 punch from 0x27b0e12 to 0x27bf7d9, (0xe9c7 bytes) 2024-09-18T00:19:36.178 INFO:teuthology.orchestra.run.smithi097.stdout:5731 writesame 0x895eb1e thru 0x896168a data_size 0x1(0x2b6d bytes) 2024-09-18T00:19:36.179 INFO:teuthology.orchestra.run.smithi097.stdout:5733 read 0x13e9ff2 thru 0x13f5f1d (0xbf2c bytes) 2024-09-18T00:19:36.179 INFO:teuthology.orchestra.run.smithi097.stdout:5734 read 0x45e6017 thru 0x45f597f (0xf969 bytes) 2024-09-18T00:19:36.179 INFO:teuthology.orchestra.run.smithi097.stdout:5739 write 0x281d00 thru 0x2871a1 (0x54a2 bytes) 2024-09-18T00:19:36.200 INFO:teuthology.orchestra.run.smithi097.stdout:5740 write 0xe043010 thru 0xe04ff6b (0xcf5c bytes) 2024-09-18T00:19:36.207 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:36.209 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:36.205+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:36.224 INFO:teuthology.orchestra.run.smithi097.stdout:5743 trunc from 0xe04ff6c to 0x18b7c10 2024-09-18T00:19:36.263 INFO:teuthology.orchestra.run.smithi097.stdout:5746 write 0x7622085 thru 0x7626ffa (0x4f76 bytes) 2024-09-18T00:19:36.270 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:36.270 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:36.270 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:36.270 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:36.270 INFO:teuthology.orchestra.run.smithi097.stdout:5747 read 0x5f5bc42 thru 0x5f669e9 (0xada8 bytes) 2024-09-18T00:19:36.270 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:36.280 INFO:teuthology.orchestra.run.smithi097.stdout:5749 trunc from 0x7626ffb to 0xa52c33 2024-09-18T00:19:36.294 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:19:36.309 INFO:teuthology.orchestra.run.smithi097.stdout:5750 write 0x15865f7 thru 0x15933d9 (0xcde3 bytes) 2024-09-18T00:19:36.310 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:36.309+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:36.314 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:36.315 INFO:teuthology.orchestra.run.smithi097.stdout:5752 trunc from 0x15933da to 0x1b100bc 2024-09-18T00:19:36.316 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:36.316 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:36.317 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:36.323 INFO:teuthology.orchestra.run.smithi097.stdout:5753 read 0xe5d815 thru 0xe695ff (0xbdeb bytes) 2024-09-18T00:19:36.323 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:36.342 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:36.342 INFO:teuthology.orchestra.run.smithi097.stdout:5755 write 0xa52e4f7 thru 0xa5352ad (0x6db7 bytes) 2024-09-18T00:19:36.345 INFO:teuthology.orchestra.run.smithi097.stdout:5756 write 0x4e7ac27 thru 0x4e7b168 (0x542 bytes) 2024-09-18T00:19:36.354 INFO:teuthology.orchestra.run.smithi097.stdout:5757 trunc from 0xa5352ae to 0x7a660d5 2024-09-18T00:19:36.359 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:36.359 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout:5758 punch from 0x723b067 to 0x724a350, (0xf2e9 bytes) 2024-09-18T00:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout:5759 read 0x6e0b0b1 thru 0x6e0d537 (0x2487 bytes) 2024-09-18T00:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout:5762 read 0x2362de2 thru 0x2372ae0 (0xfcff bytes) 2024-09-18T00:19:36.365 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout:5764 read 0x5e33961 thru 0x5e412cc (0xd96c bytes) 2024-09-18T00:19:36.366 INFO:teuthology.orchestra.run.smithi097.stdout:5765 writesame 0x2c5efbc thru 0x2c6d14a data_size 0x7(0xe18f bytes) 2024-09-18T00:19:36.380 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:36.380 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:36.400 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:36.400 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:36.411 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:36.409+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:36.413 INFO:teuthology.orchestra.run.smithi097.stdout:5766 read 0x76ef5f thru 0x778a01 (0x9aa3 bytes) 2024-09-18T00:19:36.414 INFO:teuthology.orchestra.run.smithi097.stdout:5767 read 0x5fb188d thru 0x5fbf716 (0xde8a bytes) 2024-09-18T00:19:36.414 INFO:teuthology.orchestra.run.smithi097.stdout:5768 punch from 0x47e6ff6 to 0x47ece8b, (0x5e95 bytes) 2024-09-18T00:19:36.420 INFO:teuthology.orchestra.run.smithi097.stdout:5770 trunc from 0x7a660d5 to 0x31cb3b2 2024-09-18T00:19:36.425 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:36.426 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:36.426 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:36.426 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:36.441 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:19:36.449 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:36.449 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:36.450 INFO:teuthology.orchestra.run.smithi097.stdout:5772 write 0x6c22407 thru 0x6c2762f (0x5229 bytes) 2024-09-18T00:19:36.452 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:36.452 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:36.456 INFO:teuthology.orchestra.run.smithi097.stdout:5774 read 0x578e1bc thru 0x579b5df (0xd424 bytes) 2024-09-18T00:19:36.456 INFO:teuthology.orchestra.run.smithi097.stdout:5777 punch from 0x6293d9d to 0x629fad1, (0xbd34 bytes) 2024-09-18T00:19:36.456 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:36.469 INFO:teuthology.orchestra.run.smithi097.stdout:5780 write 0xc7cfa04 thru 0xc7d65ac (0x6ba9 bytes) 2024-09-18T00:19:36.492 INFO:teuthology.orchestra.run.smithi097.stdout:5782 trunc from 0xc7d65ad to 0x237e063 2024-09-18T00:19:36.506 INFO:teuthology.orchestra.run.smithi097.stderr:set 1.0 pg_upmap_items mapping to [2->0,6->1] 2024-09-18T00:19:36.512 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:36.509+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:36.517 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:19:36.545 INFO:teuthology.orchestra.run.smithi097.stdout:5783 write 0xecfc97a thru 0xed0aca4 (0xe32b bytes) 2024-09-18T00:19:36.551 INFO:teuthology.orchestra.run.smithi097.stdout:5784 writesame 0x551bf90 thru 0x552ab12 data_size 0x3(0xeb83 bytes) 2024-09-18T00:19:36.590 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:36.590 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:36.603 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:19:36.630 INFO:teuthology.orchestra.run.smithi097.stdout:5785 clone 27 order 23 su 4194304 sc 3 2024-09-18T00:19:36.630 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:36.629+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:36.714 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:36.713+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:36.742 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:36.742 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:36.757 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:19:36.815 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:36.813+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:36.866 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:36.866 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:36.878 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:19:36.925 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:36.925+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:37.018 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:37.017+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:37.039 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:37.039 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:37.052 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:19:37.119 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:37.117+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:37.183 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:37.184 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:37.198 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:19:37.238 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:37.237+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:37.321 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:37.321+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:37.363 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:37.363 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:37.378 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:19:37.426 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:37.425+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:37.495 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:37.495 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:37.511 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:19:37.512 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is unset 2024-09-18T00:19:37.554 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:37.553+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:37.620 INFO:teuthology.orchestra.run.smithi097.stdout:truncating image image_client.0-clone26 from 0xed0aca5 (overlap 0xa52c33) to 0x529619 2024-09-18T00:19:37.623 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:37.621+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:37.679 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:37.680 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:37.694 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:19:37.724 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:37.721+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:37.741 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-18T00:19:37.818 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:37.819 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:37.825 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:37.821+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:37.833 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:19:37.926 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:37.925+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:38.005 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:38.005 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:38.019 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:19:38.027 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:38.025+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:38.129 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:38.125+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:38.143 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:38.143 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:38.158 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:19:38.229 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:38.225+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:38.278 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:38.278 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:38.292 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:19:38.334 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:38.333+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:38.430 INFO:teuthology.orchestra.run.smithi097.stdout:5786 punch from 0x10b68a2 to 0x10b8601, (0x1d5f bytes) 2024-09-18T00:19:38.431 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:38.429+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:38.448 INFO:teuthology.orchestra.run.smithi097.stdout:5788 trunc from 0xed0aca5 to 0x7accdb9 2024-09-18T00:19:38.453 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:19:38.460 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:38.460 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:38.465 INFO:teuthology.orchestra.run.smithi097.stdout:5789 punch from 0x3b0415 to 0x3b4727, (0x4312 bytes) 2024-09-18T00:19:38.466 INFO:teuthology.orchestra.run.smithi097.stdout:5790 read 0x5cf27e6 thru 0x5cf3bbe (0x13d9 bytes) 2024-09-18T00:19:38.468 INFO:teuthology.orchestra.run.smithi097.stdout:5792 trunc from 0x7accdb9 to 0x8f9d65c 2024-09-18T00:19:38.476 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:19:38.477 INFO:teuthology.orchestra.run.smithi097.stdout:5793 read 0x2424cd5 thru 0x242667b (0x19a7 bytes) 2024-09-18T00:19:38.477 INFO:teuthology.orchestra.run.smithi097.stdout:5794 writesame 0x5366c4a thru 0x536bf2d data_size 0xa(0x52e4 bytes) 2024-09-18T00:19:38.495 INFO:teuthology.orchestra.run.smithi097.stdout:5795 trunc from 0x8f9d65c to 0x3423d43 2024-09-18T00:19:38.520 INFO:teuthology.orchestra.run.smithi097.stdout:5797 punch from 0x1c64df0 to 0x1c748c6, (0xfad6 bytes) 2024-09-18T00:19:38.526 INFO:teuthology.orchestra.run.smithi097.stdout:5798 trunc from 0x3423d43 to 0x721b07a 2024-09-18T00:19:38.550 INFO:teuthology.orchestra.run.smithi097.stdout:5799 write 0xb38b844 thru 0xb398f1d (0xd6da bytes) 2024-09-18T00:19:38.554 INFO:teuthology.orchestra.run.smithi097.stdout:5800 write 0xab74cc7 thru 0xab7b591 (0x68cb bytes) 2024-09-18T00:19:38.559 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:38.559 INFO:teuthology.orchestra.run.smithi097.stdout:5804 punch from 0xde8eb6 to 0xded294, (0x43de bytes) 2024-09-18T00:19:38.559 INFO:teuthology.orchestra.run.smithi097.stdout:5805 punch from 0x9e7d62a to 0x9e848de, (0x72b4 bytes) 2024-09-18T00:19:38.559 INFO:teuthology.orchestra.run.smithi097.stdout:5806 writesame 0x7631235 thru 0x7635142 data_size 0x7(0x3f0e bytes) 2024-09-18T00:19:38.562 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:38.561+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:38.565 INFO:teuthology.orchestra.run.smithi097.stdout:5807 punch from 0x7ee7ee0 to 0x7eec7c7, (0x48e7 bytes) 2024-09-18T00:19:38.565 INFO:teuthology.orchestra.run.smithi097.stdout:5812 writesame 0x4083578 thru 0x408783b data_size 0x2(0x42c4 bytes) 2024-09-18T00:19:38.583 INFO:teuthology.orchestra.run.smithi097.stdout:5813 punch from 0x99d680c to 0x99e1afb, (0xb2ef bytes) 2024-09-18T00:19:38.597 INFO:teuthology.orchestra.run.smithi097.stdout:5814 trunc from 0xb398f1e to 0x244bd83 2024-09-18T00:19:38.615 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:38.615 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:38.615 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:38.625 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:38.626 INFO:teuthology.orchestra.run.smithi097.stdout:5816 read 0x11e2fc5 thru 0x11ebaff (0x8b3b bytes) 2024-09-18T00:19:38.626 INFO:teuthology.orchestra.run.smithi097.stdout:5818 read 0x16a4c52 thru 0x16ab750 (0x6aff bytes) 2024-09-18T00:19:38.632 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:38.629+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:38.634 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:38.635 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:38.635 INFO:teuthology.orchestra.run.smithi097.stdout:5819 writesame 0x2f47386 thru 0x2f4c045 data_size 0x20(0x4cc0 bytes) 2024-09-18T00:19:38.638 INFO:teuthology.orchestra.run.smithi097.stdout:5820 read 0x196c6ca thru 0x197b0b1 (0xe9e8 bytes) 2024-09-18T00:19:38.646 INFO:teuthology.orchestra.run.smithi097.stdout:5821 trunc from 0x2f4c046 to 0x762faea 2024-09-18T00:19:38.650 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:38.650 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:38.655 INFO:teuthology.orchestra.run.smithi097.stdout:5822 write 0xd7a9e5 thru 0xd7e253 (0x386f bytes) 2024-09-18T00:19:38.658 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:38.658 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:38.665 INFO:teuthology.orchestra.run.smithi097.stdout:5825 read 0x47e518c thru 0x47eba67 (0x68dc bytes) 2024-09-18T00:19:38.666 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:19:38.676 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:38.676 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:38.679 INFO:teuthology.orchestra.run.smithi097.stdout:5827 write 0xbffee4c thru 0xc00adc9 (0xbf7e bytes) 2024-09-18T00:19:38.692 INFO:teuthology.orchestra.run.smithi097.stdout:5828 trunc from 0xc00adca to 0x8d73f2a 2024-09-18T00:19:38.695 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:38.695 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:38.707 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:38.707 INFO:teuthology.orchestra.run.smithi097.stdout:5830 read 0x2069f6b thru 0x206e1cd (0x4263 bytes) 2024-09-18T00:19:38.718 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:38.719 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:38.720 INFO:teuthology.orchestra.run.smithi097.stdout:5831 writesame 0xb5c5bd2 thru 0xb5ce5ce data_size 0x4b(0x89fd bytes) 2024-09-18T00:19:38.720 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:38.720 INFO:teuthology.orchestra.run.smithi097.stdout:5835 write 0x32c9808 thru 0x32d16c0 (0x7eb9 bytes) 2024-09-18T00:19:38.734 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:38.733+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:38.743 INFO:teuthology.orchestra.run.smithi097.stdout:5836 trunc from 0xb5ce5cf to 0x154a6ec 2024-09-18T00:19:38.773 INFO:teuthology.orchestra.run.smithi097.stdout:5838 write 0x2dbf8bb thru 0x2dcbafc (0xc242 bytes) 2024-09-18T00:19:38.778 INFO:teuthology.orchestra.run.smithi097.stdout:5839 punch from 0x183196a to 0x1837809, (0x5e9f bytes) 2024-09-18T00:19:38.779 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:38.779 INFO:teuthology.orchestra.run.smithi097.stdout:5843 punch from 0x1abbc14 to 0x1ac79e8, (0xbdd4 bytes) 2024-09-18T00:19:38.779 INFO:teuthology.orchestra.run.smithi097.stdout:5844 read 0xf34ffb thru 0xf42995 (0xd99b bytes) 2024-09-18T00:19:38.779 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:38.789 INFO:teuthology.orchestra.run.smithi097.stdout:5846 trunc from 0x2dcbafd to 0x8e39450 2024-09-18T00:19:38.797 INFO:teuthology.orchestra.run.smithi097.stdout:5847 writesame 0x41d6742 thru 0x41d8501 data_size 0x1c(0x1dc0 bytes) 2024-09-18T00:19:38.802 INFO:teuthology.orchestra.run.smithi097.stdout:5848 read 0x246df5 thru 0x255f31 (0xf13d bytes) 2024-09-18T00:19:38.803 INFO:teuthology.orchestra.run.smithi097.stdout:5849 write 0x1b6c3b8 thru 0x1b6ff72 (0x3bbb bytes) 2024-09-18T00:19:38.811 INFO:teuthology.orchestra.run.smithi097.stdout:5850 read 0x15cea59 thru 0x15da0df (0xb687 bytes) 2024-09-18T00:19:38.824 INFO:teuthology.orchestra.run.smithi097.stdout:5851 trunc from 0x8e39450 to 0xed5a58c 2024-09-18T00:19:38.834 INFO:teuthology.orchestra.run.smithi097.stdout:5852 punch from 0x31d2aff to 0x31d7c28, (0x5129 bytes) 2024-09-18T00:19:38.834 INFO:teuthology.orchestra.run.smithi097.stdout:5854 writesame 0xa7b03e7 thru 0xa7bacbb data_size 0x1(0xa8d5 bytes) 2024-09-18T00:19:38.835 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:38.833+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:38.842 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:38.842 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:38.842 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:38.842 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:38.842 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:38.856 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:19:38.937 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:38.937+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:38.940 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:38.940 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:38.944 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:38.945 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:38.957 INFO:teuthology.orchestra.run.smithi097.stdout:5855 punch from 0xaf22924 to 0xaf2e34b, (0xba27 bytes) 2024-09-18T00:19:38.957 INFO:teuthology.orchestra.run.smithi097.stdout:5856 punch from 0xab548b0 to 0xab5d566, (0x8cb6 bytes) 2024-09-18T00:19:38.957 INFO:teuthology.orchestra.run.smithi097.stdout:5859 read 0x8cd124 thru 0x8ce150 (0x102d bytes) 2024-09-18T00:19:38.958 INFO:teuthology.orchestra.run.smithi097.stdout:5860 writesame 0x8fa96c4 thru 0x8fb580b data_size 0x14(0xc148 bytes) 2024-09-18T00:19:38.965 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:38.965 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:38.965 INFO:teuthology.orchestra.run.smithi097.stdout:5864 read 0x994844a thru 0x994c59b (0x4152 bytes) 2024-09-18T00:19:38.965 INFO:teuthology.orchestra.run.smithi097.stdout:5865 punch from 0x85b9810 to 0x85bad21, (0x1511 bytes) 2024-09-18T00:19:38.966 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:38.966 INFO:teuthology.orchestra.run.smithi097.stdout:5869 writesame 0xc289ca4 thru 0xc28f8a7 data_size 0x27(0x5c04 bytes) 2024-09-18T00:19:38.971 INFO:teuthology.orchestra.run.smithi097.stdout:5870 writesame 0x9e3f944 thru 0x9e4f0a5 data_size 0xf(0xf762 bytes) 2024-09-18T00:19:38.983 INFO:teuthology.orchestra.run.smithi097.stdout:5872 writesame 0x458458f thru 0x4585fd8 data_size 0x5(0x1a4a bytes) 2024-09-18T00:19:38.993 INFO:teuthology.orchestra.run.smithi097.stdout:5873 trunc from 0xed5a58c to 0xc970cbd 2024-09-18T00:19:39.008 INFO:teuthology.orchestra.run.smithi097.stdout:5874 read 0x38f8059 thru 0x390482b (0xc7d3 bytes) 2024-09-18T00:19:39.029 INFO:teuthology.orchestra.run.smithi097.stdout:5875 trunc from 0xc970cbd to 0x52c3aad 2024-09-18T00:19:39.031 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.031 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34543 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:19:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.735838119, 2024-09-18T00:19:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023207593, 2024-09-18T00:19:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34543 2024-09-18T00:19:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.032 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:19:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:19:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290783+0000", 2024-09-18T00:19:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5240000000000001e-06 2024-09-18T00:19:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290799+0000", 2024-09-18T00:19:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:19:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290840+0000", 2024-09-18T00:19:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0926000000000001e-05 2024-09-18T00:19:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290868+0000", 2024-09-18T00:19:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8163000000000001e-05 2024-09-18T00:19:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291294+0000", 2024-09-18T00:19:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00042588700000000001 2024-09-18T00:19:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:19:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:19:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.986e-05 2024-09-18T00:19:39.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316617+0000", 2024-09-18T00:19:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025303583000000001 2024-09-18T00:19:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316637+0000", 2024-09-18T00:19:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9388000000000001e-05 2024-09-18T00:19:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313933+0000", 2024-09-18T00:19:39.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99729681000000003 2024-09-18T00:19:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313987+0000", 2024-09-18T00:19:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4069e-05 2024-09-18T00:19:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34544 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:19:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.735689764, 2024-09-18T00:19:39.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0232412120000001, 2024-09-18T00:19:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34544 2024-09-18T00:19:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:19:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:19:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290930+0000", 2024-09-18T00:19:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5030000000000001e-06 2024-09-18T00:19:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290940+0000", 2024-09-18T00:19:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9000000000000001e-06 2024-09-18T00:19:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290964+0000", 2024-09-18T00:19:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4273e-05 2024-09-18T00:19:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290982+0000", 2024-09-18T00:19:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8216e-05 2024-09-18T00:19:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291433+0000", 2024-09-18T00:19:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000451278 2024-09-18T00:19:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291446+0000", 2024-09-18T00:19:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3039e-05 2024-09-18T00:19:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316687+0000", 2024-09-18T00:19:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025240069 2024-09-18T00:19:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316701+0000", 2024-09-18T00:19:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4100000000000001e-05 2024-09-18T00:19:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314142+0000", 2024-09-18T00:19:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99744119799999997 2024-09-18T00:19:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314169+0000", 2024-09-18T00:19:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7636000000000001e-05 2024-09-18T00:19:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34545 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:19:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.735611242000001, 2024-09-18T00:19:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023257802, 2024-09-18T00:19:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34545 2024-09-18T00:19:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:19:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:19:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291008+0000", 2024-09-18T00:19:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.237e-06 2024-09-18T00:19:39.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291015+0000", 2024-09-18T00:19:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2030000000000003e-06 2024-09-18T00:19:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291035+0000", 2024-09-18T00:19:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.978e-05 2024-09-18T00:19:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291049+0000", 2024-09-18T00:19:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4429e-05 2024-09-18T00:19:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291630+0000", 2024-09-18T00:19:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00058103199999999999 2024-09-18T00:19:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291644+0000", 2024-09-18T00:19:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.342e-05 2024-09-18T00:19:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316753+0000", 2024-09-18T00:19:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025108703999999999 2024-09-18T00:19:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316766+0000", 2024-09-18T00:19:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3237e-05 2024-09-18T00:19:39.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314238+0000", 2024-09-18T00:19:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99747232699999999 2024-09-18T00:19:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314265+0000", 2024-09-18T00:19:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6432999999999999e-05 2024-09-18T00:19:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.048 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34546 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:19:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.735552902999999, 2024-09-18T00:19:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023324535, 2024-09-18T00:19:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.049 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34546 2024-09-18T00:19:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:19:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:19:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291066+0000", 2024-09-18T00:19:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.569999999999999e-07 2024-09-18T00:19:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291073+0000", 2024-09-18T00:19:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.063e-06 2024-09-18T00:19:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291090+0000", 2024-09-18T00:19:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6977e-05 2024-09-18T00:19:39.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291105+0000", 2024-09-18T00:19:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5166e-05 2024-09-18T00:19:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291771+0000", 2024-09-18T00:19:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00066599699999999999 2024-09-18T00:19:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:39.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291785+0000", 2024-09-18T00:19:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.327e-05 2024-09-18T00:19:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316826+0000", 2024-09-18T00:19:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025041955000000001 2024-09-18T00:19:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316840+0000", 2024-09-18T00:19:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3256e-05 2024-09-18T00:19:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314335+0000", 2024-09-18T00:19:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99749496100000001 2024-09-18T00:19:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314390+0000", 2024-09-18T00:19:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4932999999999997e-05 2024-09-18T00:19:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.054 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34547 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:19:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.735493603999998, 2024-09-18T00:19:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.028510467, 2024-09-18T00:19:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34547 2024-09-18T00:19:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:19:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:19:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291126+0000", 2024-09-18T00:19:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2699999999999999e-06 2024-09-18T00:19:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291150+0000", 2024-09-18T00:19:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4669999999999999e-05 2024-09-18T00:19:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291171+0000", 2024-09-18T00:19:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0706000000000001e-05 2024-09-18T00:19:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291186+0000", 2024-09-18T00:19:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5126999999999999e-05 2024-09-18T00:19:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291885+0000", 2024-09-18T00:19:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00069850299999999995 2024-09-18T00:19:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291897+0000", 2024-09-18T00:19:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-05 2024-09-18T00:19:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316887+0000", 2024-09-18T00:19:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024990155999999999 2024-09-18T00:19:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316900+0000", 2024-09-18T00:19:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.295e-05 2024-09-18T00:19:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319594+0000", 2024-09-18T00:19:39.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0026940719999999 2024-09-18T00:19:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319635+0000", 2024-09-18T00:19:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0583000000000001e-05 2024-09-18T00:19:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34548 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:19:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.735346146000001, 2024-09-18T00:19:39.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0285287400000001, 2024-09-18T00:19:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34548 2024-09-18T00:19:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:19:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:19:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291274+0000", 2024-09-18T00:19:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.12e-06 2024-09-18T00:19:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291288+0000", 2024-09-18T00:19:39.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4367e-05 2024-09-18T00:19:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:19:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5698999999999999e-05 2024-09-18T00:19:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291333+0000", 2024-09-18T00:19:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8783e-05 2024-09-18T00:19:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291987+0000", 2024-09-18T00:19:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00065417800000000005 2024-09-18T00:19:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.292000+0000", 2024-09-18T00:19:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2652e-05 2024-09-18T00:19:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316946+0000", 2024-09-18T00:19:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024946696000000001 2024-09-18T00:19:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316956+0000", 2024-09-18T00:19:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2760000000000006e-06 2024-09-18T00:19:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319752+0000", 2024-09-18T00:19:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002796083 2024-09-18T00:19:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319801+0000", 2024-09-18T00:19:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8885999999999999e-05 2024-09-18T00:19:39.065 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41949 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693852+0000", 2024-09-18T00:19:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.332766339999999, 2024-09-18T00:19:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053245853000000003, 2024-09-18T00:19:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.066 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41949 2024-09-18T00:19:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693852+0000", 2024-09-18T00:19:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693852+0000", 2024-09-18T00:19:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693853+0000", 2024-09-18T00:19:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2999999999999999e-07 2024-09-18T00:19:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693866+0000", 2024-09-18T00:19:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3814e-05 2024-09-18T00:19:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693878+0000", 2024-09-18T00:19:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1384e-05 2024-09-18T00:19:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693893+0000", 2024-09-18T00:19:39.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4722000000000001e-05 2024-09-18T00:19:39.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.746958+0000", 2024-09-18T00:19:39.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053065734000000003 2024-09-18T00:19:39.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:39.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747015+0000", 2024-09-18T00:19:39.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6397000000000002e-05 2024-09-18T00:19:39.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747098+0000", 2024-09-18T00:19:39.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2872000000000002e-05 2024-09-18T00:19:39.070 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.070 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.070 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.070 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41950 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.070 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:19:39.070 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.332693437, 2024-09-18T00:19:39.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053368486, 2024-09-18T00:19:39.071 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.071 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.071 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:39.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.072 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41950 2024-09-18T00:19:39.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.072 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:19:39.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:19:39.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693926+0000", 2024-09-18T00:19:39.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4900000000000004e-07 2024-09-18T00:19:39.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693930+0000", 2024-09-18T00:19:39.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0130000000000004e-06 2024-09-18T00:19:39.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693940+0000", 2024-09-18T00:19:39.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0618000000000001e-05 2024-09-18T00:19:39.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693948+0000", 2024-09-18T00:19:39.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3140000000000004e-06 2024-09-18T00:19:39.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747133+0000", 2024-09-18T00:19:39.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053184689 2024-09-18T00:19:39.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:39.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747203+0000", 2024-09-18T00:19:39.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0339000000000005e-05 2024-09-18T00:19:39.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747293+0000", 2024-09-18T00:19:39.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9563999999999998e-05 2024-09-18T00:19:39.075 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.075 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.075 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.075 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41951 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.075 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:19:39.076 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.332658670000001, 2024-09-18T00:19:39.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053527695, 2024-09-18T00:19:39.076 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.076 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.076 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:39.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.076 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41951 2024-09-18T00:19:39.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.076 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:19:39.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:19:39.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693960+0000", 2024-09-18T00:19:39.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.16e-07 2024-09-18T00:19:39.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693971+0000", 2024-09-18T00:19:39.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1204e-05 2024-09-18T00:19:39.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693981+0000", 2024-09-18T00:19:39.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-18T00:19:39.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693989+0000", 2024-09-18T00:19:39.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0180000000000001e-06 2024-09-18T00:19:39.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747328+0000", 2024-09-18T00:19:39.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053338508 2024-09-18T00:19:39.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:39.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747400+0000", 2024-09-18T00:19:39.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2750999999999995e-05 2024-09-18T00:19:39.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747487+0000", 2024-09-18T00:19:39.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6527999999999996e-05 2024-09-18T00:19:39.080 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.080 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.080 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.080 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41952 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.080 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:19:39.080 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.332618263000001, 2024-09-18T00:19:39.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053728013999999998, 2024-09-18T00:19:39.080 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.080 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.080 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:39.080 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.080 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.081 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.081 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41952 2024-09-18T00:19:39.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.081 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:19:39.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:19:39.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694001+0000", 2024-09-18T00:19:39.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3799999999999996e-07 2024-09-18T00:19:39.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694004+0000", 2024-09-18T00:19:39.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.965e-06 2024-09-18T00:19:39.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694013+0000", 2024-09-18T00:19:39.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1689999999999998e-06 2024-09-18T00:19:39.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694022+0000", 2024-09-18T00:19:39.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7560000000000005e-06 2024-09-18T00:19:39.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747531+0000", 2024-09-18T00:19:39.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053508999000000002 2024-09-18T00:19:39.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:39.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747610+0000", 2024-09-18T00:19:39.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9212e-05 2024-09-18T00:19:39.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747728+0000", 2024-09-18T00:19:39.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011797499999999999 2024-09-18T00:19:39.084 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.084 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.084 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.084 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41953 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.084 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:19:39.084 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.332584634, 2024-09-18T00:19:39.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053936783000000002, 2024-09-18T00:19:39.085 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.085 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.085 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:39.085 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.085 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.085 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.085 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41953 2024-09-18T00:19:39.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.085 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:19:39.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:19:39.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694034+0000", 2024-09-18T00:19:39.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5199999999999996e-07 2024-09-18T00:19:39.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694037+0000", 2024-09-18T00:19:39.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.745e-06 2024-09-18T00:19:39.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694046+0000", 2024-09-18T00:19:39.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2490000000000007e-06 2024-09-18T00:19:39.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694054+0000", 2024-09-18T00:19:39.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1179999999999992e-06 2024-09-18T00:19:39.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747766+0000", 2024-09-18T00:19:39.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053711362999999998 2024-09-18T00:19:39.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:39.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747856+0000", 2024-09-18T00:19:39.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0779000000000006e-05 2024-09-18T00:19:39.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747970+0000", 2024-09-18T00:19:39.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011377700000000001 2024-09-18T00:19:39.088 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.088 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.088 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.088 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41954 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.089 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:39.089 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.332554480999999, 2024-09-18T00:19:39.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054104898999999998, 2024-09-18T00:19:39.089 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.089 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.089 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:39.089 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.089 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.089 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.089 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41954 2024-09-18T00:19:39.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.089 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:39.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:39.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:39.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4000000000000002e-07 2024-09-18T00:19:39.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694067+0000", 2024-09-18T00:19:39.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7329999999999999e-06 2024-09-18T00:19:39.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694076+0000", 2024-09-18T00:19:39.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2019999999999993e-06 2024-09-18T00:19:39.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694083+0000", 2024-09-18T00:19:39.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0029999999999997e-06 2024-09-18T00:19:39.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748003+0000", 2024-09-18T00:19:39.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053920463000000002 2024-09-18T00:19:39.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:39.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748077+0000", 2024-09-18T00:19:39.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4041000000000005e-05 2024-09-18T00:19:39.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748168+0000", 2024-09-18T00:19:39.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0916999999999997e-05 2024-09-18T00:19:39.093 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.093 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.093 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.093 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41955 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 231424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.093 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:19:39.093 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.332525643, 2024-09-18T00:19:39.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054272659000000001, 2024-09-18T00:19:39.093 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.093 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.094 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:39.094 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.094 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.094 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.094 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41955 2024-09-18T00:19:39.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.094 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:19:39.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:19:39.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694093+0000", 2024-09-18T00:19:39.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:19:39.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694109+0000", 2024-09-18T00:19:39.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6061000000000001e-05 2024-09-18T00:19:39.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694121+0000", 2024-09-18T00:19:39.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.216e-05 2024-09-18T00:19:39.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694129+0000", 2024-09-18T00:19:39.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0029999999999994e-06 2024-09-18T00:19:39.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748202+0000", 2024-09-18T00:19:39.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054072571 2024-09-18T00:19:39.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:39.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748274+0000", 2024-09-18T00:19:39.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2117000000000001e-05 2024-09-18T00:19:39.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748365+0000", 2024-09-18T00:19:39.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1139999999999998e-05 2024-09-18T00:19:39.098 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.098 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.098 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.098 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41956 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.098 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:39.098 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.332477435000001, 2024-09-18T00:19:39.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054420496999999998, 2024-09-18T00:19:39.098 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.098 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.098 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:39.098 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.098 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.099 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.099 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41956 2024-09-18T00:19:39.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.099 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:39.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:39.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:39.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8699999999999995e-07 2024-09-18T00:19:39.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694144+0000", 2024-09-18T00:19:39.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3510000000000002e-06 2024-09-18T00:19:39.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694154+0000", 2024-09-18T00:19:39.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0769999999999997e-06 2024-09-18T00:19:39.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694163+0000", 2024-09-18T00:19:39.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3300000000000005e-06 2024-09-18T00:19:39.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748397+0000", 2024-09-18T00:19:39.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054234497999999999 2024-09-18T00:19:39.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:39.102 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748471+0000", 2024-09-18T00:19:39.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3441000000000004e-05 2024-09-18T00:19:39.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.102 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.102 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748561+0000", 2024-09-18T00:19:39.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0212999999999998e-05 2024-09-18T00:19:39.102 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.102 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.102 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.103 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41957 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.103 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:19:39.103 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.332446182999998, 2024-09-18T00:19:39.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054590310000000003, 2024-09-18T00:19:39.103 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.103 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.103 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:39.103 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.103 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.103 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.103 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41957 2024-09-18T00:19:39.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.104 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:19:39.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:19:39.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694173+0000", 2024-09-18T00:19:39.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2500000000000005e-07 2024-09-18T00:19:39.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694175+0000", 2024-09-18T00:19:39.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.655e-06 2024-09-18T00:19:39.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694184+0000", 2024-09-18T00:19:39.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7630000000000006e-06 2024-09-18T00:19:39.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694191+0000", 2024-09-18T00:19:39.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8870000000000003e-06 2024-09-18T00:19:39.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748593+0000", 2024-09-18T00:19:39.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054402525 2024-09-18T00:19:39.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:39.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748669+0000", 2024-09-18T00:19:39.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5532999999999997e-05 2024-09-18T00:19:39.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748762+0000", 2024-09-18T00:19:39.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3221999999999996e-05 2024-09-18T00:19:39.107 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.107 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.107 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.107 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42391 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 337920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.107 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707180+0000", 2024-09-18T00:19:39.107 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.319438431000002, 2024-09-18T00:19:39.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053238496000000003, 2024-09-18T00:19:39.107 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.107 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.107 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:39.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.108 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42391 2024-09-18T00:19:39.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.108 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707180+0000", 2024-09-18T00:19:39.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707180+0000", 2024-09-18T00:19:39.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707180+0000", 2024-09-18T00:19:39.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6400000000000003e-07 2024-09-18T00:19:39.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707182+0000", 2024-09-18T00:19:39.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0870000000000001e-06 2024-09-18T00:19:39.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707189+0000", 2024-09-18T00:19:39.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7469999999999999e-06 2024-09-18T00:19:39.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707194+0000", 2024-09-18T00:19:39.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9459999999999997e-06 2024-09-18T00:19:39.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760268+0000", 2024-09-18T00:19:39.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053073899000000001 2024-09-18T00:19:39.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:39.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760351+0000", 2024-09-18T00:19:39.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3604999999999997e-05 2024-09-18T00:19:39.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760418+0000", 2024-09-18T00:19:39.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6748000000000005e-05 2024-09-18T00:19:39.111 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.111 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.111 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.111 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42392 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.111 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:19:39.111 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.319416744000002, 2024-09-18T00:19:39.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053378162, 2024-09-18T00:19:39.112 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.112 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.112 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:39.112 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.112 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.112 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.112 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42392 2024-09-18T00:19:39.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.112 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:19:39.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:19:39.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707430+0000", 2024-09-18T00:19:39.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022838599999999999 2024-09-18T00:19:39.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707434+0000", 2024-09-18T00:19:39.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5220000000000001e-06 2024-09-18T00:19:39.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707467+0000", 2024-09-18T00:19:39.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2570000000000002e-05 2024-09-18T00:19:39.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707475+0000", 2024-09-18T00:19:39.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:19:39.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760446+0000", 2024-09-18T00:19:39.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052971635000000003 2024-09-18T00:19:39.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:39.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760512+0000", 2024-09-18T00:19:39.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5679999999999995e-05 2024-09-18T00:19:39.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760579+0000", 2024-09-18T00:19:39.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7296000000000001e-05 2024-09-18T00:19:39.116 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.116 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.116 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.116 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42393 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 358400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.116 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:19:39.116 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.319127793, 2024-09-18T00:19:39.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053269618999999997, 2024-09-18T00:19:39.116 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.116 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.117 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:39.117 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.117 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.117 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.117 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42393 2024-09-18T00:19:39.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.117 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:19:39.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:19:39.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707491+0000", 2024-09-18T00:19:39.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1399999999999997e-07 2024-09-18T00:19:39.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707503+0000", 2024-09-18T00:19:39.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1912e-05 2024-09-18T00:19:39.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707512+0000", 2024-09-18T00:19:39.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1030000000000008e-06 2024-09-18T00:19:39.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707520+0000", 2024-09-18T00:19:39.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1249999999999993e-06 2024-09-18T00:19:39.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760605+0000", 2024-09-18T00:19:39.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053085495000000003 2024-09-18T00:19:39.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:39.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760680+0000", 2024-09-18T00:19:39.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5024000000000005e-05 2024-09-18T00:19:39.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760760+0000", 2024-09-18T00:19:39.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9345999999999996e-05 2024-09-18T00:19:39.120 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.120 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.121 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.121 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42394 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.121 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:19:39.121 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.319087804999999, 2024-09-18T00:19:39.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053403144999999999, 2024-09-18T00:19:39.121 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.121 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.121 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:39.121 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.121 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.121 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.121 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42394 2024-09-18T00:19:39.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.122 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:19:39.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:19:39.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711550+0000", 2024-09-18T00:19:39.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040200189999999997 2024-09-18T00:19:39.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711556+0000", 2024-09-18T00:19:39.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2339999999999999e-06 2024-09-18T00:19:39.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711569+0000", 2024-09-18T00:19:39.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2337999999999999e-05 2024-09-18T00:19:39.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711577+0000", 2024-09-18T00:19:39.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5660000000000007e-06 2024-09-18T00:19:39.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760790+0000", 2024-09-18T00:19:39.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049212196999999999 2024-09-18T00:19:39.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:39.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760857+0000", 2024-09-18T00:19:39.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7756000000000007e-05 2024-09-18T00:19:39.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760933+0000", 2024-09-18T00:19:39.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6035e-05 2024-09-18T00:19:39.125 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.125 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.125 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.126 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:55489 3.5 3:a5f1e468:::rbd_data.112a859a01c9.000000000000000c:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T00:19:39.126 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:19:39.126 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.3952336790000004, 2024-09-18T00:19:39.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.05748731, 2024-09-18T00:19:39.126 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:39.126 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:39.126 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:39.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:39.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:39.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.126 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 55489 2024-09-18T00:19:39.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.127 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:39.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:39.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:19:39.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:39.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:19:39.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:39.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:39.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631388+0000", 2024-09-18T00:19:39.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6169999999999998e-06 2024-09-18T00:19:39.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:39.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631405+0000", 2024-09-18T00:19:39.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6643e-05 2024-09-18T00:19:39.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:39.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631444+0000", 2024-09-18T00:19:39.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9226e-05 2024-09-18T00:19:39.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631465+0000", 2024-09-18T00:19:39.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0826999999999999e-05 2024-09-18T00:19:39.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:39.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631578+0000", 2024-09-18T00:19:39.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011291899999999999 2024-09-18T00:19:39.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:39.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631651+0000", 2024-09-18T00:19:39.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3264999999999997e-05 2024-09-18T00:19:39.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:19:39.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.634073+0000", 2024-09-18T00:19:39.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0024221350000000002 2024-09-18T00:19:39.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:39.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688732+0000", 2024-09-18T00:19:39.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054658610000000003 2024-09-18T00:19:39.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:39.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688794+0000", 2024-09-18T00:19:39.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2255999999999995e-05 2024-09-18T00:19:39.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:39.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688834+0000", 2024-09-18T00:19:39.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9916999999999997e-05 2024-09-18T00:19:39.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:39.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:39.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:39.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688872+0000", 2024-09-18T00:19:39.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7895000000000002e-05 2024-09-18T00:19:39.132 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.132 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.132 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.132 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:39.132 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:39.132 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:39.133 INFO:teuthology.orchestra.run.smithi097.stdout:5878 read 0x3fca62 thru 0x4085bd (0xbb5c bytes) 2024-09-18T00:19:39.133 INFO:teuthology.orchestra.run.smithi097.stdout:5879 writesame 0x3175c6 thru 0x31ecdc data_size 0x1(0x7717 bytes) 2024-09-18T00:19:39.133 INFO:teuthology.orchestra.run.smithi097.stdout:5880 read 0x1ac8acf thru 0x1acce3b (0x436d bytes) 2024-09-18T00:19:39.134 INFO:teuthology.orchestra.run.smithi097.stdout:5881 write 0x12dac29 thru 0x12e9d88 (0xf160 bytes) 2024-09-18T00:19:39.134 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:39.133+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:39.134 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:19:39.161 INFO:teuthology.orchestra.run.smithi097.stdout:5882 write 0x72d3c92 thru 0x72d876d (0x4adc bytes) 2024-09-18T00:19:39.168 INFO:teuthology.orchestra.run.smithi097.stdout:5883 read 0x41f1927 thru 0x4200025 (0xe6ff bytes) 2024-09-18T00:19:39.182 INFO:teuthology.orchestra.run.smithi097.stdout:5884 trunc from 0x72d876e to 0xd953c59 2024-09-18T00:19:39.193 INFO:teuthology.orchestra.run.smithi097.stdout:5887 write 0x5a08b3 thru 0x5a6ed6 (0x6624 bytes) 2024-09-18T00:19:39.198 INFO:teuthology.orchestra.run.smithi097.stdout:5889 write 0x51fb99 thru 0x524f7d (0x53e5 bytes) 2024-09-18T00:19:39.203 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:39.203 INFO:teuthology.orchestra.run.smithi097.stdout:5892 punch from 0xd7444fc to 0xd74fcd1, (0xb7d5 bytes) 2024-09-18T00:19:39.203 INFO:teuthology.orchestra.run.smithi097.stdout:5893 writesame 0x2e7733c thru 0x2e7eebd data_size 0x1(0x7b82 bytes) 2024-09-18T00:19:39.234 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:39.233+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:39.266 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:39.267 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:39.267 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:39.267 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:39.280 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:19:39.305 INFO:teuthology.orchestra.run.smithi097.stdout:5895 clone 28 order 18 su 65536 sc 4 2024-09-18T00:19:39.336 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:39.333+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:39.426 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:39.427 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:39.427 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:39.436 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:39.433+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:39.440 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:39.537 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:39.537+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:39.541 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2230 3.17 3:ef492888:::rbd_data.112aa7ba4578.000000000000014f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:39.617 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 79.348453974999998, 2024-09-18T00:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075065283999999996, 2024-09-18T00:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2230 2024-09-18T00:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.618 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266014+0000", 2024-09-18T00:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.619 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266029+0000", 2024-09-18T00:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4616e-05 2024-09-18T00:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266053+0000", 2024-09-18T00:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4369999999999999e-05 2024-09-18T00:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266067+0000", 2024-09-18T00:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4243999999999999e-05 2024-09-18T00:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.620 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266157+0000", 2024-09-18T00:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9909999999999998e-05 2024-09-18T00:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266327+0000", 2024-09-18T00:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016981200000000001 2024-09-18T00:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269180+0000", 2024-09-18T00:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028526559999999999 2024-09-18T00:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.621 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340955+0000", 2024-09-18T00:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.071774904 2024-09-18T00:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340974+0000", 2024-09-18T00:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8824000000000001e-05 2024-09-18T00:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341028+0000", 2024-09-18T00:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4456999999999997e-05 2024-09-18T00:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.622 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341076+0000", 2024-09-18T00:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8284000000000001e-05 2024-09-18T00:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2230 3.17 e36/23 ondisk, result = 0)", 2024-09-18T00:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 79.331110394000007, 2024-09-18T00:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057609645000000001, 2024-09-18T00:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.623 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283356+0000", 2024-09-18T00:19:39.624 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.44e-06 2024-09-18T00:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283362+0000", 2024-09-18T00:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2539999999999998e-06 2024-09-18T00:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283382+0000", 2024-09-18T00:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9896000000000001e-05 2024-09-18T00:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.625 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283398+0000", 2024-09-18T00:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5489999999999999e-05 2024-09-18T00:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340936+0000", 2024-09-18T00:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057538649999999997 2024-09-18T00:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340947+0000", 2024-09-18T00:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0981e-05 2024-09-18T00:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.626 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340964+0000", 2024-09-18T00:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6934000000000001e-05 2024-09-18T00:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2419 3.17 3:ed94b133:::rbd_data.112aa7ba4578.00000000000001b5:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:39.627 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.876988921000006, 2024-09-18T00:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.064761812000000002, 2024-09-18T00:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2419 2024-09-18T00:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.628 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737479+0000", 2024-09-18T00:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6089999999999999e-06 2024-09-18T00:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.629 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737487+0000", 2024-09-18T00:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7900000000000005e-06 2024-09-18T00:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737507+0000", 2024-09-18T00:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9936e-05 2024-09-18T00:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737518+0000", 2024-09-18T00:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1046999999999999e-05 2024-09-18T00:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.630 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737599+0000", 2024-09-18T00:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0697000000000003e-05 2024-09-18T00:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737697+0000", 2024-09-18T00:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8058000000000002e-05 2024-09-18T00:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.740445+0000", 2024-09-18T00:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027481279999999999 2024-09-18T00:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:39.631 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802023+0000", 2024-09-18T00:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.061577321999999997 2024-09-18T00:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802060+0000", 2024-09-18T00:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7586000000000002e-05 2024-09-18T00:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802158+0000", 2024-09-18T00:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7583000000000004e-05 2024-09-18T00:19:39.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802238+0000", 2024-09-18T00:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0055999999999994e-05 2024-09-18T00:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.633 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2419 3.17 e37/23 ondisk, result = 0)", 2024-09-18T00:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.864763975000002, 2024-09-18T00:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052342948, 2024-09-18T00:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.634 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749702+0000", 2024-09-18T00:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749710+0000", 2024-09-18T00:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7279999999999995e-06 2024-09-18T00:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.635 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749736+0000", 2024-09-18T00:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5797999999999999e-05 2024-09-18T00:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749751+0000", 2024-09-18T00:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5238999999999999e-05 2024-09-18T00:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.636 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801745+0000", 2024-09-18T00:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051994022000000001 2024-09-18T00:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801770+0000", 2024-09-18T00:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5103e-05 2024-09-18T00:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802044+0000", 2024-09-18T00:19:39.637 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027395799999999998 2024-09-18T00:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6503 3.1b 3:d9387a1f:::rbd_data.112a3a39fc45.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.050091004999999, 2024-09-18T00:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050510547000000003, 2024-09-18T00:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:39.638 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6503 2024-09-18T00:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:19:39.639 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564376+0000", 2024-09-18T00:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-06 2024-09-18T00:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564386+0000", 2024-09-18T00:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0312999999999999e-05 2024-09-18T00:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.640 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564408+0000", 2024-09-18T00:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2001000000000002e-05 2024-09-18T00:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564421+0000", 2024-09-18T00:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2271e-05 2024-09-18T00:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564491+0000", 2024-09-18T00:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0630000000000006e-05 2024-09-18T00:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.641 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564606+0000", 2024-09-18T00:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011449 2024-09-18T00:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.566743+0000", 2024-09-18T00:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021373830000000001 2024-09-18T00:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614738+0000", 2024-09-18T00:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047995359000000001 2024-09-18T00:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.642 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614759+0000", 2024-09-18T00:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1010999999999999e-05 2024-09-18T00:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614826+0000", 2024-09-18T00:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6654000000000003e-05 2024-09-18T00:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614884+0000", 2024-09-18T00:19:39.643 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8332999999999999e-05 2024-09-18T00:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6518 3.1 3:8061bf9f:::rbd_data.112a3a39fc45.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 73.963652730999996, 2024-09-18T00:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062303123000000002, 2024-09-18T00:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:39.644 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6518 2024-09-18T00:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.645 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650814+0000", 2024-09-18T00:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.629e-06 2024-09-18T00:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650823+0000", 2024-09-18T00:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1800000000000002e-06 2024-09-18T00:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650843+0000", 2024-09-18T00:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9836000000000001e-05 2024-09-18T00:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650854+0000", 2024-09-18T00:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1564e-05 2024-09-18T00:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650917+0000", 2024-09-18T00:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3076000000000004e-05 2024-09-18T00:19:39.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.651029+0000", 2024-09-18T00:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111154 2024-09-18T00:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.655531+0000", 2024-09-18T00:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045027629999999999 2024-09-18T00:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:39.648 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712924+0000", 2024-09-18T00:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057393006000000003 2024-09-18T00:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712988+0000", 2024-09-18T00:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000005e-05 2024-09-18T00:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713056+0000", 2024-09-18T00:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7904999999999994e-05 2024-09-18T00:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.649 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713115+0000", 2024-09-18T00:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9339999999999998e-05 2024-09-18T00:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6518 3.1 e42/23 ondisk, result = 0)", 2024-09-18T00:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 73.951227648, 2024-09-18T00:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049887304, 2024-09-18T00:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.650 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663239+0000", 2024-09-18T00:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.651 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663246+0000", 2024-09-18T00:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4189999999999996e-06 2024-09-18T00:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663265+0000", 2024-09-18T00:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8323000000000001e-05 2024-09-18T00:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663277+0000", 2024-09-18T00:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2065999999999999e-05 2024-09-18T00:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.652 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712965+0000", 2024-09-18T00:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049688317000000003 2024-09-18T00:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712977+0000", 2024-09-18T00:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.206e-05 2024-09-18T00:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713125+0000", 2024-09-18T00:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014749899999999999 2024-09-18T00:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.653 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:8831 3.12 3:49cac8cf:::rbd_data.112a9f4259df.0000000000000130:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T00:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.333367398999997, 2024-09-18T00:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104125047, 2024-09-18T00:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 8831 2024-09-18T00:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.654 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281099+0000", 2024-09-18T00:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.029e-06 2024-09-18T00:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.655 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281103+0000", 2024-09-18T00:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7759999999999997e-06 2024-09-18T00:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281114+0000", 2024-09-18T00:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1090000000000001e-05 2024-09-18T00:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281120+0000", 2024-09-18T00:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.604e-06 2024-09-18T00:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.656 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281164+0000", 2024-09-18T00:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3696999999999999e-05 2024-09-18T00:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281270+0000", 2024-09-18T00:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106041 2024-09-18T00:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.285596+0000", 2024-09-18T00:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043266659999999998 2024-09-18T00:19:39.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385028+0000", 2024-09-18T00:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.099431808999999996 2024-09-18T00:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385094+0000", 2024-09-18T00:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5740000000000004e-05 2024-09-18T00:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385166+0000", 2024-09-18T00:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1637000000000005e-05 2024-09-18T00:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.658 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385223+0000", 2024-09-18T00:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6957999999999999e-05 2024-09-18T00:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:8831 3.12 e48/32 ondisk, result = 0)", 2024-09-18T00:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 67.316853288999994, 2024-09-18T00:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087621558000000002, 2024-09-18T00:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.659 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.660 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297613+0000", 2024-09-18T00:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3400000000000001e-06 2024-09-18T00:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297620+0000", 2024-09-18T00:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5869999999999998e-06 2024-09-18T00:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297639+0000", 2024-09-18T00:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9888999999999999e-05 2024-09-18T00:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.661 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297661+0000", 2024-09-18T00:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1772000000000001e-05 2024-09-18T00:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385073+0000", 2024-09-18T00:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087411303999999995 2024-09-18T00:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385086+0000", 2024-09-18T00:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3040000000000001e-05 2024-09-18T00:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.662 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385233+0000", 2024-09-18T00:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014762600000000001 2024-09-18T00:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22269 3.17 3:e9a1eb35:::rbd_data.112a948d5956.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.705321462999997, 2024-09-18T00:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.063462693000000001, 2024-09-18T00:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:39.663 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22269 2024-09-18T00:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.664 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909146+0000", 2024-09-18T00:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3700000000000002e-06 2024-09-18T00:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.665 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909160+0000", 2024-09-18T00:19:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3825000000000001e-05 2024-09-18T00:19:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909178+0000", 2024-09-18T00:19:39.666 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8567999999999999e-05 2024-09-18T00:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909189+0000", 2024-09-18T00:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0881e-05 2024-09-18T00:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909270+0000", 2024-09-18T00:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0846999999999994e-05 2024-09-18T00:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.667 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909383+0000", 2024-09-18T00:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112704 2024-09-18T00:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972310+0000", 2024-09-18T00:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062927081999999995 2024-09-18T00:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972433+0000", 2024-09-18T00:19:39.668 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-18T00:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972606+0000", 2024-09-18T00:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017317799999999999 2024-09-18T00:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22379 3.17 3:e9a1eb35:::rbd_data.112a948d5956.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.211246166000002, 2024-09-18T00:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.12560279099999999, 2024-09-18T00:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22379 2024-09-18T00:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.670 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403221+0000", 2024-09-18T00:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3269999999999999e-06 2024-09-18T00:19:39.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403232+0000", 2024-09-18T00:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1392e-05 2024-09-18T00:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.672 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403264+0000", 2024-09-18T00:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1146999999999998e-05 2024-09-18T00:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403279+0000", 2024-09-18T00:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5109000000000001e-05 2024-09-18T00:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.673 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403424+0000", 2024-09-18T00:19:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014516999999999999 2024-09-18T00:19:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403567+0000", 2024-09-18T00:19:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142824 2024-09-18T00:19:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528723+0000", 2024-09-18T00:19:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.12515647299999999 2024-09-18T00:19:39.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528764+0000", 2024-09-18T00:19:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0624000000000003e-05 2024-09-18T00:19:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528822+0000", 2024-09-18T00:19:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7725e-05 2024-09-18T00:19:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39268 3.1f 3:fdf3a436:::rbd_data.112a3496573c.0000000000000001:17 [read 352256~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.576221+0000", 2024-09-18T00:19:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.038244122999998, 2024-09-18T00:19:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047293515000000001, 2024-09-18T00:19:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.676 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39268 2024-09-18T00:19:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576221+0000", 2024-09-18T00:19:39.677 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576221+0000", 2024-09-18T00:19:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576221+0000", 2024-09-18T00:19:39.678 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-07 2024-09-18T00:19:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576229+0000", 2024-09-18T00:19:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9950000000000005e-06 2024-09-18T00:19:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.679 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576237+0000", 2024-09-18T00:19:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0010000000000001e-06 2024-09-18T00:19:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576243+0000", 2024-09-18T00:19:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.592e-06 2024-09-18T00:19:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.680 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623336+0000", 2024-09-18T00:19:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047092698000000002 2024-09-18T00:19:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623418+0000", 2024-09-18T00:19:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1977000000000005e-05 2024-09-18T00:19:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623514+0000", 2024-09-18T00:19:39.681 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6676999999999996e-05 2024-09-18T00:19:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39269 3.1f 3:fdf3a436:::rbd_data.112a3496573c.0000000000000001:17 [read 362496~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.576250+0000", 2024-09-18T00:19:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.038215212000001, 2024-09-18T00:19:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047454169999999997, 2024-09-18T00:19:39.682 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39269 2024-09-18T00:19:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.683 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576250+0000", 2024-09-18T00:19:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576250+0000", 2024-09-18T00:19:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576250+0000", 2024-09-18T00:19:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:19:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576252+0000", 2024-09-18T00:19:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1739999999999999e-06 2024-09-18T00:19:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576258+0000", 2024-09-18T00:19:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6670000000000001e-06 2024-09-18T00:19:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576264+0000", 2024-09-18T00:19:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1360000000000002e-06 2024-09-18T00:19:39.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623543+0000", 2024-09-18T00:19:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047278496000000003 2024-09-18T00:19:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623609+0000", 2024-09-18T00:19:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5901000000000005e-05 2024-09-18T00:19:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.686 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623704+0000", 2024-09-18T00:19:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5291999999999995e-05 2024-09-18T00:19:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39270 3.1f 3:fdf3a436:::rbd_data.112a3496573c.0000000000000001:17 [read 372736~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.576271+0000", 2024-09-18T00:19:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.038193883000002, 2024-09-18T00:19:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047626546999999998, 2024-09-18T00:19:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.687 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39270 2024-09-18T00:19:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576271+0000", 2024-09-18T00:19:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.688 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576271+0000", 2024-09-18T00:19:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576271+0000", 2024-09-18T00:19:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.01e-07 2024-09-18T00:19:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576274+0000", 2024-09-18T00:19:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1150000000000001e-06 2024-09-18T00:19:39.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576280+0000", 2024-09-18T00:19:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9399999999999999e-06 2024-09-18T00:19:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576285+0000", 2024-09-18T00:19:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0719999999999999e-06 2024-09-18T00:19:39.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623731+0000", 2024-09-18T00:19:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047446629999999997 2024-09-18T00:19:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623809+0000", 2024-09-18T00:19:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8141999999999998e-05 2024-09-18T00:19:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.691 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623898+0000", 2024-09-18T00:19:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8246999999999997e-05 2024-09-18T00:19:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39360 3.1f 3:fec8fa94:::rbd_data.112a3496573c.0000000000000006:17 [read 354304~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.579954+0000", 2024-09-18T00:19:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.034511035000001, 2024-09-18T00:19:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047250546999999997, 2024-09-18T00:19:39.692 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39360 2024-09-18T00:19:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579954+0000", 2024-09-18T00:19:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579954+0000", 2024-09-18T00:19:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579954+0000", 2024-09-18T00:19:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7800000000000002e-07 2024-09-18T00:19:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579956+0000", 2024-09-18T00:19:39.694 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.114e-06 2024-09-18T00:19:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579962+0000", 2024-09-18T00:19:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7400000000000001e-06 2024-09-18T00:19:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579967+0000", 2024-09-18T00:19:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.792e-06 2024-09-18T00:19:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627056+0000", 2024-09-18T00:19:39.695 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047089131999999999 2024-09-18T00:19:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627122+0000", 2024-09-18T00:19:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6229000000000006e-05 2024-09-18T00:19:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627204+0000", 2024-09-18T00:19:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2062000000000001e-05 2024-09-18T00:19:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.696 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39361 3.1f 3:fec8fa94:::rbd_data.112a3496573c.0000000000000006:17 [read 364544~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.579975+0000", 2024-09-18T00:19:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.034489878999999, 2024-09-18T00:19:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047405364999999998, 2024-09-18T00:19:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39361 2024-09-18T00:19:39.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579975+0000", 2024-09-18T00:19:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579975+0000", 2024-09-18T00:19:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579976+0000", 2024-09-18T00:19:39.698 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7399999999999998e-07 2024-09-18T00:19:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579978+0000", 2024-09-18T00:19:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5359999999999999e-06 2024-09-18T00:19:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579984+0000", 2024-09-18T00:19:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2469999999999997e-06 2024-09-18T00:19:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579989+0000", 2024-09-18T00:19:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6700000000000002e-06 2024-09-18T00:19:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627232+0000", 2024-09-18T00:19:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047243016999999998 2024-09-18T00:19:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627299+0000", 2024-09-18T00:19:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7145999999999997e-05 2024-09-18T00:19:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627380+0000", 2024-09-18T00:19:39.700 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1274999999999997e-05 2024-09-18T00:19:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39473 3.1f 3:fd448eba:::rbd_data.112a3496573c.000000000000000c:17 [read 366592~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.583299+0000", 2024-09-18T00:19:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.031165558000001, 2024-09-18T00:19:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047411813999999997, 2024-09-18T00:19:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:39.701 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39473 2024-09-18T00:19:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583299+0000", 2024-09-18T00:19:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583299+0000", 2024-09-18T00:19:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.702 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583300+0000", 2024-09-18T00:19:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6699999999999999e-07 2024-09-18T00:19:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583302+0000", 2024-09-18T00:19:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.979e-06 2024-09-18T00:19:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583308+0000", 2024-09-18T00:19:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.835e-06 2024-09-18T00:19:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.703 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583313+0000", 2024-09-18T00:19:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9409999999999998e-06 2024-09-18T00:19:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.630537+0000", 2024-09-18T00:19:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047224592000000003 2024-09-18T00:19:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.630623+0000", 2024-09-18T00:19:39.704 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5473e-05 2024-09-18T00:19:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.630711+0000", 2024-09-18T00:19:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8527000000000001e-05 2024-09-18T00:19:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.705 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:54153 3.b e90/23 3:d3f7b2f3:::rbd_data.112a859a01c9.0000000000000001:head v 90'238, mlcod=90'236)", 2024-09-18T00:19:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:30.858396+0000", 2024-09-18T00:19:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 8.7560691899999998, 2024-09-18T00:19:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056676272999999999, 2024-09-18T00:19:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858396+0000", 2024-09-18T00:19:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.706 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858396+0000", 2024-09-18T00:19:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858398+0000", 2024-09-18T00:19:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.838e-06 2024-09-18T00:19:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.707 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860747+0000", 2024-09-18T00:19:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0023496160000000001 2024-09-18T00:19:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860777+0000", 2024-09-18T00:19:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9754e-05 2024-09-18T00:19:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860791+0000", 2024-09-18T00:19:39.708 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.377e-05 2024-09-18T00:19:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860879+0000", 2024-09-18T00:19:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7818999999999994e-05 2024-09-18T00:19:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.709 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860917+0000", 2024-09-18T00:19:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8124999999999998e-05 2024-09-18T00:19:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915020+0000", 2024-09-18T00:19:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.054103126000000001 2024-09-18T00:19:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915072+0000", 2024-09-18T00:19:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2225000000000002e-05 2024-09-18T00:19:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.710 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:58293 3.1b 3:d96f405d:::rbd_data.112a17f95ba0.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T00:19:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:38.838123+0000", 2024-09-18T00:19:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.77634154799999999, 2024-09-18T00:19:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11992721000000001, 2024-09-18T00:19:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:39.711 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 58293 2024-09-18T00:19:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.712 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838123+0000", 2024-09-18T00:19:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838123+0000", 2024-09-18T00:19:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838125+0000", 2024-09-18T00:19:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.158e-06 2024-09-18T00:19:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838130+0000", 2024-09-18T00:19:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7019999999999998e-06 2024-09-18T00:19:39.714 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838145+0000", 2024-09-18T00:19:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4871999999999999e-05 2024-09-18T00:19:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838152+0000", 2024-09-18T00:19:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3309999999999998e-06 2024-09-18T00:19:39.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838227+0000", 2024-09-18T00:19:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4685999999999996e-05 2024-09-18T00:19:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.716 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838375+0000", 2024-09-18T00:19:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014751 2024-09-18T00:19:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.845187+0000", 2024-09-18T00:19:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0068119749999999996 2024-09-18T00:19:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.957945+0000", 2024-09-18T00:19:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.112758517 2024-09-18T00:19:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.957963+0000", 2024-09-18T00:19:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7580000000000001e-05 2024-09-18T00:19:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.958009+0000", 2024-09-18T00:19:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6377e-05 2024-09-18T00:19:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.958051+0000", 2024-09-18T00:19:39.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1501999999999997e-05 2024-09-18T00:19:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:58293 3.1b e96/23 ondisk, result = 0)", 2024-09-18T00:19:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:38.867986+0000", 2024-09-18T00:19:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.74647943400000005, 2024-09-18T00:19:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.089969120999999999, 2024-09-18T00:19:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:39.719 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.867986+0000", 2024-09-18T00:19:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.867986+0000", 2024-09-18T00:19:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:39.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.867987+0000", 2024-09-18T00:19:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9649999999999998e-06 2024-09-18T00:19:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.867996+0000", 2024-09-18T00:19:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3259999999999996e-06 2024-09-18T00:19:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.868030+0000", 2024-09-18T00:19:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4428999999999999e-05 2024-09-18T00:19:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.868050+0000", 2024-09-18T00:19:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9368000000000002e-05 2024-09-18T00:19:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.957905+0000", 2024-09-18T00:19:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.089855395000000005 2024-09-18T00:19:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.957915+0000", 2024-09-18T00:19:39.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7280000000000007e-06 2024-09-18T00:19:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.957955+0000", 2024-09-18T00:19:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9910000000000002e-05 2024-09-18T00:19:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:39.723 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:39.723 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:39.725 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:19:39.725 INFO:teuthology.orchestra.run.smithi097.stdout:truncating image image_client.0-clone27 from 0xd953c59 (overlap 0x154a6ec) to 0xaa5376 2024-09-18T00:19:39.726 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:39.721+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:39.761 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-18T00:19:39.786 INFO:teuthology.orchestra.run.smithi097.stdout:5896 write 0xedd8132 thru 0xede2297 (0xa166 bytes) 2024-09-18T00:19:39.794 INFO:teuthology.orchestra.run.smithi097.stdout:5897 writesame 0xb63a9c5 thru 0xb644f09 data_size 0x9(0xa545 bytes) 2024-09-18T00:19:39.803 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:39.803 INFO:teuthology.orchestra.run.smithi097.stdout:5899 punch from 0x5a5222e to 0x5a573d0, (0x51a2 bytes) 2024-09-18T00:19:39.821 INFO:teuthology.orchestra.run.smithi097.stdout:5900 trunc from 0xede2298 to 0x568b821 2024-09-18T00:19:39.824 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:39.825+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:39.853 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:39.853 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:39.853 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:39.853 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:39.870 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:19:39.870 INFO:teuthology.orchestra.run.smithi097.stdout:5901 read 0x2a8a1f4 thru 0x2a8e8ec (0x46f9 bytes) 2024-09-18T00:19:39.890 INFO:teuthology.orchestra.run.smithi097.stdout:5903 trunc from 0x568b821 to 0xec1b149 2024-09-18T00:19:39.900 INFO:teuthology.orchestra.run.smithi097.stdout:5907 write 0xde68d76 thru 0xde6f4ae (0x6739 bytes) 2024-09-18T00:19:39.908 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:39.908 INFO:teuthology.orchestra.run.smithi097.stdout:5909 write 0xe1b9e8d thru 0xe1ba5e7 (0x75b bytes) 2024-09-18T00:19:39.914 INFO:teuthology.orchestra.run.smithi097.stdout:5910 write 0x95d8a2c thru 0x95ddae1 (0x50b6 bytes) 2024-09-18T00:19:39.922 INFO:teuthology.orchestra.run.smithi097.stdout:5911 read 0xdd39787 thru 0xdd426bd (0x8f37 bytes) 2024-09-18T00:19:39.923 INFO:teuthology.orchestra.run.smithi097.stdout:5912 trunc from 0xec1b149 to 0xe16dc2b 2024-09-18T00:19:39.925 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:39.925+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:39.934 INFO:teuthology.orchestra.run.smithi097.stdout:5913 write 0x44e68c4 thru 0x44efbaf (0x92ec bytes) 2024-09-18T00:19:39.944 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:39.945 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:39.945 INFO:teuthology.orchestra.run.smithi097.stdout:5917 read 0x2f77fe9 thru 0x2f7f084 (0x709c bytes) 2024-09-18T00:19:39.945 INFO:teuthology.orchestra.run.smithi097.stdout:5918 read 0xbc007e2 thru 0xbc0b002 (0xa821 bytes) 2024-09-18T00:19:39.945 INFO:teuthology.orchestra.run.smithi097.stdout:5919 writesame 0x98f674b thru 0x98f680a data_size 0x8(0xc0 bytes) 2024-09-18T00:19:39.951 INFO:teuthology.orchestra.run.smithi097.stdout:5921 trunc from 0xe16dc2b to 0xebadf31 2024-09-18T00:19:39.960 INFO:teuthology.orchestra.run.smithi097.stdout:5923 punch from 0x414bf43 to 0x415b42d, (0xf4ea bytes) 2024-09-18T00:19:39.960 INFO:teuthology.orchestra.run.smithi097.stdout:5924 read 0x4af8040 thru 0x4b0706b (0xf02c bytes) 2024-09-18T00:19:39.960 INFO:teuthology.orchestra.run.smithi097.stdout:5926 writesame 0xb300177 thru 0xb30e162 data_size 0x11(0xdfec bytes) 2024-09-18T00:19:39.971 INFO:teuthology.orchestra.run.smithi097.stdout:5928 writesame 0xb421be2 thru 0xb42855b data_size 0x17(0x697a bytes) 2024-09-18T00:19:39.980 INFO:teuthology.orchestra.run.smithi097.stdout:truncating to largest ever: 0xee581ad 2024-09-18T00:19:39.981 INFO:teuthology.orchestra.run.smithi097.stdout:5930 trunc from 0xebadf31 to 0xee581ad 2024-09-18T00:19:39.993 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:39.993 INFO:teuthology.orchestra.run.smithi097.stdout:5932 read 0xde106e9 thru 0xde129d8 (0x22f0 bytes) 2024-09-18T00:19:39.994 INFO:teuthology.orchestra.run.smithi097.stdout:5933 read 0x320cdd4 thru 0x321711c (0xa349 bytes) 2024-09-18T00:19:39.994 INFO:teuthology.orchestra.run.smithi097.stdout:5934 write 0x6551fad thru 0x6554187 (0x21db bytes) 2024-09-18T00:19:40.000 INFO:teuthology.orchestra.run.smithi097.stdout:5935 writesame 0xc88057a thru 0xc881be3 data_size 0x2(0x166a bytes) 2024-09-18T00:19:40.007 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:40.007 INFO:teuthology.orchestra.run.smithi097.stdout:5938 write 0x47fc341 thru 0x47fec8f (0x294f bytes) 2024-09-18T00:19:40.016 INFO:teuthology.orchestra.run.smithi097.stdout:5940 write 0x6bf0c5a thru 0x6c00499 (0xf840 bytes) 2024-09-18T00:19:40.017 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:40.017 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:40.017 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:40.017 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:40.017 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:40.027 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:40.025+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:40.032 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:19:40.036 INFO:teuthology.orchestra.run.smithi097.stdout:5941 read 0x6d39309 thru 0x6d3bf5a (0x2c52 bytes) 2024-09-18T00:19:40.036 INFO:teuthology.orchestra.run.smithi097.stdout:5942 write 0x9b70f36 thru 0x9b77e39 (0x6f04 bytes) 2024-09-18T00:19:40.046 INFO:teuthology.orchestra.run.smithi097.stdout:5945 read 0xa6315a0 thru 0xa637800 (0x6261 bytes) 2024-09-18T00:19:40.046 INFO:teuthology.orchestra.run.smithi097.stdout:5946 punch from 0x6dfeecd to 0x6e077e1, (0x8914 bytes) 2024-09-18T00:19:40.074 INFO:teuthology.orchestra.run.smithi097.stdout:5947 trunc from 0xee581ad to 0x2cde954 2024-09-18T00:19:40.111 INFO:teuthology.orchestra.run.smithi097.stdout:5948 trunc from 0x2cde954 to 0xe74511 2024-09-18T00:19:40.128 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:40.125+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:40.143 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:40.143 INFO:teuthology.orchestra.run.smithi097.stdout:5951 read 0x52e93c thru 0x53b049 (0xc70e bytes) 2024-09-18T00:19:40.144 INFO:teuthology.orchestra.run.smithi097.stdout:5952 write 0xd4ef71 thru 0xd5016b (0x11fb bytes) 2024-09-18T00:19:40.158 INFO:teuthology.orchestra.run.smithi097.stdout:5954 read 0x69c9c1 thru 0x6a167f (0x4cbf bytes) 2024-09-18T00:19:40.158 INFO:teuthology.orchestra.run.smithi097.stdout:5957 punch from 0xd4bbcd to 0xd56210, (0xa643 bytes) 2024-09-18T00:19:40.158 INFO:teuthology.orchestra.run.smithi097.stdout:5958 read 0xcb02e1 thru 0xcbd8d6 (0xd5f6 bytes) 2024-09-18T00:19:40.171 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:40.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:5722 3.4 e41/23 3:20381d86:::rbd_data.112a3a39fc45.000000000000001d:head v 41'107, mlcod=40'105)", 2024-09-18T00:19:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.831448533, 2024-09-18T00:19:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01185172, 2024-09-18T00:19:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335497+0000", 2024-09-18T00:19:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.505e-06 2024-09-18T00:19:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335945+0000", 2024-09-18T00:19:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000447687 2024-09-18T00:19:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335961+0000", 2024-09-18T00:19:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6430999999999999e-05 2024-09-18T00:19:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335969+0000", 2024-09-18T00:19:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:19:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336030+0000", 2024-09-18T00:19:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1255999999999998e-05 2024-09-18T00:19:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336061+0000", 2024-09-18T00:19:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0579e-05 2024-09-18T00:19:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347296+0000", 2024-09-18T00:19:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011235663 2024-09-18T00:19:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347347+0000", 2024-09-18T00:19:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0914000000000001e-05 2024-09-18T00:19:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.177 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11457 3.0 3:0214e7b3:::rbd_data.112ab1fa102a.0000000000000150:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:40.177 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:40.177 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 64.907082595000006, 2024-09-18T00:19:40.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028499165, 2024-09-18T00:19:40.177 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.177 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.177 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.177 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.177 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.177 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.177 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11457 2024-09-18T00:19:40.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.177 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:40.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:40.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259865+0000", 2024-09-18T00:19:40.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7230000000000002e-06 2024-09-18T00:19:40.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259883+0000", 2024-09-18T00:19:40.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8067000000000002e-05 2024-09-18T00:19:40.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259924+0000", 2024-09-18T00:19:40.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0550999999999999e-05 2024-09-18T00:19:40.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259943+0000", 2024-09-18T00:19:40.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:19:40.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260054+0000", 2024-09-18T00:19:40.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011075 2024-09-18T00:19:40.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260138+0000", 2024-09-18T00:19:40.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4549000000000007e-05 2024-09-18T00:19:40.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.287926+0000", 2024-09-18T00:19:40.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027787642000000001 2024-09-18T00:19:40.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288090+0000", 2024-09-18T00:19:40.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016402499999999999 2024-09-18T00:19:40.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288361+0000", 2024-09-18T00:19:40.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027097800000000001 2024-09-18T00:19:40.182 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.182 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.182 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.182 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18661 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:40.182 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:40.182 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.232314729999999, 2024-09-18T00:19:40.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020008037999999999, 2024-09-18T00:19:40.182 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.182 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.182 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.183 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.183 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.183 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.183 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18661 2024-09-18T00:19:40.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.183 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:40.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:40.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934633+0000", 2024-09-18T00:19:40.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8e-06 2024-09-18T00:19:40.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935511+0000", 2024-09-18T00:19:40.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087785700000000001 2024-09-18T00:19:40.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935561+0000", 2024-09-18T00:19:40.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9472000000000003e-05 2024-09-18T00:19:40.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935586+0000", 2024-09-18T00:19:40.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5616e-05 2024-09-18T00:19:40.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935838+0000", 2024-09-18T00:19:40.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025202699999999999 2024-09-18T00:19:40.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.936023+0000", 2024-09-18T00:19:40.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018480300000000001 2024-09-18T00:19:40.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953545+0000", 2024-09-18T00:19:40.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017522157 2024-09-18T00:19:40.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953723+0000", 2024-09-18T00:19:40.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017753800000000001 2024-09-18T00:19:40.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954637+0000", 2024-09-18T00:19:40.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091476800000000005 2024-09-18T00:19:40.187 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.187 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.187 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.187 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18662 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:40.187 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:40.187 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.231309793000001, 2024-09-18T00:19:40.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019388843999999999, 2024-09-18T00:19:40.188 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.188 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.188 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.188 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.188 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.188 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.188 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18662 2024-09-18T00:19:40.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.188 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.188 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.188 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:40.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:40.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935636+0000", 2024-09-18T00:19:40.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.663e-06 2024-09-18T00:19:40.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935646+0000", 2024-09-18T00:19:40.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000006e-06 2024-09-18T00:19:40.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935672+0000", 2024-09-18T00:19:40.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5868999999999999e-05 2024-09-18T00:19:40.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935692+0000", 2024-09-18T00:19:40.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0604000000000001e-05 2024-09-18T00:19:40.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950738+0000", 2024-09-18T00:19:40.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015045868 2024-09-18T00:19:40.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950992+0000", 2024-09-18T00:19:40.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025368400000000001 2024-09-18T00:19:40.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954674+0000", 2024-09-18T00:19:40.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003682154 2024-09-18T00:19:40.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954806+0000", 2024-09-18T00:19:40.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013252000000000001 2024-09-18T00:19:40.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.955023+0000", 2024-09-18T00:19:40.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021681200000000001 2024-09-18T00:19:40.192 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.192 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.192 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.192 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19382 3.4 3:22f09d3a:::rbd_data.112a7fe29ab4.000000000000015a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:40.193 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:40.193 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.544899743000002, 2024-09-18T00:19:40.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075119302999999998, 2024-09-18T00:19:40.193 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.193 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.193 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.193 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19382 2024-09-18T00:19:40.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.193 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:40.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:40.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622045+0000", 2024-09-18T00:19:40.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0020000000000001e-06 2024-09-18T00:19:40.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622053+0000", 2024-09-18T00:19:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2679999999999997e-06 2024-09-18T00:19:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622062+0000", 2024-09-18T00:19:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3749999999999992e-06 2024-09-18T00:19:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622068+0000", 2024-09-18T00:19:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5729999999999998e-06 2024-09-18T00:19:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622118+0000", 2024-09-18T00:19:40.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0696999999999999e-05 2024-09-18T00:19:40.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622222+0000", 2024-09-18T00:19:40.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103286 2024-09-18T00:19:40.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697061+0000", 2024-09-18T00:19:40.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074839141999999997 2024-09-18T00:19:40.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697102+0000", 2024-09-18T00:19:40.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1513e-05 2024-09-18T00:19:40.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.197 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.197 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697164+0000", 2024-09-18T00:19:40.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1446999999999996e-05 2024-09-18T00:19:40.197 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.197 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.197 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.198 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20245 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:40.198 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:40.198 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.011241413999997, 2024-09-18T00:19:40.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099981159999999996, 2024-09-18T00:19:40.198 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.198 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.198 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:40.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.198 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.198 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20245 2024-09-18T00:19:40.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.198 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:40.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:40.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:40.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.65e-07 2024-09-18T00:19:40.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155711+0000", 2024-09-18T00:19:40.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:19:40.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155720+0000", 2024-09-18T00:19:40.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6580000000000008e-06 2024-09-18T00:19:40.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155725+0000", 2024-09-18T00:19:40.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3430000000000003e-06 2024-09-18T00:19:40.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165479+0000", 2024-09-18T00:19:40.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097538239999999995 2024-09-18T00:19:40.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165556+0000", 2024-09-18T00:19:40.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6724000000000006e-05 2024-09-18T00:19:40.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.202 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.202 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165701+0000", 2024-09-18T00:19:40.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145129 2024-09-18T00:19:40.202 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.202 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.202 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.202 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20246 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:40.202 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:40.202 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.011211551000002, 2024-09-18T00:19:40.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01018784, 2024-09-18T00:19:40.202 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.202 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.203 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:40.203 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.203 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.203 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.203 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20246 2024-09-18T00:19:40.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.203 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:40.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:40.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:40.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2599999999999998e-07 2024-09-18T00:19:40.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155735+0000", 2024-09-18T00:19:40.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2699999999999999e-06 2024-09-18T00:19:40.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155744+0000", 2024-09-18T00:19:40.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8629999999999997e-06 2024-09-18T00:19:40.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155749+0000", 2024-09-18T00:19:40.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0359999999999997e-06 2024-09-18T00:19:40.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165748+0000", 2024-09-18T00:19:40.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099991449999999992 2024-09-18T00:19:40.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165819+0000", 2024-09-18T00:19:40.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0964999999999996e-05 2024-09-18T00:19:40.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.206 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165920+0000", 2024-09-18T00:19:40.206 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101135 2024-09-18T00:19:40.206 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.206 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.206 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.207 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20247 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:40.207 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:40.207 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.011188208999997, 2024-09-18T00:19:40.207 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010443987, 2024-09-18T00:19:40.207 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.207 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.207 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:40.207 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.207 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.207 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.207 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20247 2024-09-18T00:19:40.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.207 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.207 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:40.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:40.208 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.208 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.208 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:40.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1899999999999998e-07 2024-09-18T00:19:40.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155760+0000", 2024-09-18T00:19:40.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6459999999999999e-06 2024-09-18T00:19:40.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155767+0000", 2024-09-18T00:19:40.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0350000000000004e-06 2024-09-18T00:19:40.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155774+0000", 2024-09-18T00:19:40.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:19:40.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165960+0000", 2024-09-18T00:19:40.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186051999999999 2024-09-18T00:19:40.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.210 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.210 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166048+0000", 2024-09-18T00:19:40.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8848e-05 2024-09-18T00:19:40.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166200+0000", 2024-09-18T00:19:40.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015147000000000001 2024-09-18T00:19:40.211 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.211 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.211 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.211 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20248 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:40.211 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:40.211 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.011163045000004, 2024-09-18T00:19:40.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010617721, 2024-09-18T00:19:40.211 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.212 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.212 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:40.212 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.212 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.212 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.212 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20248 2024-09-18T00:19:40.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.212 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:40.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:40.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155782+0000", 2024-09-18T00:19:40.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.84e-07 2024-09-18T00:19:40.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155784+0000", 2024-09-18T00:19:40.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2800000000000002e-06 2024-09-18T00:19:40.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155790+0000", 2024-09-18T00:19:40.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5679999999999996e-06 2024-09-18T00:19:40.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155795+0000", 2024-09-18T00:19:40.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6929999999999998e-06 2024-09-18T00:19:40.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166245+0000", 2024-09-18T00:19:40.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010449847999999999 2024-09-18T00:19:40.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166307+0000", 2024-09-18T00:19:40.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2287999999999997e-05 2024-09-18T00:19:40.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166399+0000", 2024-09-18T00:19:40.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.166e-05 2024-09-18T00:19:40.215 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.215 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.215 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.216 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20249 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:40.216 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:40.216 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.011142669000002, 2024-09-18T00:19:40.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010811983000000001, 2024-09-18T00:19:40.216 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.216 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.216 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:40.216 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.216 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.216 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.216 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20249 2024-09-18T00:19:40.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.217 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:40.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:40.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.217 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.217 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155802+0000", 2024-09-18T00:19:40.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:19:40.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155804+0000", 2024-09-18T00:19:40.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:19:40.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155810+0000", 2024-09-18T00:19:40.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0580000000000003e-06 2024-09-18T00:19:40.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155817+0000", 2024-09-18T00:19:40.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3980000000000004e-06 2024-09-18T00:19:40.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166451+0000", 2024-09-18T00:19:40.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010634825000000001 2024-09-18T00:19:40.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166520+0000", 2024-09-18T00:19:40.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8317000000000003e-05 2024-09-18T00:19:40.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166613+0000", 2024-09-18T00:19:40.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3801000000000005e-05 2024-09-18T00:19:40.221 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.221 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.221 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.221 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:21913 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:40.221 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:40.222 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.873297522999998, 2024-09-18T00:19:40.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090508916999999994, 2024-09-18T00:19:40.222 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.222 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.222 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.222 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.222 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.222 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.222 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 21913 2024-09-18T00:19:40.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.223 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:40.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:40.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293648+0000", 2024-09-18T00:19:40.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6400000000000001e-07 2024-09-18T00:19:40.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293653+0000", 2024-09-18T00:19:40.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3820000000000003e-06 2024-09-18T00:19:40.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293665+0000", 2024-09-18T00:19:40.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1778e-05 2024-09-18T00:19:40.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293673+0000", 2024-09-18T00:19:40.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4130000000000006e-06 2024-09-18T00:19:40.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293719+0000", 2024-09-18T00:19:40.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5534999999999997e-05 2024-09-18T00:19:40.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293773+0000", 2024-09-18T00:19:40.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4389999999999999e-05 2024-09-18T00:19:40.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384053+0000", 2024-09-18T00:19:40.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090279617000000006 2024-09-18T00:19:40.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384088+0000", 2024-09-18T00:19:40.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5466000000000002e-05 2024-09-18T00:19:40.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384156+0000", 2024-09-18T00:19:40.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7471999999999994e-05 2024-09-18T00:19:40.227 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.227 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.227 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.227 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22274 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:40.228 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:40.228 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.062181303000003, 2024-09-18T00:19:40.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.069820742000000005, 2024-09-18T00:19:40.228 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.228 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.228 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.228 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.228 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.228 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.228 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22274 2024-09-18T00:19:40.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.228 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:40.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:40.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104766+0000", 2024-09-18T00:19:40.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3859999999999999e-06 2024-09-18T00:19:40.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104783+0000", 2024-09-18T00:19:40.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6697e-05 2024-09-18T00:19:40.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104816+0000", 2024-09-18T00:19:40.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2786000000000001e-05 2024-09-18T00:19:40.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104833+0000", 2024-09-18T00:19:40.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7125999999999998e-05 2024-09-18T00:19:40.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104965+0000", 2024-09-18T00:19:40.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013215899999999999 2024-09-18T00:19:40.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.105103+0000", 2024-09-18T00:19:40.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138193 2024-09-18T00:19:40.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174496+0000", 2024-09-18T00:19:40.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.06939244 2024-09-18T00:19:40.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174530+0000", 2024-09-18T00:19:40.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4601999999999998e-05 2024-09-18T00:19:40.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174584+0000", 2024-09-18T00:19:40.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3353000000000002e-05 2024-09-18T00:19:40.232 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.232 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.233 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.233 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22485 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write 16521958~30847] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:40.233 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:40.233 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.213392292999998, 2024-09-18T00:19:40.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012574648000000001, 2024-09-18T00:19:40.233 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.233 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.233 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.233 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.233 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.233 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.233 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22485 2024-09-18T00:19:40.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.234 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:40.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:40.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953554+0000", 2024-09-18T00:19:40.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1150000000000001e-06 2024-09-18T00:19:40.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962036+0000", 2024-09-18T00:19:40.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0084817599999999996 2024-09-18T00:19:40.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962062+0000", 2024-09-18T00:19:40.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6499000000000002e-05 2024-09-18T00:19:40.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962074+0000", 2024-09-18T00:19:40.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2095e-05 2024-09-18T00:19:40.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962165+0000", 2024-09-18T00:19:40.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0703000000000002e-05 2024-09-18T00:19:40.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962276+0000", 2024-09-18T00:19:40.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111406 2024-09-18T00:19:40.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965925+0000", 2024-09-18T00:19:40.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036488919999999999 2024-09-18T00:19:40.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965992+0000", 2024-09-18T00:19:40.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6806999999999999e-05 2024-09-18T00:19:40.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.966127+0000", 2024-09-18T00:19:40.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013437099999999999 2024-09-18T00:19:40.238 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.238 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.238 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.238 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22681 3.4 3:22e76021:::rbd_data.112a948d5956.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:40.238 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:40.238 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 40.879392987999999, 2024-09-18T00:19:40.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076591189000000004, 2024-09-18T00:19:40.238 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.238 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.238 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.239 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.239 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.239 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.239 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22681 2024-09-18T00:19:40.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.239 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:40.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:40.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287554+0000", 2024-09-18T00:19:40.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.514e-06 2024-09-18T00:19:40.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287567+0000", 2024-09-18T00:19:40.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-18T00:19:40.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287597+0000", 2024-09-18T00:19:40.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0406e-05 2024-09-18T00:19:40.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287626+0000", 2024-09-18T00:19:40.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9306e-05 2024-09-18T00:19:40.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287801+0000", 2024-09-18T00:19:40.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017411400000000001 2024-09-18T00:19:40.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.288053+0000", 2024-09-18T00:19:40.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025223500000000001 2024-09-18T00:19:40.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364016+0000", 2024-09-18T00:19:40.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075963301999999996 2024-09-18T00:19:40.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364064+0000", 2024-09-18T00:19:40.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7886000000000002e-05 2024-09-18T00:19:40.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364142+0000", 2024-09-18T00:19:40.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8345999999999998e-05 2024-09-18T00:19:40.243 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.243 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.243 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.243 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:24700 3.4 3:23a8f5d4:::rbd_data.112a990280ef.0000000000000016:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:40.243 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:19:40.243 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 33.228927009000003, 2024-09-18T00:19:40.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091975649000000007, 2024-09-18T00:19:40.243 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.243 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.244 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.244 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.244 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.244 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.244 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 24700 2024-09-18T00:19:40.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.244 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:19:40.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:19:40.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938019+0000", 2024-09-18T00:19:40.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:19:40.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938030+0000", 2024-09-18T00:19:40.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0278000000000001e-05 2024-09-18T00:19:40.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938053+0000", 2024-09-18T00:19:40.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2946999999999999e-05 2024-09-18T00:19:40.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938062+0000", 2024-09-18T00:19:40.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8509999999999994e-06 2024-09-18T00:19:40.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938140+0000", 2024-09-18T00:19:40.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7664999999999996e-05 2024-09-18T00:19:40.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938314+0000", 2024-09-18T00:19:40.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017398500000000001 2024-09-18T00:19:40.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029896+0000", 2024-09-18T00:19:40.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091582078999999997 2024-09-18T00:19:40.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.247 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.247 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029930+0000", 2024-09-18T00:19:40.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3723999999999998e-05 2024-09-18T00:19:40.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029993+0000", 2024-09-18T00:19:40.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2901e-05 2024-09-18T00:19:40.248 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.248 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.248 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.248 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37798 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:40.248 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410235+0000", 2024-09-18T00:19:40.248 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.756709208, 2024-09-18T00:19:40.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099276060000000003, 2024-09-18T00:19:40.249 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.249 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.249 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:40.249 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.249 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.249 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.249 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37798 2024-09-18T00:19:40.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.249 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410235+0000", 2024-09-18T00:19:40.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410235+0000", 2024-09-18T00:19:40.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410235+0000", 2024-09-18T00:19:40.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4699999999999998e-07 2024-09-18T00:19:40.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410236+0000", 2024-09-18T00:19:40.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1069999999999999e-06 2024-09-18T00:19:40.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410240+0000", 2024-09-18T00:19:40.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3119999999999998e-06 2024-09-18T00:19:40.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410242+0000", 2024-09-18T00:19:40.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3860000000000001e-06 2024-09-18T00:19:40.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.419934+0000", 2024-09-18T00:19:40.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0096915720000000007 2024-09-18T00:19:40.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420021+0000", 2024-09-18T00:19:40.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6941999999999995e-05 2024-09-18T00:19:40.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420163+0000", 2024-09-18T00:19:40.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014203999999999999 2024-09-18T00:19:40.252 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.252 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.253 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.253 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37799 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:40.253 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:19:40.253 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.756698858, 2024-09-18T00:19:40.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186016000000001, 2024-09-18T00:19:40.253 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.253 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.253 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:40.253 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.253 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.253 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.254 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37799 2024-09-18T00:19:40.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.254 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:19:40.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:19:40.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410246+0000", 2024-09-18T00:19:40.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.35e-07 2024-09-18T00:19:40.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410247+0000", 2024-09-18T00:19:40.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0389999999999999e-06 2024-09-18T00:19:40.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410250+0000", 2024-09-18T00:19:40.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3380000000000001e-06 2024-09-18T00:19:40.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410252+0000", 2024-09-18T00:19:40.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5150000000000001e-06 2024-09-18T00:19:40.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420198+0000", 2024-09-18T00:19:40.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099454360000000002 2024-09-18T00:19:40.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420284+0000", 2024-09-18T00:19:40.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5638999999999998e-05 2024-09-18T00:19:40.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420431+0000", 2024-09-18T00:19:40.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014781399999999999 2024-09-18T00:19:40.257 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.257 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.257 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.257 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37800 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:40.257 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:19:40.258 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.756688581999999, 2024-09-18T00:19:40.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010452559, 2024-09-18T00:19:40.258 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.258 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.258 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:40.258 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.258 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.258 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.258 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37800 2024-09-18T00:19:40.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.258 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:19:40.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:19:40.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:19:40.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2499999999999999e-07 2024-09-18T00:19:40.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410257+0000", 2024-09-18T00:19:40.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0750000000000001e-06 2024-09-18T00:19:40.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410260+0000", 2024-09-18T00:19:40.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.428e-06 2024-09-18T00:19:40.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410263+0000", 2024-09-18T00:19:40.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5339999999999998e-06 2024-09-18T00:19:40.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420467+0000", 2024-09-18T00:19:40.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010204615 2024-09-18T00:19:40.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420557+0000", 2024-09-18T00:19:40.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9183000000000003e-05 2024-09-18T00:19:40.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420708+0000", 2024-09-18T00:19:40.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015149900000000001 2024-09-18T00:19:40.262 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.262 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.262 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.262 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37801 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:40.262 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:19:40.262 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.756678245, 2024-09-18T00:19:40.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010623357, 2024-09-18T00:19:40.262 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.262 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.262 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:40.263 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.263 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.263 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.263 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37801 2024-09-18T00:19:40.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.263 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:19:40.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.263 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:19:40.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:19:40.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.23e-07 2024-09-18T00:19:40.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410271+0000", 2024-09-18T00:19:40.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8040000000000001e-06 2024-09-18T00:19:40.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410274+0000", 2024-09-18T00:19:40.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5310000000000002e-06 2024-09-18T00:19:40.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410277+0000", 2024-09-18T00:19:40.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.605e-06 2024-09-18T00:19:40.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420743+0000", 2024-09-18T00:19:40.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010465487000000001 2024-09-18T00:19:40.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420811+0000", 2024-09-18T00:19:40.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8845999999999998e-05 2024-09-18T00:19:40.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420889+0000", 2024-09-18T00:19:40.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7861000000000006e-05 2024-09-18T00:19:40.266 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.266 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.266 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37802 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:19:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 19.756663619000001, 2024-09-18T00:19:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010745463, 2024-09-18T00:19:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37802 2024-09-18T00:19:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:19:40.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:19:40.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:19:40.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6199999999999999e-07 2024-09-18T00:19:40.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410282+0000", 2024-09-18T00:19:40.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.085e-06 2024-09-18T00:19:40.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410286+0000", 2024-09-18T00:19:40.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6610000000000002e-06 2024-09-18T00:19:40.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410288+0000", 2024-09-18T00:19:40.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6410000000000002e-06 2024-09-18T00:19:40.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420910+0000", 2024-09-18T00:19:40.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010621479 2024-09-18T00:19:40.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420952+0000", 2024-09-18T00:19:40.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2497999999999999e-05 2024-09-18T00:19:40.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.421026+0000", 2024-09-18T00:19:40.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3837000000000005e-05 2024-09-18T00:19:40.271 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.271 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.271 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.271 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.271 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.271 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:40.273 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:19:40.274 INFO:teuthology.orchestra.run.smithi097.stdout:5959 writesame 0xebfb88e thru 0xebff081 data_size 0x2(0x37f4 bytes) 2024-09-18T00:19:40.274 INFO:teuthology.orchestra.run.smithi097.stdout:5960 write 0xbfd6af6 thru 0xbfdda80 (0x6f8b bytes) 2024-09-18T00:19:40.274 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:40.274 INFO:teuthology.orchestra.run.smithi097.stdout:5962 read 0xa70d2b8 thru 0xa712bb4 (0x58fd bytes) 2024-09-18T00:19:40.274 INFO:teuthology.orchestra.run.smithi097.stdout:5963 read 0x7e6d215 thru 0x7e721ca (0x4fb6 bytes) 2024-09-18T00:19:40.274 INFO:teuthology.orchestra.run.smithi097.stdout:5964 trunc from 0xebff082 to 0x39a8e5f 2024-09-18T00:19:40.274 INFO:teuthology.orchestra.run.smithi097.stdout:5966 read 0x2885f27 thru 0x288e13f (0x8219 bytes) 2024-09-18T00:19:40.274 INFO:teuthology.orchestra.run.smithi097.stdout:5967 write 0x5f94d8d thru 0x5f98bfc (0x3e70 bytes) 2024-09-18T00:19:40.274 INFO:teuthology.orchestra.run.smithi097.stdout:5968 read 0x305cf51 thru 0x3064fae (0x805e bytes) 2024-09-18T00:19:40.274 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:40.275 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:40.269+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:40.290 INFO:teuthology.orchestra.run.smithi097.stdout:5972 writesame 0xecaf03f thru 0xecbc19c data_size 0x3(0xd15e bytes) 2024-09-18T00:19:40.290 INFO:teuthology.orchestra.run.smithi097.stdout:5973 read 0x79ca1ec thru 0x79d5dad (0xbbc2 bytes) 2024-09-18T00:19:40.290 INFO:teuthology.orchestra.run.smithi097.stdout:5974 write 0x2e67a44 thru 0x2e766d4 (0xec91 bytes) 2024-09-18T00:19:40.304 INFO:teuthology.orchestra.run.smithi097.stdout:5976 read 0xdf8fae9 thru 0xdf94dd0 (0x52e8 bytes) 2024-09-18T00:19:40.305 INFO:teuthology.orchestra.run.smithi097.stdout:5978 read 0x2c87b38 thru 0x2c959b4 (0xde7d bytes) 2024-09-18T00:19:40.305 INFO:teuthology.orchestra.run.smithi097.stdout:5979 punch from 0x95cf6d3 to 0x95de4c3, (0xedf0 bytes) 2024-09-18T00:19:40.305 INFO:teuthology.orchestra.run.smithi097.stdout:5980 write 0x839b245 thru 0x83aa006 (0xedc2 bytes) 2024-09-18T00:19:40.318 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:40.318 INFO:teuthology.orchestra.run.smithi097.stdout:5983 write 0x631defb thru 0x63229a9 (0x4aaf bytes) 2024-09-18T00:19:40.327 INFO:teuthology.orchestra.run.smithi097.stdout:5984 punch from 0x41e117f to 0x41e6170, (0x4ff1 bytes) 2024-09-18T00:19:40.351 INFO:teuthology.orchestra.run.smithi097.stdout:5986 trunc from 0xecbc19d to 0x2132a9d 2024-09-18T00:19:40.370 INFO:teuthology.orchestra.run.smithi097.stdout:5987 write 0x16b4f0c thru 0x16c3bae (0xeca3 bytes) 2024-09-18T00:19:40.373 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:40.373+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:40.383 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:40.399 INFO:teuthology.orchestra.run.smithi097.stdout:5989 write 0x4ff8c0c thru 0x4ff9228 (0x61d bytes) 2024-09-18T00:19:40.419 INFO:teuthology.orchestra.run.smithi097.stdout:5990 trunc from 0x4ff9229 to 0xcd2d679 2024-09-18T00:19:40.430 INFO:teuthology.orchestra.run.smithi097.stdout:5994 read 0x8fa677a thru 0x8fb0b53 (0xa3da bytes) 2024-09-18T00:19:40.431 INFO:teuthology.orchestra.run.smithi097.stdout:5995 trunc from 0xcd2d679 to 0xd903924 2024-09-18T00:19:40.441 INFO:teuthology.orchestra.run.smithi097.stdout:5996 punch from 0xd4560f3 to 0xd46267b, (0xc588 bytes) 2024-09-18T00:19:40.441 INFO:teuthology.orchestra.run.smithi097.stdout:5997 read 0x7b07a51 thru 0x7b08098 (0x648 bytes) 2024-09-18T00:19:40.452 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:40.452 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:40.459 INFO:teuthology.orchestra.run.smithi097.stdout:5999 trunc from 0xd903924 to 0x34a33c3 2024-09-18T00:19:40.474 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:19:40.474 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:40.469+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:40.479 INFO:teuthology.orchestra.run.smithi097.stdout:compare and write disabled 2024-09-18T00:19:40.536 INFO:teuthology.orchestra.run.smithi097.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-18T00:19:40.542 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is set 2024-09-18T00:19:40.574 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:40.573+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:40.606 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:40.607 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:40.607 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:40.621 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:19:40.676 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:40.673+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:40.776 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:40.773+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:40.789 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:40.789 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:19:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.986010187000005, 2024-09-18T00:19:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:19:40.790 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:19:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:19:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:19:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:19:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:19:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:19:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:19:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:19:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:19:40.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:19:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:19:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:19:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:19:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:40.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:19:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:19:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:19:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:19:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:19:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:19:40.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:19:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:19:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:19:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:19:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:19:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 87.969802314000006, 2024-09-18T00:19:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:19:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:19:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:19:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:19:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:19:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:19:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:19:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:19:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:19:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:19:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:19:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:19:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:19:40.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:19:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.942760993999997, 2024-09-18T00:19:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:19:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.801 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:19:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:40.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:19:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:19:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:19:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:19:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:19:40.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:19:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:19:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:19:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:19:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:19:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:19:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:19:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:19:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:19:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:19:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:19:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:19:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:19:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:19:40.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:19:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:19:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:19:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:19:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.921484441999993, 2024-09-18T00:19:40.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:19:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:40.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:19:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:19:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:19:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:19:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:19:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:19:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:19:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:19:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:19:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:19:40.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:19:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:19:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:19:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:19:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11460 3.19 3:9e578b6c:::rbd_data.112ab1fa102a.0000000000000153:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.519112989999996, 2024-09-18T00:19:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041989011999999999, 2024-09-18T00:19:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11460 2024-09-18T00:19:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.812 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267067+0000", 2024-09-18T00:19:40.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:19:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267088+0000", 2024-09-18T00:19:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0908999999999999e-05 2024-09-18T00:19:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267133+0000", 2024-09-18T00:19:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5232000000000003e-05 2024-09-18T00:19:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267157+0000", 2024-09-18T00:19:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3833e-05 2024-09-18T00:19:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267300+0000", 2024-09-18T00:19:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142282 2024-09-18T00:19:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267447+0000", 2024-09-18T00:19:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147851 2024-09-18T00:19:40.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308889+0000", 2024-09-18T00:19:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041441779999999998 2024-09-18T00:19:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308956+0000", 2024-09-18T00:19:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7136000000000002e-05 2024-09-18T00:19:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.309053+0000", 2024-09-18T00:19:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6552999999999995e-05 2024-09-18T00:19:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:16947 3.18 3:18b58c7b:::rbd_data.112a6efb7564.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:19:40.817 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 52.169335289000003, 2024-09-18T00:19:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.101692484, 2024-09-18T00:19:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 16947 2024-09-18T00:19:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616843+0000", 2024-09-18T00:19:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0780000000000001e-06 2024-09-18T00:19:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617785+0000", 2024-09-18T00:19:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094204100000000002 2024-09-18T00:19:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617801+0000", 2024-09-18T00:19:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6436999999999999e-05 2024-09-18T00:19:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617809+0000", 2024-09-18T00:19:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9049999999999997e-06 2024-09-18T00:19:40.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617904+0000", 2024-09-18T00:19:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5204999999999994e-05 2024-09-18T00:19:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.618071+0000", 2024-09-18T00:19:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016640000000000001 2024-09-18T00:19:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.624534+0000", 2024-09-18T00:19:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064634499999999999 2024-09-18T00:19:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718299+0000", 2024-09-18T00:19:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093764739 2024-09-18T00:19:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718375+0000", 2024-09-18T00:19:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6532999999999994e-05 2024-09-18T00:19:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718466+0000", 2024-09-18T00:19:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0581000000000006e-05 2024-09-18T00:19:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718534+0000", 2024-09-18T00:19:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8114999999999994e-05 2024-09-18T00:19:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.823 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:16947 3.18 e59/23 ondisk, result = 0)", 2024-09-18T00:19:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 52.144118595000002, 2024-09-18T00:19:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076488069000000006, 2024-09-18T00:19:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642060+0000", 2024-09-18T00:19:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6640000000000001e-06 2024-09-18T00:19:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642071+0000", 2024-09-18T00:19:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1204e-05 2024-09-18T00:19:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642091+0000", 2024-09-18T00:19:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0313999999999999e-05 2024-09-18T00:19:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642103+0000", 2024-09-18T00:19:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1214e-05 2024-09-18T00:19:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718350+0000", 2024-09-18T00:19:40.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076247681999999997 2024-09-18T00:19:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718366+0000", 2024-09-18T00:19:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5420000000000001e-05 2024-09-18T00:19:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718546+0000", 2024-09-18T00:19:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000180571 2024-09-18T00:19:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18686 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.837441224999999, 2024-09-18T00:19:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041132573999999998, 2024-09-18T00:19:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18686 2024-09-18T00:19:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.828 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948737+0000", 2024-09-18T00:19:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3769999999999999e-06 2024-09-18T00:19:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949968+0000", 2024-09-18T00:19:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001230493 2024-09-18T00:19:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949993+0000", 2024-09-18T00:19:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5282e-05 2024-09-18T00:19:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950011+0000", 2024-09-18T00:19:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8199999999999999e-05 2024-09-18T00:19:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950180+0000", 2024-09-18T00:19:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016867600000000001 2024-09-18T00:19:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950337+0000", 2024-09-18T00:19:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156902 2024-09-18T00:19:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:40.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950818+0000", 2024-09-18T00:19:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048192599999999998 2024-09-18T00:19:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978491+0000", 2024-09-18T00:19:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027672568000000002 2024-09-18T00:19:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989759+0000", 2024-09-18T00:19:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011268144000000001 2024-09-18T00:19:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989824+0000", 2024-09-18T00:19:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4556999999999999e-05 2024-09-18T00:19:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989868+0000", 2024-09-18T00:19:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4449000000000001e-05 2024-09-18T00:19:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18687 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:40.833 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.83611346, 2024-09-18T00:19:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040034156000000001, 2024-09-18T00:19:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18687 2024-09-18T00:19:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950065+0000", 2024-09-18T00:19:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5820000000000001e-06 2024-09-18T00:19:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950072+0000", 2024-09-18T00:19:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1029999999999996e-06 2024-09-18T00:19:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950092+0000", 2024-09-18T00:19:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0288e-05 2024-09-18T00:19:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950107+0000", 2024-09-18T00:19:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4290999999999999e-05 2024-09-18T00:19:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975820+0000", 2024-09-18T00:19:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025713573999999999 2024-09-18T00:19:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975980+0000", 2024-09-18T00:19:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016002799999999999 2024-09-18T00:19:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.976299+0000", 2024-09-18T00:19:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031853300000000002 2024-09-18T00:19:40.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989936+0000", 2024-09-18T00:19:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013637046999999999 2024-09-18T00:19:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990025+0000", 2024-09-18T00:19:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9040000000000001e-05 2024-09-18T00:19:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990064+0000", 2024-09-18T00:19:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9394000000000003e-05 2024-09-18T00:19:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990098+0000", 2024-09-18T00:19:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3275999999999998e-05 2024-09-18T00:19:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18688 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:40.839 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.836012107999998, 2024-09-18T00:19:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040041259000000003, 2024-09-18T00:19:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18688 2024-09-18T00:19:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:40.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950166+0000", 2024-09-18T00:19:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:19:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951295+0000", 2024-09-18T00:19:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011293480000000001 2024-09-18T00:19:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951328+0000", 2024-09-18T00:19:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2524999999999998e-05 2024-09-18T00:19:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951346+0000", 2024-09-18T00:19:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8170999999999999e-05 2024-09-18T00:19:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978546+0000", 2024-09-18T00:19:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027200418000000001 2024-09-18T00:19:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978666+0000", 2024-09-18T00:19:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119913 2024-09-18T00:19:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.979098+0000", 2024-09-18T00:19:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00043169399999999999 2024-09-18T00:19:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990000+0000", 2024-09-18T00:19:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010901719000000001 2024-09-18T00:19:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990109+0000", 2024-09-18T00:19:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010964299999999999 2024-09-18T00:19:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990152+0000", 2024-09-18T00:19:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2877000000000003e-05 2024-09-18T00:19:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990206+0000", 2024-09-18T00:19:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3717000000000001e-05 2024-09-18T00:19:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18689 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.83480041, 2024-09-18T00:19:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051022338, 2024-09-18T00:19:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18689 2024-09-18T00:19:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951378+0000", 2024-09-18T00:19:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:19:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951385+0000", 2024-09-18T00:19:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-18T00:19:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951402+0000", 2024-09-18T00:19:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6538999999999999e-05 2024-09-18T00:19:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951565+0000", 2024-09-18T00:19:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016307999999999999 2024-09-18T00:19:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988232+0000", 2024-09-18T00:19:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036667656999999999 2024-09-18T00:19:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988318+0000", 2024-09-18T00:19:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5348999999999999e-05 2024-09-18T00:19:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988574+0000", 2024-09-18T00:19:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025670800000000002 2024-09-18T00:19:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002254+0000", 2024-09-18T00:19:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013679274 2024-09-18T00:19:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002326+0000", 2024-09-18T00:19:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2720999999999997e-05 2024-09-18T00:19:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002364+0000", 2024-09-18T00:19:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7314000000000002e-05 2024-09-18T00:19:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002399+0000", 2024-09-18T00:19:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5162e-05 2024-09-18T00:19:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18690 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.834583391999999, 2024-09-18T00:19:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050872134999999999, 2024-09-18T00:19:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18690 2024-09-18T00:19:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951595+0000", 2024-09-18T00:19:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4479999999999999e-06 2024-09-18T00:19:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952167+0000", 2024-09-18T00:19:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00057188400000000004 2024-09-18T00:19:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952192+0000", 2024-09-18T00:19:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4936999999999999e-05 2024-09-18T00:19:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952206+0000", 2024-09-18T00:19:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4565e-05 2024-09-18T00:19:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990303+0000", 2024-09-18T00:19:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038096954000000002 2024-09-18T00:19:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990403+0000", 2024-09-18T00:19:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9486000000000003e-05 2024-09-18T00:19:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990773+0000", 2024-09-18T00:19:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037037799999999998 2024-09-18T00:19:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002316+0000", 2024-09-18T00:19:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011542581 2024-09-18T00:19:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002407+0000", 2024-09-18T00:19:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0968e-05 2024-09-18T00:19:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002442+0000", 2024-09-18T00:19:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4962000000000002e-05 2024-09-18T00:19:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002466+0000", 2024-09-18T00:19:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3972e-05 2024-09-18T00:19:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18691 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.833950514999998, 2024-09-18T00:19:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051495530999999997, 2024-09-18T00:19:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18691 2024-09-18T00:19:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952228+0000", 2024-09-18T00:19:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:19:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952234+0000", 2024-09-18T00:19:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0310000000000004e-06 2024-09-18T00:19:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952248+0000", 2024-09-18T00:19:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4577e-05 2024-09-18T00:19:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952260+0000", 2024-09-18T00:19:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1659e-05 2024-09-18T00:19:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.000989+0000", 2024-09-18T00:19:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048729445000000003 2024-09-18T00:19:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001076+0000", 2024-09-18T00:19:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6323999999999995e-05 2024-09-18T00:19:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001295+0000", 2024-09-18T00:19:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021944299999999999 2024-09-18T00:19:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003502+0000", 2024-09-18T00:19:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022065209999999999 2024-09-18T00:19:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003581+0000", 2024-09-18T00:19:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9404000000000001e-05 2024-09-18T00:19:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003652+0000", 2024-09-18T00:19:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0506999999999994e-05 2024-09-18T00:19:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003722+0000", 2024-09-18T00:19:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0300000000000001e-05 2024-09-18T00:19:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18692 3.7 3:e69124d0:::rbd_data.112a6efb7564.0000000000000001:f [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.833902453999997, 2024-09-18T00:19:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050418114999999999, 2024-09-18T00:19:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18692 2024-09-18T00:19:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952275+0000", 2024-09-18T00:19:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4e-07 2024-09-18T00:19:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952281+0000", 2024-09-18T00:19:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2000000000000002e-06 2024-09-18T00:19:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952295+0000", 2024-09-18T00:19:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4731e-05 2024-09-18T00:19:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952307+0000", 2024-09-18T00:19:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1877e-05 2024-09-18T00:19:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002506+0000", 2024-09-18T00:19:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050199230999999997 2024-09-18T00:19:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002574+0000", 2024-09-18T00:19:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8076000000000003e-05 2024-09-18T00:19:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002693+0000", 2024-09-18T00:19:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011806 2024-09-18T00:19:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19383 3.5 3:a1859bb0:::rbd_data.112a7fe29ab4.000000000000015b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.163890856999998, 2024-09-18T00:19:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056760192000000001, 2024-09-18T00:19:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19383 2024-09-18T00:19:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622287+0000", 2024-09-18T00:19:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-18T00:19:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622292+0000", 2024-09-18T00:19:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6110000000000004e-06 2024-09-18T00:19:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622304+0000", 2024-09-18T00:19:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2106000000000001e-05 2024-09-18T00:19:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622309+0000", 2024-09-18T00:19:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.818e-06 2024-09-18T00:19:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622360+0000", 2024-09-18T00:19:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0430999999999999e-05 2024-09-18T00:19:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622438+0000", 2024-09-18T00:19:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8362999999999995e-05 2024-09-18T00:19:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678811+0000", 2024-09-18T00:19:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056372894 2024-09-18T00:19:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678906+0000", 2024-09-18T00:19:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5078e-05 2024-09-18T00:19:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.679046+0000", 2024-09-18T00:19:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139987 2024-09-18T00:19:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22468 3.5 3:a250f1ef:::rbd_data.112a948d5956.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.014809450999998, 2024-09-18T00:19:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.055599787999999997, 2024-09-18T00:19:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22468 2024-09-18T00:19:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771370+0000", 2024-09-18T00:19:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9629999999999998e-06 2024-09-18T00:19:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771385+0000", 2024-09-18T00:19:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5053e-05 2024-09-18T00:19:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771429+0000", 2024-09-18T00:19:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3192999999999998e-05 2024-09-18T00:19:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771453+0000", 2024-09-18T00:19:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3995999999999999e-05 2024-09-18T00:19:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771623+0000", 2024-09-18T00:19:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000170846 2024-09-18T00:19:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771838+0000", 2024-09-18T00:19:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021416600000000001 2024-09-18T00:19:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826866+0000", 2024-09-18T00:19:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.05502866 2024-09-18T00:19:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826906+0000", 2024-09-18T00:19:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9805999999999998e-05 2024-09-18T00:19:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826967+0000", 2024-09-18T00:19:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1105000000000005e-05 2024-09-18T00:19:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22470 3.2 3:474595b6:::rbd_data.112a948d5956.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 41.958618467000001, 2024-09-18T00:19:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088321282000000001, 2024-09-18T00:19:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22470 2024-09-18T00:19:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827560+0000", 2024-09-18T00:19:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.762e-06 2024-09-18T00:19:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827569+0000", 2024-09-18T00:19:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8740000000000001e-06 2024-09-18T00:19:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827585+0000", 2024-09-18T00:19:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5962000000000001e-05 2024-09-18T00:19:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827595+0000", 2024-09-18T00:19:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5910000000000002e-06 2024-09-18T00:19:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827811+0000", 2024-09-18T00:19:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000216858 2024-09-18T00:19:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:19:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827818+0000", 2024-09-18T00:19:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.99e-06 2024-09-18T00:19:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829799+0000", 2024-09-18T00:19:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019803849999999999 2024-09-18T00:19:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829884+0000", 2024-09-18T00:19:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4852999999999995e-05 2024-09-18T00:19:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915616+0000", 2024-09-18T00:19:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085732808999999993 2024-09-18T00:19:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915711+0000", 2024-09-18T00:19:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4578000000000001e-05 2024-09-18T00:19:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915880+0000", 2024-09-18T00:19:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016861999999999999 2024-09-18T00:19:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:42788 3.7 3:e5a8432b:::rbd_data.112a3496573c.0000000000000027:17 [read 309248~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.067206667000001, 2024-09-18T00:19:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.039621847000000002, 2024-09-18T00:19:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 42788 2024-09-18T00:19:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718971+0000", 2024-09-18T00:19:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2500000000000001e-07 2024-09-18T00:19:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718973+0000", 2024-09-18T00:19:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2510000000000001e-06 2024-09-18T00:19:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718979+0000", 2024-09-18T00:19:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1700000000000002e-06 2024-09-18T00:19:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718984+0000", 2024-09-18T00:19:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9110000000000001e-06 2024-09-18T00:19:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.749126+0000", 2024-09-18T00:19:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030142164999999999 2024-09-18T00:19:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.749199+0000", 2024-09-18T00:19:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2985999999999995e-05 2024-09-18T00:19:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.758592+0000", 2024-09-18T00:19:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093929389999999995 2024-09-18T00:19:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.889 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:54153 3.b 3:d3f7b2f3:::rbd_data.112a859a01c9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,zero 4596947~21151] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:19:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:19:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.9334057310000006, 2024-09-18T00:19:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062612886000000006, 2024-09-18T00:19:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.890 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 54153 2024-09-18T00:19:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:19:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:19:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852773+0000", 2024-09-18T00:19:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.426e-06 2024-09-18T00:19:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855337+0000", 2024-09-18T00:19:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025647830000000002 2024-09-18T00:19:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855369+0000", 2024-09-18T00:19:40.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1300999999999997e-05 2024-09-18T00:19:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855387+0000", 2024-09-18T00:19:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7949999999999999e-05 2024-09-18T00:19:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855487+0000", 2024-09-18T00:19:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010062 2024-09-18T00:19:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855686+0000", 2024-09-18T00:19:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019901399999999999 2024-09-18T00:19:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858037+0000", 2024-09-18T00:19:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002351264 2024-09-18T00:19:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.896302+0000", 2024-09-18T00:19:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038264864000000003 2024-09-18T00:19:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915250+0000", 2024-09-18T00:19:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018947538999999999 2024-09-18T00:19:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915318+0000", 2024-09-18T00:19:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7838999999999999e-05 2024-09-18T00:19:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915384+0000", 2024-09-18T00:19:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6285999999999995e-05 2024-09-18T00:19:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.895 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:55488 3.18 3:1d5573d8:::rbd_data.112a859a01c9.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T00:19:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:19:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.2005755219999994, 2024-09-18T00:19:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045116205999999999, 2024-09-18T00:19:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 55488 2024-09-18T00:19:40.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:19:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:19:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585602+0000", 2024-09-18T00:19:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4000000000000001e-07 2024-09-18T00:19:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585607+0000", 2024-09-18T00:19:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5240000000000002e-06 2024-09-18T00:19:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585618+0000", 2024-09-18T00:19:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1493e-05 2024-09-18T00:19:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585624+0000", 2024-09-18T00:19:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7180000000000001e-06 2024-09-18T00:19:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585672+0000", 2024-09-18T00:19:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8158000000000002e-05 2024-09-18T00:19:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585780+0000", 2024-09-18T00:19:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010781399999999999 2024-09-18T00:19:40.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.588202+0000", 2024-09-18T00:19:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002422156 2024-09-18T00:19:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630375+0000", 2024-09-18T00:19:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042172820999999999 2024-09-18T00:19:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630426+0000", 2024-09-18T00:19:40.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1369000000000002e-05 2024-09-18T00:19:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630581+0000", 2024-09-18T00:19:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000155041 2024-09-18T00:19:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630718+0000", 2024-09-18T00:19:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000136372 2024-09-18T00:19:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:40.902 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:40.902 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:40.903 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:19:40.904 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:40.901+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:40.946 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:19:41.004 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:41.001+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:41.034 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:41.034 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:41.034 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:41.034 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:41.049 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:19:41.096 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:41.097 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:41.104 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:41.101+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:41.113 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:41.113 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:41.130 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:41.130 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:41.150 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:41.150 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:41.167 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:41.168 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:41.170 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:41.170 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:41.182 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:19:41.189 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:41.189 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:41.230 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:41.229+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:41.306 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:41.305+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:41.309 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:41.310 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:41.324 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:19:41.407 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:41.405+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:41.415 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:41.415 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:41.422 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:41.422 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:41.483 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:41.484 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:41.498 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:19:41.507 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:41.505+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:41.538 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 7] out_osds: [6, 1] dead_osds: [1] live_osds: [0, 2, 3, 4, 6, 7, 5] 2024-09-18T00:19:41.538 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:19:41.538 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-18T00:19:41.538 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-18T00:19:41.538 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-18T00:19:41.541 INFO:tasks.ceph.osd.1:Started 2024-09-18T00:19:41.541 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:19:41.608 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:41.605+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:41.622 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:41.622 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:41.636 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:19:41.663 INFO:teuthology.orchestra.run.smithi097.stderr:no valid command found; 10 closest matches: 2024-09-18T00:19:41.663 INFO:teuthology.orchestra.run.smithi097.stderr:0 2024-09-18T00:19:41.663 INFO:teuthology.orchestra.run.smithi097.stderr:1 2024-09-18T00:19:41.663 INFO:teuthology.orchestra.run.smithi097.stderr:2 2024-09-18T00:19:41.663 INFO:teuthology.orchestra.run.smithi097.stderr:abort 2024-09-18T00:19:41.663 INFO:teuthology.orchestra.run.smithi097.stderr:assert 2024-09-18T00:19:41.663 INFO:teuthology.orchestra.run.smithi097.stderr:config diff 2024-09-18T00:19:41.663 INFO:teuthology.orchestra.run.smithi097.stderr:config diff get 2024-09-18T00:19:41.663 INFO:teuthology.orchestra.run.smithi097.stderr:config get 2024-09-18T00:19:41.663 INFO:teuthology.orchestra.run.smithi097.stderr:config help [] 2024-09-18T00:19:41.663 INFO:teuthology.orchestra.run.smithi097.stderr:config set ... 2024-09-18T00:19:41.664 INFO:teuthology.orchestra.run.smithi097.stderr:admin_socket: invalid command 2024-09-18T00:19:41.665 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T00:19:41.665 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-18T00:19:41.709 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:41.705+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:41.753 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:41.753 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:41.768 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:19:41.810 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:41.809+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:41.911 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:41.909+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:41.932 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:41.932 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:41.947 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:19:42.012 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:42.009+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:42.063 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:42.064 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:42.078 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:19:42.122 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:42.121+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:42.213 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:42.209+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:42.247 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:42.247 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:42.261 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:19:42.346 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:42.345+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:42.417 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:42.417+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:42.429 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:42.429 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:42.443 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:19:42.530 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:42.529+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:42.568 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:19:42.607 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:42.607 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:42.622 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:19:42.622 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:42.617+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:42.662 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:19:42.661+0000 7f8bf65e57c0 -1 Falling back to public interface 2024-09-18T00:19:42.722 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:42.721+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:42.793 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:42.793 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:42.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:19:42.826 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:42.821+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:42.925 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:42.921+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:42.925 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:42.925 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:42.939 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:19:43.029 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:43.029+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:43.102 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:43.103 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:43.117 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:43.165 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:43.165+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:43.228 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:43.225+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:43.285 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:43.285 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:43.300 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:19:43.345 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:43.345+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:43.415 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:43.416 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:43.423 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:19:43.430 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:19:43.431 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:43.429+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:43.531 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:43.529+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:43.569 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:43.569 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:43.572 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is set 2024-09-18T00:19:43.576 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:43.601 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:43.601 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:43.601 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:43.601 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:43.601 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:43.601 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:43.601 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:43.618 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:19:43.620 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:43.620 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:43.643 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:43.643 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:43.643 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:43.641+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:43.665 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:43.665 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:43.687 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:43.688 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:43.709 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:43.709 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:43.712 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:43.712 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:43.744 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:43.744 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:43.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:43.744 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:43.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.744 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:5722 3.4 e41/23 3:20381d86:::rbd_data.112a3a39fc45.000000000000001d:head v 41'107, mlcod=40'105)", 2024-09-18T00:19:43.745 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:43.745 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.406729681000002, 2024-09-18T00:19:43.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01185172, 2024-09-18T00:19:43.745 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.745 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.745 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:43.745 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:43.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:19:43.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335497+0000", 2024-09-18T00:19:43.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.505e-06 2024-09-18T00:19:43.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335945+0000", 2024-09-18T00:19:43.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000447687 2024-09-18T00:19:43.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335961+0000", 2024-09-18T00:19:43.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6430999999999999e-05 2024-09-18T00:19:43.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335969+0000", 2024-09-18T00:19:43.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:19:43.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.747 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336030+0000", 2024-09-18T00:19:43.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1255999999999998e-05 2024-09-18T00:19:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336061+0000", 2024-09-18T00:19:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0579e-05 2024-09-18T00:19:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347296+0000", 2024-09-18T00:19:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011235663 2024-09-18T00:19:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347347+0000", 2024-09-18T00:19:43.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0914000000000001e-05 2024-09-18T00:19:43.749 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.749 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.749 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.749 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11457 3.0 3:0214e7b3:::rbd_data.112ab1fa102a.0000000000000150:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:43.749 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:43.749 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 68.482363742999993, 2024-09-18T00:19:43.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028499165, 2024-09-18T00:19:43.749 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.749 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.749 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:43.750 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.750 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.750 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.750 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11457 2024-09-18T00:19:43.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.750 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:43.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:19:43.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259865+0000", 2024-09-18T00:19:43.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7230000000000002e-06 2024-09-18T00:19:43.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259883+0000", 2024-09-18T00:19:43.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8067000000000002e-05 2024-09-18T00:19:43.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259924+0000", 2024-09-18T00:19:43.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0550999999999999e-05 2024-09-18T00:19:43.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259943+0000", 2024-09-18T00:19:43.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:19:43.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260054+0000", 2024-09-18T00:19:43.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011075 2024-09-18T00:19:43.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260138+0000", 2024-09-18T00:19:43.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4549000000000007e-05 2024-09-18T00:19:43.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:43.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.287926+0000", 2024-09-18T00:19:43.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027787642000000001 2024-09-18T00:19:43.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:43.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288090+0000", 2024-09-18T00:19:43.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016402499999999999 2024-09-18T00:19:43.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.754 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288361+0000", 2024-09-18T00:19:43.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027097800000000001 2024-09-18T00:19:43.754 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.754 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.755 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.755 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18661 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:43.755 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:43.755 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 50.807595878000001, 2024-09-18T00:19:43.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020008037999999999, 2024-09-18T00:19:43.755 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.755 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.755 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:43.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.755 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.755 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18661 2024-09-18T00:19:43.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.756 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:43.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:19:43.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934633+0000", 2024-09-18T00:19:43.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8e-06 2024-09-18T00:19:43.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935511+0000", 2024-09-18T00:19:43.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087785700000000001 2024-09-18T00:19:43.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935561+0000", 2024-09-18T00:19:43.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9472000000000003e-05 2024-09-18T00:19:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935586+0000", 2024-09-18T00:19:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5616e-05 2024-09-18T00:19:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935838+0000", 2024-09-18T00:19:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025202699999999999 2024-09-18T00:19:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.936023+0000", 2024-09-18T00:19:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018480300000000001 2024-09-18T00:19:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953545+0000", 2024-09-18T00:19:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017522157 2024-09-18T00:19:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953723+0000", 2024-09-18T00:19:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017753800000000001 2024-09-18T00:19:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954637+0000", 2024-09-18T00:19:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091476800000000005 2024-09-18T00:19:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18662 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 50.806590941000003, 2024-09-18T00:19:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019388843999999999, 2024-09-18T00:19:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18662 2024-09-18T00:19:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:19:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935636+0000", 2024-09-18T00:19:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.663e-06 2024-09-18T00:19:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935646+0000", 2024-09-18T00:19:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000006e-06 2024-09-18T00:19:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935672+0000", 2024-09-18T00:19:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5868999999999999e-05 2024-09-18T00:19:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935692+0000", 2024-09-18T00:19:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0604000000000001e-05 2024-09-18T00:19:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950738+0000", 2024-09-18T00:19:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015045868 2024-09-18T00:19:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950992+0000", 2024-09-18T00:19:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025368400000000001 2024-09-18T00:19:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954674+0000", 2024-09-18T00:19:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003682154 2024-09-18T00:19:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954806+0000", 2024-09-18T00:19:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013252000000000001 2024-09-18T00:19:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.955023+0000", 2024-09-18T00:19:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021681200000000001 2024-09-18T00:19:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19382 3.4 3:22f09d3a:::rbd_data.112a7fe29ab4.000000000000015a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 50.120180890999997, 2024-09-18T00:19:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075119302999999998, 2024-09-18T00:19:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19382 2024-09-18T00:19:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:19:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622045+0000", 2024-09-18T00:19:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0020000000000001e-06 2024-09-18T00:19:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622053+0000", 2024-09-18T00:19:43.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2679999999999997e-06 2024-09-18T00:19:43.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622062+0000", 2024-09-18T00:19:43.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3749999999999992e-06 2024-09-18T00:19:43.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622068+0000", 2024-09-18T00:19:43.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5729999999999998e-06 2024-09-18T00:19:43.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622118+0000", 2024-09-18T00:19:43.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0696999999999999e-05 2024-09-18T00:19:43.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622222+0000", 2024-09-18T00:19:43.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103286 2024-09-18T00:19:43.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:43.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697061+0000", 2024-09-18T00:19:43.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074839141999999997 2024-09-18T00:19:43.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:43.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697102+0000", 2024-09-18T00:19:43.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1513e-05 2024-09-18T00:19:43.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697164+0000", 2024-09-18T00:19:43.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1446999999999996e-05 2024-09-18T00:19:43.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.772 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.772 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20245 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:43.772 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:43.772 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.586522561999999, 2024-09-18T00:19:43.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099981159999999996, 2024-09-18T00:19:43.772 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.772 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.773 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:43.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.773 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20245 2024-09-18T00:19:43.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.773 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.773 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.773 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:43.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:43.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155703+0000", 2024-09-18T00:19:43.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.65e-07 2024-09-18T00:19:43.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155711+0000", 2024-09-18T00:19:43.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:19:43.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155720+0000", 2024-09-18T00:19:43.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6580000000000008e-06 2024-09-18T00:19:43.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155725+0000", 2024-09-18T00:19:43.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3430000000000003e-06 2024-09-18T00:19:43.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165479+0000", 2024-09-18T00:19:43.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0097538239999999995 2024-09-18T00:19:43.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165556+0000", 2024-09-18T00:19:43.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6724000000000006e-05 2024-09-18T00:19:43.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165701+0000", 2024-09-18T00:19:43.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000145129 2024-09-18T00:19:43.778 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.778 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.778 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.778 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20246 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:43.778 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:43.778 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.586492698999997, 2024-09-18T00:19:43.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01018784, 2024-09-18T00:19:43.779 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.779 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.779 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:43.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.779 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.779 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20246 2024-09-18T00:19:43.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.779 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:43.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:43.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:19:43.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2599999999999998e-07 2024-09-18T00:19:43.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.780 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.780 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155735+0000", 2024-09-18T00:19:43.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2699999999999999e-06 2024-09-18T00:19:43.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155744+0000", 2024-09-18T00:19:43.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8629999999999997e-06 2024-09-18T00:19:43.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155749+0000", 2024-09-18T00:19:43.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0359999999999997e-06 2024-09-18T00:19:43.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165748+0000", 2024-09-18T00:19:43.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099991449999999992 2024-09-18T00:19:43.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165819+0000", 2024-09-18T00:19:43.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0964999999999996e-05 2024-09-18T00:19:43.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165920+0000", 2024-09-18T00:19:43.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101135 2024-09-18T00:19:43.782 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.782 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.783 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.783 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20247 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:43.783 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:43.783 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.586469356999999, 2024-09-18T00:19:43.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010443987, 2024-09-18T00:19:43.784 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.784 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.784 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:43.784 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.784 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.784 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.784 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20247 2024-09-18T00:19:43.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.785 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:43.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:43.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:19:43.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1899999999999998e-07 2024-09-18T00:19:43.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155760+0000", 2024-09-18T00:19:43.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6459999999999999e-06 2024-09-18T00:19:43.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155767+0000", 2024-09-18T00:19:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0350000000000004e-06 2024-09-18T00:19:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155774+0000", 2024-09-18T00:19:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:19:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.788 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165960+0000", 2024-09-18T00:19:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186051999999999 2024-09-18T00:19:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166048+0000", 2024-09-18T00:19:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8848e-05 2024-09-18T00:19:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166200+0000", 2024-09-18T00:19:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015147000000000001 2024-09-18T00:19:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.789 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20248 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.586444192999998, 2024-09-18T00:19:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010617721, 2024-09-18T00:19:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.790 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20248 2024-09-18T00:19:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:19:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155782+0000", 2024-09-18T00:19:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.84e-07 2024-09-18T00:19:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155784+0000", 2024-09-18T00:19:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2800000000000002e-06 2024-09-18T00:19:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155790+0000", 2024-09-18T00:19:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5679999999999996e-06 2024-09-18T00:19:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155795+0000", 2024-09-18T00:19:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6929999999999998e-06 2024-09-18T00:19:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166245+0000", 2024-09-18T00:19:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010449847999999999 2024-09-18T00:19:43.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166307+0000", 2024-09-18T00:19:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2287999999999997e-05 2024-09-18T00:19:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166399+0000", 2024-09-18T00:19:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.166e-05 2024-09-18T00:19:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20249 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.586423816999996, 2024-09-18T00:19:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010811983000000001, 2024-09-18T00:19:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20249 2024-09-18T00:19:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.795 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:19:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155802+0000", 2024-09-18T00:19:43.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:19:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155804+0000", 2024-09-18T00:19:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:19:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155810+0000", 2024-09-18T00:19:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0580000000000003e-06 2024-09-18T00:19:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155817+0000", 2024-09-18T00:19:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3980000000000004e-06 2024-09-18T00:19:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166451+0000", 2024-09-18T00:19:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010634825000000001 2024-09-18T00:19:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166520+0000", 2024-09-18T00:19:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8317000000000003e-05 2024-09-18T00:19:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166613+0000", 2024-09-18T00:19:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3801000000000005e-05 2024-09-18T00:19:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:21913 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.448578671, 2024-09-18T00:19:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090508916999999994, 2024-09-18T00:19:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 21913 2024-09-18T00:19:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:19:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293648+0000", 2024-09-18T00:19:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6400000000000001e-07 2024-09-18T00:19:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293653+0000", 2024-09-18T00:19:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3820000000000003e-06 2024-09-18T00:19:43.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293665+0000", 2024-09-18T00:19:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1778e-05 2024-09-18T00:19:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293673+0000", 2024-09-18T00:19:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4130000000000006e-06 2024-09-18T00:19:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293719+0000", 2024-09-18T00:19:43.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5534999999999997e-05 2024-09-18T00:19:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293773+0000", 2024-09-18T00:19:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4389999999999999e-05 2024-09-18T00:19:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384053+0000", 2024-09-18T00:19:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090279617000000006 2024-09-18T00:19:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384088+0000", 2024-09-18T00:19:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5466000000000002e-05 2024-09-18T00:19:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384156+0000", 2024-09-18T00:19:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7471999999999994e-05 2024-09-18T00:19:43.804 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22274 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.637462450999998, 2024-09-18T00:19:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.069820742000000005, 2024-09-18T00:19:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.805 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22274 2024-09-18T00:19:43.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:19:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104766+0000", 2024-09-18T00:19:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3859999999999999e-06 2024-09-18T00:19:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104783+0000", 2024-09-18T00:19:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6697e-05 2024-09-18T00:19:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104816+0000", 2024-09-18T00:19:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2786000000000001e-05 2024-09-18T00:19:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104833+0000", 2024-09-18T00:19:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7125999999999998e-05 2024-09-18T00:19:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104965+0000", 2024-09-18T00:19:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013215899999999999 2024-09-18T00:19:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.105103+0000", 2024-09-18T00:19:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138193 2024-09-18T00:19:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174496+0000", 2024-09-18T00:19:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.06939244 2024-09-18T00:19:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174530+0000", 2024-09-18T00:19:43.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4601999999999998e-05 2024-09-18T00:19:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174584+0000", 2024-09-18T00:19:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3353000000000002e-05 2024-09-18T00:19:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22485 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write 16521958~30847] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 44.788673441, 2024-09-18T00:19:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012574648000000001, 2024-09-18T00:19:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.812 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22485 2024-09-18T00:19:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:19:43.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953554+0000", 2024-09-18T00:19:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1150000000000001e-06 2024-09-18T00:19:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962036+0000", 2024-09-18T00:19:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0084817599999999996 2024-09-18T00:19:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962062+0000", 2024-09-18T00:19:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6499000000000002e-05 2024-09-18T00:19:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962074+0000", 2024-09-18T00:19:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2095e-05 2024-09-18T00:19:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962165+0000", 2024-09-18T00:19:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0703000000000002e-05 2024-09-18T00:19:43.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962276+0000", 2024-09-18T00:19:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111406 2024-09-18T00:19:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965925+0000", 2024-09-18T00:19:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036488919999999999 2024-09-18T00:19:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:43.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965992+0000", 2024-09-18T00:19:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6806999999999999e-05 2024-09-18T00:19:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.966127+0000", 2024-09-18T00:19:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013437099999999999 2024-09-18T00:19:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.817 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22681 3.4 3:22e76021:::rbd_data.112a948d5956.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 44.454674136000001, 2024-09-18T00:19:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076591189000000004, 2024-09-18T00:19:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22681 2024-09-18T00:19:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.818 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:19:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.819 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287554+0000", 2024-09-18T00:19:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.514e-06 2024-09-18T00:19:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287567+0000", 2024-09-18T00:19:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-18T00:19:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287597+0000", 2024-09-18T00:19:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0406e-05 2024-09-18T00:19:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287626+0000", 2024-09-18T00:19:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9306e-05 2024-09-18T00:19:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287801+0000", 2024-09-18T00:19:43.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017411400000000001 2024-09-18T00:19:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.288053+0000", 2024-09-18T00:19:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025223500000000001 2024-09-18T00:19:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364016+0000", 2024-09-18T00:19:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075963301999999996 2024-09-18T00:19:43.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364064+0000", 2024-09-18T00:19:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7886000000000002e-05 2024-09-18T00:19:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364142+0000", 2024-09-18T00:19:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8345999999999998e-05 2024-09-18T00:19:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:24700 3.4 3:23a8f5d4:::rbd_data.112a990280ef.0000000000000016:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:19:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 36.804208156999998, 2024-09-18T00:19:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091975649000000007, 2024-09-18T00:19:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.824 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 24700 2024-09-18T00:19:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:19:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:19:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938019+0000", 2024-09-18T00:19:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:19:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938030+0000", 2024-09-18T00:19:43.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0278000000000001e-05 2024-09-18T00:19:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938053+0000", 2024-09-18T00:19:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2946999999999999e-05 2024-09-18T00:19:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938062+0000", 2024-09-18T00:19:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8509999999999994e-06 2024-09-18T00:19:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938140+0000", 2024-09-18T00:19:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7664999999999996e-05 2024-09-18T00:19:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938314+0000", 2024-09-18T00:19:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017398500000000001 2024-09-18T00:19:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029896+0000", 2024-09-18T00:19:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091582078999999997 2024-09-18T00:19:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029930+0000", 2024-09-18T00:19:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3723999999999998e-05 2024-09-18T00:19:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029993+0000", 2024-09-18T00:19:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2901e-05 2024-09-18T00:19:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.829 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37798 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410235+0000", 2024-09-18T00:19:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.331990355999999, 2024-09-18T00:19:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099276060000000003, 2024-09-18T00:19:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:43.830 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37798 2024-09-18T00:19:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410235+0000", 2024-09-18T00:19:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410235+0000", 2024-09-18T00:19:43.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410235+0000", 2024-09-18T00:19:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4699999999999998e-07 2024-09-18T00:19:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410236+0000", 2024-09-18T00:19:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1069999999999999e-06 2024-09-18T00:19:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410240+0000", 2024-09-18T00:19:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3119999999999998e-06 2024-09-18T00:19:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410242+0000", 2024-09-18T00:19:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3860000000000001e-06 2024-09-18T00:19:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.419934+0000", 2024-09-18T00:19:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0096915720000000007 2024-09-18T00:19:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420021+0000", 2024-09-18T00:19:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6941999999999995e-05 2024-09-18T00:19:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420163+0000", 2024-09-18T00:19:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014203999999999999 2024-09-18T00:19:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37799 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:19:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.331980005999998, 2024-09-18T00:19:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186016000000001, 2024-09-18T00:19:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.835 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37799 2024-09-18T00:19:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:19:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:19:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410246+0000", 2024-09-18T00:19:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.35e-07 2024-09-18T00:19:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410247+0000", 2024-09-18T00:19:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0389999999999999e-06 2024-09-18T00:19:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410250+0000", 2024-09-18T00:19:43.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3380000000000001e-06 2024-09-18T00:19:43.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410252+0000", 2024-09-18T00:19:43.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5150000000000001e-06 2024-09-18T00:19:43.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420198+0000", 2024-09-18T00:19:43.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099454360000000002 2024-09-18T00:19:43.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420284+0000", 2024-09-18T00:19:43.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5638999999999998e-05 2024-09-18T00:19:43.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420431+0000", 2024-09-18T00:19:43.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014781399999999999 2024-09-18T00:19:43.841 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.841 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.841 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.841 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37800 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:43.841 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:19:43.841 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.331969730000001, 2024-09-18T00:19:43.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010452559, 2024-09-18T00:19:43.841 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.841 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.842 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:43.842 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.842 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.842 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.842 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37800 2024-09-18T00:19:43.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.842 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:19:43.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:19:43.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:19:43.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2499999999999999e-07 2024-09-18T00:19:43.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410257+0000", 2024-09-18T00:19:43.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0750000000000001e-06 2024-09-18T00:19:43.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410260+0000", 2024-09-18T00:19:43.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.428e-06 2024-09-18T00:19:43.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410263+0000", 2024-09-18T00:19:43.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5339999999999998e-06 2024-09-18T00:19:43.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420467+0000", 2024-09-18T00:19:43.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010204615 2024-09-18T00:19:43.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420557+0000", 2024-09-18T00:19:43.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9183000000000003e-05 2024-09-18T00:19:43.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420708+0000", 2024-09-18T00:19:43.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015149900000000001 2024-09-18T00:19:43.846 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.846 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.846 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.846 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37801 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:43.847 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:19:43.847 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.331959392999998, 2024-09-18T00:19:43.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010623357, 2024-09-18T00:19:43.847 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.847 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.847 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:43.847 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.847 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.847 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.847 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37801 2024-09-18T00:19:43.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.848 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:19:43.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:19:43.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:19:43.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.23e-07 2024-09-18T00:19:43.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410271+0000", 2024-09-18T00:19:43.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8040000000000001e-06 2024-09-18T00:19:43.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410274+0000", 2024-09-18T00:19:43.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5310000000000002e-06 2024-09-18T00:19:43.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410277+0000", 2024-09-18T00:19:43.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.605e-06 2024-09-18T00:19:43.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420743+0000", 2024-09-18T00:19:43.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010465487000000001 2024-09-18T00:19:43.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420811+0000", 2024-09-18T00:19:43.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8845999999999998e-05 2024-09-18T00:19:43.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.851 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420889+0000", 2024-09-18T00:19:43.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7861000000000006e-05 2024-09-18T00:19:43.851 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.851 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.852 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.852 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37802 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:43.852 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:19:43.852 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.331944767, 2024-09-18T00:19:43.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010745463, 2024-09-18T00:19:43.852 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:43.852 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:43.852 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:43.852 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:43.853 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:43.853 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:43.853 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37802 2024-09-18T00:19:43.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.853 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:43.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:43.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:19:43.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:43.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:19:43.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:43.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:43.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:19:43.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6199999999999999e-07 2024-09-18T00:19:43.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:43.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410282+0000", 2024-09-18T00:19:43.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.085e-06 2024-09-18T00:19:43.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:43.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410286+0000", 2024-09-18T00:19:43.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6610000000000002e-06 2024-09-18T00:19:43.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:43.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410288+0000", 2024-09-18T00:19:43.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6410000000000002e-06 2024-09-18T00:19:43.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:43.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420910+0000", 2024-09-18T00:19:43.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010621479 2024-09-18T00:19:43.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:43.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420952+0000", 2024-09-18T00:19:43.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2497999999999999e-05 2024-09-18T00:19:43.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:43.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:43.856 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:43.856 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.421026+0000", 2024-09-18T00:19:43.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3837000000000005e-05 2024-09-18T00:19:43.856 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.856 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.856 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.856 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:43.857 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:43.857 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:43.857 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:43.741+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:43.858 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:19:43.901 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:43.901+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:43.960 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:43.957+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:44.031 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:44.031 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:44.031 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:44.031 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:44.046 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:19:44.061 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:44.057+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:44.162 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:44.161+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:44.169 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:44.169 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:44.169 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:44.183 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:19:44.263 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:44.261+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:44.307 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:19:44.307 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 91.504840353000006, 2024-09-18T00:19:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:19:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:19:44.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:19:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:19:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:19:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:19:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:19:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:19:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:19:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:19:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:19:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:19:44.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:19:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:19:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:19:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:19:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:19:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:19:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:44.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:19:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:19:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:19:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:19:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:19:44.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:19:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:19:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 91.488632480000007, 2024-09-18T00:19:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:19:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.316 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:19:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:19:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:19:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:19:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:19:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:19:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:19:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:19:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:19:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:19:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:19:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:19:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:19:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.461591159999998, 2024-09-18T00:19:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:19:44.321 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:19:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:19:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:19:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:19:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:19:44.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:19:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:19:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:19:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:19:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:19:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:19:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:19:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:19:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:19:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:19:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:19:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:19:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:19:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:19:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:19:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:19:44.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:19:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:19:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:19:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.440314607999994, 2024-09-18T00:19:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:19:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:19:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:19:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:19:44.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:19:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:19:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:19:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:19:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:19:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:19:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:19:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:19:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:19:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:19:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:19:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11460 3.19 3:9e578b6c:::rbd_data.112ab1fa102a.0000000000000153:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 69.037943155999997, 2024-09-18T00:19:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041989011999999999, 2024-09-18T00:19:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.332 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11460 2024-09-18T00:19:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267067+0000", 2024-09-18T00:19:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:19:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267088+0000", 2024-09-18T00:19:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0908999999999999e-05 2024-09-18T00:19:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267133+0000", 2024-09-18T00:19:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5232000000000003e-05 2024-09-18T00:19:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267157+0000", 2024-09-18T00:19:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3833e-05 2024-09-18T00:19:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267300+0000", 2024-09-18T00:19:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142282 2024-09-18T00:19:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267447+0000", 2024-09-18T00:19:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147851 2024-09-18T00:19:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308889+0000", 2024-09-18T00:19:44.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041441779999999998 2024-09-18T00:19:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308956+0000", 2024-09-18T00:19:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7136000000000002e-05 2024-09-18T00:19:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.309053+0000", 2024-09-18T00:19:44.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6552999999999995e-05 2024-09-18T00:19:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:16947 3.18 3:18b58c7b:::rbd_data.112a6efb7564.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:19:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.688165455000004, 2024-09-18T00:19:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.101692484, 2024-09-18T00:19:44.338 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 16947 2024-09-18T00:19:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616843+0000", 2024-09-18T00:19:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0780000000000001e-06 2024-09-18T00:19:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617785+0000", 2024-09-18T00:19:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094204100000000002 2024-09-18T00:19:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617801+0000", 2024-09-18T00:19:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6436999999999999e-05 2024-09-18T00:19:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617809+0000", 2024-09-18T00:19:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9049999999999997e-06 2024-09-18T00:19:44.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617904+0000", 2024-09-18T00:19:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5204999999999994e-05 2024-09-18T00:19:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.618071+0000", 2024-09-18T00:19:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016640000000000001 2024-09-18T00:19:44.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.624534+0000", 2024-09-18T00:19:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064634499999999999 2024-09-18T00:19:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:44.344 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718299+0000", 2024-09-18T00:19:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093764739 2024-09-18T00:19:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718375+0000", 2024-09-18T00:19:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6532999999999994e-05 2024-09-18T00:19:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718466+0000", 2024-09-18T00:19:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0581000000000006e-05 2024-09-18T00:19:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718534+0000", 2024-09-18T00:19:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8114999999999994e-05 2024-09-18T00:19:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:16947 3.18 e59/23 ondisk, result = 0)", 2024-09-18T00:19:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.662948761000003, 2024-09-18T00:19:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076488069000000006, 2024-09-18T00:19:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642060+0000", 2024-09-18T00:19:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6640000000000001e-06 2024-09-18T00:19:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642071+0000", 2024-09-18T00:19:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1204e-05 2024-09-18T00:19:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642091+0000", 2024-09-18T00:19:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0313999999999999e-05 2024-09-18T00:19:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642103+0000", 2024-09-18T00:19:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1214e-05 2024-09-18T00:19:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718350+0000", 2024-09-18T00:19:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076247681999999997 2024-09-18T00:19:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718366+0000", 2024-09-18T00:19:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5420000000000001e-05 2024-09-18T00:19:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718546+0000", 2024-09-18T00:19:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000180571 2024-09-18T00:19:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18686 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.356271391, 2024-09-18T00:19:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041132573999999998, 2024-09-18T00:19:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18686 2024-09-18T00:19:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948737+0000", 2024-09-18T00:19:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3769999999999999e-06 2024-09-18T00:19:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949968+0000", 2024-09-18T00:19:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001230493 2024-09-18T00:19:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949993+0000", 2024-09-18T00:19:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5282e-05 2024-09-18T00:19:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950011+0000", 2024-09-18T00:19:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8199999999999999e-05 2024-09-18T00:19:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950180+0000", 2024-09-18T00:19:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016867600000000001 2024-09-18T00:19:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950337+0000", 2024-09-18T00:19:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156902 2024-09-18T00:19:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950818+0000", 2024-09-18T00:19:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048192599999999998 2024-09-18T00:19:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978491+0000", 2024-09-18T00:19:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027672568000000002 2024-09-18T00:19:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989759+0000", 2024-09-18T00:19:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011268144000000001 2024-09-18T00:19:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989824+0000", 2024-09-18T00:19:44.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4556999999999999e-05 2024-09-18T00:19:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989868+0000", 2024-09-18T00:19:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4449000000000001e-05 2024-09-18T00:19:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18687 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.354943626000001, 2024-09-18T00:19:44.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040034156000000001, 2024-09-18T00:19:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18687 2024-09-18T00:19:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950065+0000", 2024-09-18T00:19:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5820000000000001e-06 2024-09-18T00:19:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.360 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950072+0000", 2024-09-18T00:19:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1029999999999996e-06 2024-09-18T00:19:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950092+0000", 2024-09-18T00:19:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0288e-05 2024-09-18T00:19:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950107+0000", 2024-09-18T00:19:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4290999999999999e-05 2024-09-18T00:19:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975820+0000", 2024-09-18T00:19:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025713573999999999 2024-09-18T00:19:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975980+0000", 2024-09-18T00:19:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016002799999999999 2024-09-18T00:19:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:44.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.976299+0000", 2024-09-18T00:19:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031853300000000002 2024-09-18T00:19:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989936+0000", 2024-09-18T00:19:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013637046999999999 2024-09-18T00:19:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990025+0000", 2024-09-18T00:19:44.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9040000000000001e-05 2024-09-18T00:19:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990064+0000", 2024-09-18T00:19:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9394000000000003e-05 2024-09-18T00:19:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990098+0000", 2024-09-18T00:19:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3275999999999998e-05 2024-09-18T00:19:44.364 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18688 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.354842273999999, 2024-09-18T00:19:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040041259000000003, 2024-09-18T00:19:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.365 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18688 2024-09-18T00:19:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.366 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950166+0000", 2024-09-18T00:19:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:19:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.367 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951295+0000", 2024-09-18T00:19:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011293480000000001 2024-09-18T00:19:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951328+0000", 2024-09-18T00:19:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2524999999999998e-05 2024-09-18T00:19:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.368 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951346+0000", 2024-09-18T00:19:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8170999999999999e-05 2024-09-18T00:19:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978546+0000", 2024-09-18T00:19:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027200418000000001 2024-09-18T00:19:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978666+0000", 2024-09-18T00:19:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119913 2024-09-18T00:19:44.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.979098+0000", 2024-09-18T00:19:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00043169399999999999 2024-09-18T00:19:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990000+0000", 2024-09-18T00:19:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010901719000000001 2024-09-18T00:19:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990109+0000", 2024-09-18T00:19:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010964299999999999 2024-09-18T00:19:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990152+0000", 2024-09-18T00:19:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2877000000000003e-05 2024-09-18T00:19:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.371 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990206+0000", 2024-09-18T00:19:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3717000000000001e-05 2024-09-18T00:19:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18689 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.353630576, 2024-09-18T00:19:44.372 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051022338, 2024-09-18T00:19:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18689 2024-09-18T00:19:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.373 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.374 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951378+0000", 2024-09-18T00:19:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:19:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951385+0000", 2024-09-18T00:19:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-18T00:19:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.375 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951402+0000", 2024-09-18T00:19:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6538999999999999e-05 2024-09-18T00:19:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951565+0000", 2024-09-18T00:19:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016307999999999999 2024-09-18T00:19:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.376 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988232+0000", 2024-09-18T00:19:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036667656999999999 2024-09-18T00:19:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988318+0000", 2024-09-18T00:19:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5348999999999999e-05 2024-09-18T00:19:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.377 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:44.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988574+0000", 2024-09-18T00:19:44.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025670800000000002 2024-09-18T00:19:44.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:44.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002254+0000", 2024-09-18T00:19:44.378 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013679274 2024-09-18T00:19:44.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.378 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.378 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:44.378 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002326+0000", 2024-09-18T00:19:44.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2720999999999997e-05 2024-09-18T00:19:44.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:44.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002364+0000", 2024-09-18T00:19:44.379 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7314000000000002e-05 2024-09-18T00:19:44.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.379 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.379 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.379 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002399+0000", 2024-09-18T00:19:44.380 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5162e-05 2024-09-18T00:19:44.380 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.380 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.380 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.380 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.380 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18690 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:44.380 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:44.380 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.353413558, 2024-09-18T00:19:44.381 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050872134999999999, 2024-09-18T00:19:44.381 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.381 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.381 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.381 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.381 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.381 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.381 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18690 2024-09-18T00:19:44.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.381 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.381 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:44.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.382 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:44.382 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.382 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.382 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951595+0000", 2024-09-18T00:19:44.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4479999999999999e-06 2024-09-18T00:19:44.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.383 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952167+0000", 2024-09-18T00:19:44.383 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00057188400000000004 2024-09-18T00:19:44.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.383 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.383 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952192+0000", 2024-09-18T00:19:44.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4936999999999999e-05 2024-09-18T00:19:44.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.384 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.384 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.384 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952206+0000", 2024-09-18T00:19:44.384 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4565e-05 2024-09-18T00:19:44.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990303+0000", 2024-09-18T00:19:44.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038096954000000002 2024-09-18T00:19:44.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990403+0000", 2024-09-18T00:19:44.385 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9486000000000003e-05 2024-09-18T00:19:44.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.385 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.385 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:44.385 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990773+0000", 2024-09-18T00:19:44.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037037799999999998 2024-09-18T00:19:44.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:44.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002316+0000", 2024-09-18T00:19:44.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011542581 2024-09-18T00:19:44.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:44.386 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002407+0000", 2024-09-18T00:19:44.386 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0968e-05 2024-09-18T00:19:44.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.386 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.386 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:44.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002442+0000", 2024-09-18T00:19:44.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4962000000000002e-05 2024-09-18T00:19:44.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.387 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.387 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002466+0000", 2024-09-18T00:19:44.387 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3972e-05 2024-09-18T00:19:44.387 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.387 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.387 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.387 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.387 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18691 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:44.387 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:44.388 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.352780680999999, 2024-09-18T00:19:44.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051495530999999997, 2024-09-18T00:19:44.388 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.388 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.388 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.388 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.388 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.388 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.388 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18691 2024-09-18T00:19:44.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.388 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.388 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.388 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.388 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:44.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:44.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.389 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952228+0000", 2024-09-18T00:19:44.389 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:19:44.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.389 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.389 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952234+0000", 2024-09-18T00:19:44.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0310000000000004e-06 2024-09-18T00:19:44.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952248+0000", 2024-09-18T00:19:44.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4577e-05 2024-09-18T00:19:44.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.390 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.390 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.390 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952260+0000", 2024-09-18T00:19:44.390 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1659e-05 2024-09-18T00:19:44.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.000989+0000", 2024-09-18T00:19:44.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048729445000000003 2024-09-18T00:19:44.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001076+0000", 2024-09-18T00:19:44.391 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6323999999999995e-05 2024-09-18T00:19:44.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.391 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.391 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:44.391 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001295+0000", 2024-09-18T00:19:44.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021944299999999999 2024-09-18T00:19:44.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:44.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003502+0000", 2024-09-18T00:19:44.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022065209999999999 2024-09-18T00:19:44.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.392 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.392 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:44.392 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003581+0000", 2024-09-18T00:19:44.392 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9404000000000001e-05 2024-09-18T00:19:44.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:44.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003652+0000", 2024-09-18T00:19:44.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0506999999999994e-05 2024-09-18T00:19:44.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.393 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.393 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.393 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003722+0000", 2024-09-18T00:19:44.393 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0300000000000001e-05 2024-09-18T00:19:44.393 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.393 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.393 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.394 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.394 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18692 3.7 3:e69124d0:::rbd_data.112a6efb7564.0000000000000001:f [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:44.394 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:44.394 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 51.352732619999998, 2024-09-18T00:19:44.394 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050418114999999999, 2024-09-18T00:19:44.394 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.394 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.394 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:44.394 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.394 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.394 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.395 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18692 2024-09-18T00:19:44.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.395 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.395 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:44.395 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.395 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.395 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:44.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952275+0000", 2024-09-18T00:19:44.396 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4e-07 2024-09-18T00:19:44.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.396 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.396 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.396 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952281+0000", 2024-09-18T00:19:44.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2000000000000002e-06 2024-09-18T00:19:44.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952295+0000", 2024-09-18T00:19:44.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4731e-05 2024-09-18T00:19:44.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.397 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.397 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.397 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952307+0000", 2024-09-18T00:19:44.397 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1877e-05 2024-09-18T00:19:44.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002506+0000", 2024-09-18T00:19:44.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050199230999999997 2024-09-18T00:19:44.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.398 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.398 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.398 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002574+0000", 2024-09-18T00:19:44.398 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8076000000000003e-05 2024-09-18T00:19:44.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.399 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.399 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002693+0000", 2024-09-18T00:19:44.399 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011806 2024-09-18T00:19:44.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.399 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.399 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.399 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19383 3.5 3:a1859bb0:::rbd_data.112a7fe29ab4.000000000000015b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:44.399 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:44.400 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 50.682721022999999, 2024-09-18T00:19:44.400 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056760192000000001, 2024-09-18T00:19:44.400 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.400 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.400 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.400 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.400 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.400 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.400 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19383 2024-09-18T00:19:44.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.400 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:44.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.401 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.401 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:44.401 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.401 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622287+0000", 2024-09-18T00:19:44.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-18T00:19:44.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.402 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622292+0000", 2024-09-18T00:19:44.402 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6110000000000004e-06 2024-09-18T00:19:44.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.402 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.402 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622304+0000", 2024-09-18T00:19:44.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2106000000000001e-05 2024-09-18T00:19:44.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622309+0000", 2024-09-18T00:19:44.403 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.818e-06 2024-09-18T00:19:44.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.403 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.403 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.403 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622360+0000", 2024-09-18T00:19:44.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0430999999999999e-05 2024-09-18T00:19:44.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622438+0000", 2024-09-18T00:19:44.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8362999999999995e-05 2024-09-18T00:19:44.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.404 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.404 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:44.404 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678811+0000", 2024-09-18T00:19:44.404 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056372894 2024-09-18T00:19:44.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:44.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678906+0000", 2024-09-18T00:19:44.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5078e-05 2024-09-18T00:19:44.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.405 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.405 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.405 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.679046+0000", 2024-09-18T00:19:44.405 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139987 2024-09-18T00:19:44.405 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.405 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.406 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.406 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.406 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22468 3.5 3:a250f1ef:::rbd_data.112a948d5956.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:44.406 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:44.406 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.533639616999999, 2024-09-18T00:19:44.406 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.055599787999999997, 2024-09-18T00:19:44.406 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.406 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.407 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.407 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.407 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.407 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.407 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22468 2024-09-18T00:19:44.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.407 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.407 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.407 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.407 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:44.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:44.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.408 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.408 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.408 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771370+0000", 2024-09-18T00:19:44.408 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9629999999999998e-06 2024-09-18T00:19:44.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771385+0000", 2024-09-18T00:19:44.409 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5053e-05 2024-09-18T00:19:44.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.409 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.409 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.409 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771429+0000", 2024-09-18T00:19:44.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3192999999999998e-05 2024-09-18T00:19:44.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771453+0000", 2024-09-18T00:19:44.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3995999999999999e-05 2024-09-18T00:19:44.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.410 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.410 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771623+0000", 2024-09-18T00:19:44.410 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000170846 2024-09-18T00:19:44.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771838+0000", 2024-09-18T00:19:44.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021416600000000001 2024-09-18T00:19:44.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.411 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:44.411 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826866+0000", 2024-09-18T00:19:44.411 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.05502866 2024-09-18T00:19:44.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.411 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:44.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826906+0000", 2024-09-18T00:19:44.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9805999999999998e-05 2024-09-18T00:19:44.412 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.412 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.412 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.412 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826967+0000", 2024-09-18T00:19:44.412 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1105000000000005e-05 2024-09-18T00:19:44.412 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.412 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.413 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.413 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.413 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22470 3.2 3:474595b6:::rbd_data.112a948d5956.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:44.413 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:44.413 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.477448633000002, 2024-09-18T00:19:44.413 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088321282000000001, 2024-09-18T00:19:44.413 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.413 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.413 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.413 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.413 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.414 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.414 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22470 2024-09-18T00:19:44.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.414 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.414 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.414 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.414 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:44.414 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.414 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:44.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.415 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827560+0000", 2024-09-18T00:19:44.415 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.762e-06 2024-09-18T00:19:44.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.415 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.415 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827569+0000", 2024-09-18T00:19:44.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8740000000000001e-06 2024-09-18T00:19:44.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.416 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827585+0000", 2024-09-18T00:19:44.416 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5962000000000001e-05 2024-09-18T00:19:44.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.416 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.416 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827595+0000", 2024-09-18T00:19:44.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5910000000000002e-06 2024-09-18T00:19:44.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.417 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.417 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827811+0000", 2024-09-18T00:19:44.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000216858 2024-09-18T00:19:44.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.417 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:19:44.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827818+0000", 2024-09-18T00:19:44.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.99e-06 2024-09-18T00:19:44.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.418 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.418 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829799+0000", 2024-09-18T00:19:44.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019803849999999999 2024-09-18T00:19:44.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829884+0000", 2024-09-18T00:19:44.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4852999999999995e-05 2024-09-18T00:19:44.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.419 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:44.419 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915616+0000", 2024-09-18T00:19:44.419 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085732808999999993 2024-09-18T00:19:44.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.419 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:44.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915711+0000", 2024-09-18T00:19:44.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4578000000000001e-05 2024-09-18T00:19:44.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915880+0000", 2024-09-18T00:19:44.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016861999999999999 2024-09-18T00:19:44.420 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.420 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.420 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.421 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:42788 3.7 3:e5a8432b:::rbd_data.112a3496573c.0000000000000027:17 [read 309248~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:44.421 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:44.421 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 23.586036833000001, 2024-09-18T00:19:44.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.039621847000000002, 2024-09-18T00:19:44.421 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.421 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.421 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:44.421 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.421 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.421 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.422 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 42788 2024-09-18T00:19:44.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.422 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:44.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:44.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718971+0000", 2024-09-18T00:19:44.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2500000000000001e-07 2024-09-18T00:19:44.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718973+0000", 2024-09-18T00:19:44.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2510000000000001e-06 2024-09-18T00:19:44.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718979+0000", 2024-09-18T00:19:44.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1700000000000002e-06 2024-09-18T00:19:44.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718984+0000", 2024-09-18T00:19:44.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9110000000000001e-06 2024-09-18T00:19:44.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.749126+0000", 2024-09-18T00:19:44.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030142164999999999 2024-09-18T00:19:44.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.749199+0000", 2024-09-18T00:19:44.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2985999999999995e-05 2024-09-18T00:19:44.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.758592+0000", 2024-09-18T00:19:44.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093929389999999995 2024-09-18T00:19:44.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.426 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.426 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.426 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:54153 3.b 3:d3f7b2f3:::rbd_data.112a859a01c9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,zero 4596947~21151] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:19:44.426 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:19:44.426 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 13.452235897, 2024-09-18T00:19:44.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062612886000000006, 2024-09-18T00:19:44.427 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.427 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.427 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.427 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.427 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.427 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.427 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 54153 2024-09-18T00:19:44.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.428 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:19:44.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:19:44.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852773+0000", 2024-09-18T00:19:44.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.426e-06 2024-09-18T00:19:44.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855337+0000", 2024-09-18T00:19:44.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025647830000000002 2024-09-18T00:19:44.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855369+0000", 2024-09-18T00:19:44.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1300999999999997e-05 2024-09-18T00:19:44.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855387+0000", 2024-09-18T00:19:44.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7949999999999999e-05 2024-09-18T00:19:44.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855487+0000", 2024-09-18T00:19:44.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010062 2024-09-18T00:19:44.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855686+0000", 2024-09-18T00:19:44.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019901399999999999 2024-09-18T00:19:44.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.431 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:44.431 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858037+0000", 2024-09-18T00:19:44.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002351264 2024-09-18T00:19:44.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:44.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.896302+0000", 2024-09-18T00:19:44.432 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038264864000000003 2024-09-18T00:19:44.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:44.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915250+0000", 2024-09-18T00:19:44.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018947538999999999 2024-09-18T00:19:44.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:44.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915318+0000", 2024-09-18T00:19:44.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7838999999999999e-05 2024-09-18T00:19:44.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915384+0000", 2024-09-18T00:19:44.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6285999999999995e-05 2024-09-18T00:19:44.433 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.434 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.434 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.434 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:55488 3.18 3:1d5573d8:::rbd_data.112a859a01c9.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T00:19:44.434 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:19:44.434 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 12.719405688, 2024-09-18T00:19:44.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045116205999999999, 2024-09-18T00:19:44.434 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:44.434 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:44.435 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.435 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:44.435 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:44.435 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.435 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 55488 2024-09-18T00:19:44.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.435 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:44.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:44.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:19:44.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:44.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:19:44.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:44.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:44.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585602+0000", 2024-09-18T00:19:44.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4000000000000001e-07 2024-09-18T00:19:44.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:44.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585607+0000", 2024-09-18T00:19:44.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5240000000000002e-06 2024-09-18T00:19:44.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:44.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585618+0000", 2024-09-18T00:19:44.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1493e-05 2024-09-18T00:19:44.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585624+0000", 2024-09-18T00:19:44.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7180000000000001e-06 2024-09-18T00:19:44.437 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:44.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585672+0000", 2024-09-18T00:19:44.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8158000000000002e-05 2024-09-18T00:19:44.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:44.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585780+0000", 2024-09-18T00:19:44.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010781399999999999 2024-09-18T00:19:44.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:44.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.588202+0000", 2024-09-18T00:19:44.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002422156 2024-09-18T00:19:44.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630375+0000", 2024-09-18T00:19:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042172820999999999 2024-09-18T00:19:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630426+0000", 2024-09-18T00:19:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1369000000000002e-05 2024-09-18T00:19:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630581+0000", 2024-09-18T00:19:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000155041 2024-09-18T00:19:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:44.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:44.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:44.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630718+0000", 2024-09-18T00:19:44.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000136372 2024-09-18T00:19:44.440 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.440 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.440 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.440 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:44.440 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:44.440 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:44.442 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:19:44.443 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:44.437+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:44.543 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:44.541+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:44.572 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:44.572 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:44.572 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:44.572 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:44.587 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:19:44.678 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:44.677+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:44.745 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:44.741+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:44.747 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:44.748 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:44.748 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:44.748 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:44.748 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:44.762 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:19:44.854 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:44.853+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:44.938 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:44.939 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:44.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:44.939 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:44.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.939 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2418 3.1a 3:59a8b40d:::rbd_data.112aa7ba4578.00000000000001b0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:19:44.939 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:44.939 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.201528057999994, 2024-09-18T00:19:44.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10488562999999999, 2024-09-18T00:19:44.939 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:44.939 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:44.939 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.940 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:44.940 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:44.940 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.940 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2418 2024-09-18T00:19:44.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.940 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:44.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:44.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:44.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:44.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:44.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:44.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735592+0000", 2024-09-18T00:19:44.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5229999999999998e-06 2024-09-18T00:19:44.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:44.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735604+0000", 2024-09-18T00:19:44.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2143e-05 2024-09-18T00:19:44.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:44.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735629+0000", 2024-09-18T00:19:44.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.497e-05 2024-09-18T00:19:44.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735648+0000", 2024-09-18T00:19:44.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8896999999999999e-05 2024-09-18T00:19:44.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:44.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735775+0000", 2024-09-18T00:19:44.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126935 2024-09-18T00:19:44.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:44.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.736118+0000", 2024-09-18T00:19:44.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034330800000000001 2024-09-18T00:19:44.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-18T00:19:44.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.741633+0000", 2024-09-18T00:19:44.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055144139999999996 2024-09-18T00:19:44.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:44.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840134+0000", 2024-09-18T00:19:44.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.098501618999999999 2024-09-18T00:19:44.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:44.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840184+0000", 2024-09-18T00:19:44.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9953000000000001e-05 2024-09-18T00:19:44.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:44.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840336+0000", 2024-09-18T00:19:44.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151476 2024-09-18T00:19:44.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:44.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840475+0000", 2024-09-18T00:19:44.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013939200000000001 2024-09-18T00:19:44.945 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.945 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:44.945 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.945 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:22272 3.3 3:c617f123:::rbd_data.112a948d5956.0000000000000009:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:44.945 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:44.945 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.946019859000003, 2024-09-18T00:19:44.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10752577300000001, 2024-09-18T00:19:44.945 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:44.946 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:44.946 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.946 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:44.946 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:44.946 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.946 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 22272 2024-09-18T00:19:44.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.946 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:44.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:44.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:44.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:44.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:44.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:44.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991102+0000", 2024-09-18T00:19:44.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9369999999999997e-06 2024-09-18T00:19:44.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:44.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991118+0000", 2024-09-18T00:19:44.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6396e-05 2024-09-18T00:19:44.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:44.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991150+0000", 2024-09-18T00:19:44.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2132999999999999e-05 2024-09-18T00:19:44.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991170+0000", 2024-09-18T00:19:44.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0085000000000002e-05 2024-09-18T00:19:44.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:44.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991370+0000", 2024-09-18T00:19:44.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019963700000000001 2024-09-18T00:19:44.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:44.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991518+0000", 2024-09-18T00:19:44.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000148424 2024-09-18T00:19:44.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:44.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.998850+0000", 2024-09-18T00:19:44.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073315480000000002 2024-09-18T00:19:44.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:44.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098178+0000", 2024-09-18T00:19:44.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.099328067000000006 2024-09-18T00:19:44.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:44.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098314+0000", 2024-09-18T00:19:44.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000135678 2024-09-18T00:19:44.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:44.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098498+0000", 2024-09-18T00:19:44.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018479600000000001 2024-09-18T00:19:44.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.952 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:44.952 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098623+0000", 2024-09-18T00:19:44.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000125072 2024-09-18T00:19:44.952 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.952 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:44.952 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.953 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:23689 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-18T00:19:44.953 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:44.953 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 43.084695081, 2024-09-18T00:19:44.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9562625809999998, 2024-09-18T00:19:44.953 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:44.953 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:44.953 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:44.953 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:44.953 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:44.953 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.954 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 23689 2024-09-18T00:19:44.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.954 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:44.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:44.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:44.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:44.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:44.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:44.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852423+0000", 2024-09-18T00:19:44.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9999999999999995e-07 2024-09-18T00:19:44.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:44.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852429+0000", 2024-09-18T00:19:44.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3279999999999996e-06 2024-09-18T00:19:44.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:44.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852442+0000", 2024-09-18T00:19:44.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3052e-05 2024-09-18T00:19:44.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.957 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.957 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852448+0000", 2024-09-18T00:19:44.957 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2779999999999999e-06 2024-09-18T00:19:44.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.957 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:44.957 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852523+0000", 2024-09-18T00:19:44.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4463999999999997e-05 2024-09-18T00:19:44.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:44.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852538+0000", 2024-09-18T00:19:44.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5466999999999999e-05 2024-09-18T00:19:44.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:44.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793478+0000", 2024-09-18T00:19:44.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.94094012699999996 2024-09-18T00:19:44.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.960 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:44.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793494+0000", 2024-09-18T00:19:44.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5807999999999998e-05 2024-09-18T00:19:44.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.960 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:44.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.804875+0000", 2024-09-18T00:19:44.961 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0113811939999999 2024-09-18T00:19:44.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.961 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.961 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:44.961 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805359+0000", 2024-09-18T00:19:44.961 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000484248 2024-09-18T00:19:44.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.961 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.961 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-18T00:19:44.961 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805872+0000", 2024-09-18T00:19:44.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00051218100000000001 2024-09-18T00:19:44.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:44.962 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.807790+0000", 2024-09-18T00:19:44.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001918763 2024-09-18T00:19:44.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:44.962 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808242+0000", 2024-09-18T00:19:44.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045126200000000002 2024-09-18T00:19:44.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:44.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808428+0000", 2024-09-18T00:19:44.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018633000000000001 2024-09-18T00:19:44.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:44.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808685+0000", 2024-09-18T00:19:44.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025707900000000001 2024-09-18T00:19:44.963 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.963 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:44.964 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.964 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47924 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7831552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:44.964 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032564+0000", 2024-09-18T00:19:44.964 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904553102000001, 2024-09-18T00:19:44.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086266190000000006, 2024-09-18T00:19:44.964 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:44.964 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:44.964 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:44.965 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:44.965 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:44.965 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.965 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47924 2024-09-18T00:19:44.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.965 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:44.965 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.965 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:44.965 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032564+0000", 2024-09-18T00:19:44.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:44.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032564+0000", 2024-09-18T00:19:44.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:44.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032565+0000", 2024-09-18T00:19:44.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9599999999999999e-07 2024-09-18T00:19:44.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:44.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032567+0000", 2024-09-18T00:19:44.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.142e-06 2024-09-18T00:19:44.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:44.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032573+0000", 2024-09-18T00:19:44.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2659999999999998e-06 2024-09-18T00:19:44.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032578+0000", 2024-09-18T00:19:44.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9019999999999998e-06 2024-09-18T00:19:44.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:44.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118632+0000", 2024-09-18T00:19:44.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086053942999999994 2024-09-18T00:19:44.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:44.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118706+0000", 2024-09-18T00:19:44.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4266999999999999e-05 2024-09-18T00:19:44.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:44.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118831+0000", 2024-09-18T00:19:44.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012417400000000001 2024-09-18T00:19:44.969 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.969 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:44.969 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.969 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47925 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7864320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:44.969 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:19:44.969 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904532979999999, 2024-09-18T00:19:44.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086484302999999998, 2024-09-18T00:19:44.969 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:44.969 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:44.969 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:44.969 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:44.970 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:44.970 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.970 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47925 2024-09-18T00:19:44.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.970 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:44.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.970 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:44.970 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:19:44.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.970 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:44.970 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:19:44.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:44.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:19:44.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.82e-07 2024-09-18T00:19:44.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:44.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032587+0000", 2024-09-18T00:19:44.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.058e-06 2024-09-18T00:19:44.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:44.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032593+0000", 2024-09-18T00:19:44.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1380000000000004e-06 2024-09-18T00:19:44.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032598+0000", 2024-09-18T00:19:44.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9819999999999999e-06 2024-09-18T00:19:44.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:44.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118873+0000", 2024-09-18T00:19:44.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086274609000000002 2024-09-18T00:19:44.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:44.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118950+0000", 2024-09-18T00:19:44.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6687999999999995e-05 2024-09-18T00:19:44.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:44.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119069+0000", 2024-09-18T00:19:44.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011934600000000001 2024-09-18T00:19:44.973 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.973 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:44.973 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.973 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47926 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7897088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:44.973 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032604+0000", 2024-09-18T00:19:44.973 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904513103999999, 2024-09-18T00:19:44.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086712168000000006, 2024-09-18T00:19:44.974 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:44.974 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:44.974 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:44.974 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:44.974 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:44.974 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.974 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47926 2024-09-18T00:19:44.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.974 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:44.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.974 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:44.974 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032604+0000", 2024-09-18T00:19:44.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:44.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032604+0000", 2024-09-18T00:19:44.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:44.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032605+0000", 2024-09-18T00:19:44.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:19:44.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:44.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032611+0000", 2024-09-18T00:19:44.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4860000000000002e-06 2024-09-18T00:19:44.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:44.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032618+0000", 2024-09-18T00:19:44.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6699999999999997e-06 2024-09-18T00:19:44.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032623+0000", 2024-09-18T00:19:44.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0860000000000001e-06 2024-09-18T00:19:44.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:44.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119111+0000", 2024-09-18T00:19:44.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086488067000000002 2024-09-18T00:19:44.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:44.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119185+0000", 2024-09-18T00:19:44.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3758999999999996e-05 2024-09-18T00:19:44.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:44.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119317+0000", 2024-09-18T00:19:44.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013159599999999999 2024-09-18T00:19:44.977 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.977 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:44.978 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.978 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47927 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7929856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:44.978 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:19:44.978 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904487762999999, 2024-09-18T00:19:44.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086929532000000004, 2024-09-18T00:19:44.978 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:44.978 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:44.978 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:44.978 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:44.979 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:44.979 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.979 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47927 2024-09-18T00:19:44.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.979 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:44.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:44.979 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:19:44.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:44.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:19:44.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:44.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:19:44.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-18T00:19:44.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:44.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032632+0000", 2024-09-18T00:19:44.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1320000000000001e-06 2024-09-18T00:19:44.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:44.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032639+0000", 2024-09-18T00:19:44.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1840000000000004e-06 2024-09-18T00:19:44.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032644+0000", 2024-09-18T00:19:44.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2519999999999999e-06 2024-09-18T00:19:44.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:44.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119358+0000", 2024-09-18T00:19:44.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086714075000000002 2024-09-18T00:19:44.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.983 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:44.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119438+0000", 2024-09-18T00:19:44.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9756e-05 2024-09-18T00:19:44.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.983 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:44.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119559+0000", 2024-09-18T00:19:44.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121579 2024-09-18T00:19:44.983 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.983 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:44.984 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.984 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47928 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7962624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:44.984 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032650+0000", 2024-09-18T00:19:44.984 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904467339, 2024-09-18T00:19:44.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087152732999999996, 2024-09-18T00:19:44.985 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:44.985 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:44.985 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:44.985 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:44.985 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:44.985 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.985 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47928 2024-09-18T00:19:44.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.985 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:44.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:44.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032650+0000", 2024-09-18T00:19:44.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:44.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032650+0000", 2024-09-18T00:19:44.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:44.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032651+0000", 2024-09-18T00:19:44.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:19:44.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:44.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032653+0000", 2024-09-18T00:19:44.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.182e-06 2024-09-18T00:19:44.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:44.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032659+0000", 2024-09-18T00:19:44.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2319999999999998e-06 2024-09-18T00:19:44.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032664+0000", 2024-09-18T00:19:44.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0420000000000002e-06 2024-09-18T00:19:44.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:44.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119602+0000", 2024-09-18T00:19:44.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086938213 2024-09-18T00:19:44.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:44.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119676+0000", 2024-09-18T00:19:44.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3708000000000006e-05 2024-09-18T00:19:44.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:44.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119803+0000", 2024-09-18T00:19:44.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126852 2024-09-18T00:19:44.990 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.990 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:44.990 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.990 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47929 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7995392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:44.990 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032670+0000", 2024-09-18T00:19:44.991 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904447081000001, 2024-09-18T00:19:44.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087373082000000005, 2024-09-18T00:19:44.991 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:44.991 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:44.991 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:44.992 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:44.992 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:44.992 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.992 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47929 2024-09-18T00:19:44.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.992 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:44.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:44.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032670+0000", 2024-09-18T00:19:44.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:44.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032670+0000", 2024-09-18T00:19:44.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:44.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:44.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032671+0000", 2024-09-18T00:19:44.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:19:44.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:44.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032673+0000", 2024-09-18T00:19:44.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.928e-06 2024-09-18T00:19:44.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:44.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032679+0000", 2024-09-18T00:19:44.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.02e-06 2024-09-18T00:19:44.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:44.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032684+0000", 2024-09-18T00:19:44.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1660000000000002e-06 2024-09-18T00:19:44.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:44.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119844+0000", 2024-09-18T00:19:44.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087160248999999995 2024-09-18T00:19:44.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:44.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119920+0000", 2024-09-18T00:19:44.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5181999999999999e-05 2024-09-18T00:19:44.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:44.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120044+0000", 2024-09-18T00:19:44.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012403300000000001 2024-09-18T00:19:44.998 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.998 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:44.998 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:44.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.998 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47930 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8028160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:44.998 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032690+0000", 2024-09-18T00:19:44.998 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904427228999999, 2024-09-18T00:19:44.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087595941999999996, 2024-09-18T00:19:44.999 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:44.999 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:44.999 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:44.999 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:44.999 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:44.999 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:44.999 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47930 2024-09-18T00:19:44.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:44.999 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:44.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:44.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:44.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032690+0000", 2024-09-18T00:19:44.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032690+0000", 2024-09-18T00:19:45.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032691+0000", 2024-09-18T00:19:45.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9399999999999995e-07 2024-09-18T00:19:45.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032693+0000", 2024-09-18T00:19:45.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1299999999999999e-06 2024-09-18T00:19:45.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032699+0000", 2024-09-18T00:19:45.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1879999999999999e-06 2024-09-18T00:19:45.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032704+0000", 2024-09-18T00:19:45.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0939999999999998e-06 2024-09-18T00:19:45.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120087+0000", 2024-09-18T00:19:45.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087383049000000004 2024-09-18T00:19:45.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120164+0000", 2024-09-18T00:19:45.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6679000000000002e-05 2024-09-18T00:19:45.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120286+0000", 2024-09-18T00:19:45.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012230800000000001 2024-09-18T00:19:45.002 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.002 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.002 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.003 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47931 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8060928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:45.003 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:19:45.003 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904406986000001, 2024-09-18T00:19:45.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087823688999999996, 2024-09-18T00:19:45.003 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.003 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.003 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.003 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.003 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.003 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.003 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47931 2024-09-18T00:19:45.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.004 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:19:45.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:19:45.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:19:45.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9200000000000001e-07 2024-09-18T00:19:45.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032713+0000", 2024-09-18T00:19:45.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1239999999999999e-06 2024-09-18T00:19:45.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032719+0000", 2024-09-18T00:19:45.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1659999999999999e-06 2024-09-18T00:19:45.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032724+0000", 2024-09-18T00:19:45.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9359999999999998e-06 2024-09-18T00:19:45.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120330+0000", 2024-09-18T00:19:45.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087605559 2024-09-18T00:19:45.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120403+0000", 2024-09-18T00:19:45.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-05 2024-09-18T00:19:45.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120534+0000", 2024-09-18T00:19:45.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013121199999999999 2024-09-18T00:19:45.007 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.007 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.007 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.008 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47932 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8093696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:45.008 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032730+0000", 2024-09-18T00:19:45.008 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904387088, 2024-09-18T00:19:45.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088052486999999999, 2024-09-18T00:19:45.008 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.008 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.008 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.008 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.008 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.008 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.009 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47932 2024-09-18T00:19:45.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.009 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032730+0000", 2024-09-18T00:19:45.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032730+0000", 2024-09-18T00:19:45.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032731+0000", 2024-09-18T00:19:45.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2200000000000004e-07 2024-09-18T00:19:45.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032743+0000", 2024-09-18T00:19:45.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1796e-05 2024-09-18T00:19:45.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032749+0000", 2024-09-18T00:19:45.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6959999999999999e-06 2024-09-18T00:19:45.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032755+0000", 2024-09-18T00:19:45.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.22e-06 2024-09-18T00:19:45.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120578+0000", 2024-09-18T00:19:45.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087823017000000003 2024-09-18T00:19:45.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120654+0000", 2024-09-18T00:19:45.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6476000000000004e-05 2024-09-18T00:19:45.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120783+0000", 2024-09-18T00:19:45.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012876000000000001 2024-09-18T00:19:45.013 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.013 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.013 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.013 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47933 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8126464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:45.013 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032761+0000", 2024-09-18T00:19:45.013 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904356069999999, 2024-09-18T00:19:45.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088263233999999996, 2024-09-18T00:19:45.013 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.014 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.014 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.014 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.014 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.014 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.014 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47933 2024-09-18T00:19:45.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.014 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032761+0000", 2024-09-18T00:19:45.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032761+0000", 2024-09-18T00:19:45.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032762+0000", 2024-09-18T00:19:45.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:19:45.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032764+0000", 2024-09-18T00:19:45.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1519999999999999e-06 2024-09-18T00:19:45.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032770+0000", 2024-09-18T00:19:45.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1700000000000002e-06 2024-09-18T00:19:45.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032778+0000", 2024-09-18T00:19:45.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2400000000000001e-06 2024-09-18T00:19:45.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120825+0000", 2024-09-18T00:19:45.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088047046000000004 2024-09-18T00:19:45.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120898+0000", 2024-09-18T00:19:45.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3011999999999998e-05 2024-09-18T00:19:45.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121025+0000", 2024-09-18T00:19:45.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012710999999999999 2024-09-18T00:19:45.018 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.018 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.018 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.019 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47934 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8159232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:45.019 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032784+0000", 2024-09-18T00:19:45.019 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904333519000001, 2024-09-18T00:19:45.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088479458999999996, 2024-09-18T00:19:45.019 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.019 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.019 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.019 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.019 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.019 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.020 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47934 2024-09-18T00:19:45.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.020 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032784+0000", 2024-09-18T00:19:45.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032784+0000", 2024-09-18T00:19:45.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032785+0000", 2024-09-18T00:19:45.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9599999999999999e-07 2024-09-18T00:19:45.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032787+0000", 2024-09-18T00:19:45.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.052e-06 2024-09-18T00:19:45.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032793+0000", 2024-09-18T00:19:45.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2439999999999998e-06 2024-09-18T00:19:45.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032798+0000", 2024-09-18T00:19:45.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9080000000000003e-06 2024-09-18T00:19:45.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121066+0000", 2024-09-18T00:19:45.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088267795999999996 2024-09-18T00:19:45.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121139+0000", 2024-09-18T00:19:45.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2910999999999994e-05 2024-09-18T00:19:45.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121263+0000", 2024-09-18T00:19:45.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000124852 2024-09-18T00:19:45.024 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.024 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.024 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.024 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47935 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8192000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:45.024 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032841+0000", 2024-09-18T00:19:45.025 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904276439, 2024-09-18T00:19:45.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088666069, 2024-09-18T00:19:45.025 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.025 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.025 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.025 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.025 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.025 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.025 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47935 2024-09-18T00:19:45.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.025 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032841+0000", 2024-09-18T00:19:45.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032841+0000", 2024-09-18T00:19:45.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032842+0000", 2024-09-18T00:19:45.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6400000000000002e-07 2024-09-18T00:19:45.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032845+0000", 2024-09-18T00:19:45.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8959999999999999e-06 2024-09-18T00:19:45.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032852+0000", 2024-09-18T00:19:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1640000000000004e-06 2024-09-18T00:19:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032857+0000", 2024-09-18T00:19:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3340000000000001e-06 2024-09-18T00:19:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121304+0000", 2024-09-18T00:19:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088447001999999997 2024-09-18T00:19:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121376+0000", 2024-09-18T00:19:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2121999999999998e-05 2024-09-18T00:19:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121507+0000", 2024-09-18T00:19:45.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130987 2024-09-18T00:19:45.030 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.030 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.030 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.030 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47936 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:45.031 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:19:45.031 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904253718, 2024-09-18T00:19:45.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088878349999999995, 2024-09-18T00:19:45.031 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.031 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.031 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.031 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.031 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.031 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.031 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47936 2024-09-18T00:19:45.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.031 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:19:45.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:19:45.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:19:45.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8999999999999997e-07 2024-09-18T00:19:45.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032866+0000", 2024-09-18T00:19:45.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.142e-06 2024-09-18T00:19:45.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032873+0000", 2024-09-18T00:19:45.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:19:45.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032878+0000", 2024-09-18T00:19:45.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1220000000000003e-06 2024-09-18T00:19:45.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121548+0000", 2024-09-18T00:19:45.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088670530999999997 2024-09-18T00:19:45.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121621+0000", 2024-09-18T00:19:45.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2941999999999994e-05 2024-09-18T00:19:45.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121742+0000", 2024-09-18T00:19:45.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120903 2024-09-18T00:19:45.035 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.035 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.035 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.035 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47937 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8257536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:45.035 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:19:45.035 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904233609999999, 2024-09-18T00:19:45.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089098572000000001, 2024-09-18T00:19:45.036 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.036 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.036 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.036 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.036 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.036 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.036 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47937 2024-09-18T00:19:45.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.036 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:19:45.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:19:45.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:19:45.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9200000000000001e-07 2024-09-18T00:19:45.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032886+0000", 2024-09-18T00:19:45.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9999999999999999e-06 2024-09-18T00:19:45.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032893+0000", 2024-09-18T00:19:45.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2020000000000001e-06 2024-09-18T00:19:45.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032898+0000", 2024-09-18T00:19:45.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9860000000000002e-06 2024-09-18T00:19:45.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121785+0000", 2024-09-18T00:19:45.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088887318000000007 2024-09-18T00:19:45.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121858+0000", 2024-09-18T00:19:45.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2900999999999999e-05 2024-09-18T00:19:45.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121983+0000", 2024-09-18T00:19:45.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000124673 2024-09-18T00:19:45.040 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.040 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.040 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.041 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47938 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8290304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:45.041 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:19:45.041 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904213825999999, 2024-09-18T00:19:45.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089335780000000004, 2024-09-18T00:19:45.041 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.041 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.041 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.041 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.041 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.041 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.041 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47938 2024-09-18T00:19:45.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.042 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:19:45.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:19:45.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:19:45.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2799999999999996e-07 2024-09-18T00:19:45.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032923+0000", 2024-09-18T00:19:45.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8218000000000001e-05 2024-09-18T00:19:45.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032930+0000", 2024-09-18T00:19:45.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5440000000000001e-06 2024-09-18T00:19:45.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032936+0000", 2024-09-18T00:19:45.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4720000000000003e-06 2024-09-18T00:19:45.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122023+0000", 2024-09-18T00:19:45.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089087728000000005 2024-09-18T00:19:45.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122117+0000", 2024-09-18T00:19:45.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3295999999999996e-05 2024-09-18T00:19:45.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122240+0000", 2024-09-18T00:19:45.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012289399999999999 2024-09-18T00:19:45.045 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.046 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.046 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.046 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47939 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8323072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:45.046 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032943+0000", 2024-09-18T00:19:45.046 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904174272999999, 2024-09-18T00:19:45.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089547929999999998, 2024-09-18T00:19:45.046 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.046 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.046 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.046 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.046 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.047 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.047 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47939 2024-09-18T00:19:45.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.047 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032943+0000", 2024-09-18T00:19:45.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032943+0000", 2024-09-18T00:19:45.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032944+0000", 2024-09-18T00:19:45.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6199999999999998e-07 2024-09-18T00:19:45.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032946+0000", 2024-09-18T00:19:45.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2840000000000001e-06 2024-09-18T00:19:45.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032952+0000", 2024-09-18T00:19:45.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2700000000000001e-06 2024-09-18T00:19:45.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032957+0000", 2024-09-18T00:19:45.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0200000000000002e-06 2024-09-18T00:19:45.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122280+0000", 2024-09-18T00:19:45.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089322330000000005 2024-09-18T00:19:45.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122354+0000", 2024-09-18T00:19:45.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3794000000000005e-05 2024-09-18T00:19:45.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122491+0000", 2024-09-18T00:19:45.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013767 2024-09-18T00:19:45.050 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.051 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.051 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.051 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47940 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8355840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:45.051 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:19:45.051 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.904153845, 2024-09-18T00:19:45.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089758663000000002, 2024-09-18T00:19:45.051 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.051 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.051 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.051 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.052 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.052 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.052 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47940 2024-09-18T00:19:45.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.052 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:19:45.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:19:45.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:19:45.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8599999999999998e-07 2024-09-18T00:19:45.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032966+0000", 2024-09-18T00:19:45.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1160000000000002e-06 2024-09-18T00:19:45.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032972+0000", 2024-09-18T00:19:45.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1639999999999998e-06 2024-09-18T00:19:45.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032978+0000", 2024-09-18T00:19:45.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1320000000000002e-06 2024-09-18T00:19:45.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122520+0000", 2024-09-18T00:19:45.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089541930000000006 2024-09-18T00:19:45.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122601+0000", 2024-09-18T00:19:45.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1207999999999999e-05 2024-09-18T00:19:45.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122722+0000", 2024-09-18T00:19:45.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012162700000000001 2024-09-18T00:19:45.056 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.056 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.056 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.056 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.056 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.056 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:45.058 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:19:45.058 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:45.053+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:45.158 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:45.157+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:45.228 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:45.229 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:45.229 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:45.229 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:45.242 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:19:45.286 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:45.285+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:45.360 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:45.357+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:45.408 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:45.408 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:45.409 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:45.409 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:45.409 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:45.423 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:19:45.462 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:45.461+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:45.563 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:45.561+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:45.594 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:45.594 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:45.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2229 3.10 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.327304565999995, 2024-09-18T00:19:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048730086999999998, 2024-09-18T00:19:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2229 2024-09-18T00:19:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263362+0000", 2024-09-18T00:19:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3869999999999999e-06 2024-09-18T00:19:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263381+0000", 2024-09-18T00:19:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9153000000000001e-05 2024-09-18T00:19:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263418+0000", 2024-09-18T00:19:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6346000000000001e-05 2024-09-18T00:19:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263435+0000", 2024-09-18T00:19:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7309999999999999e-05 2024-09-18T00:19:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263589+0000", 2024-09-18T00:19:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015451199999999999 2024-09-18T00:19:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263902+0000", 2024-09-18T00:19:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031291799999999997 2024-09-18T00:19:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.265995+0000", 2024-09-18T00:19:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020931809999999999 2024-09-18T00:19:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311788+0000", 2024-09-18T00:19:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045792078 2024-09-18T00:19:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311903+0000", 2024-09-18T00:19:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011510699999999999 2024-09-18T00:19:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312009+0000", 2024-09-18T00:19:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010630599999999999 2024-09-18T00:19:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312089+0000", 2024-09-18T00:19:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9789000000000005e-05 2024-09-18T00:19:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2229 3.10 e36/23 ondisk, result = 0)", 2024-09-18T00:19:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.308158907000006, 2024-09-18T00:19:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029600955000000002, 2024-09-18T00:19:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282507+0000", 2024-09-18T00:19:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9160000000000001e-06 2024-09-18T00:19:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282522+0000", 2024-09-18T00:19:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5e-05 2024-09-18T00:19:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282558+0000", 2024-09-18T00:19:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5694000000000001e-05 2024-09-18T00:19:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282580+0000", 2024-09-18T00:19:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2323e-05 2024-09-18T00:19:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311873+0000", 2024-09-18T00:19:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029293013999999999 2024-09-18T00:19:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311892+0000", 2024-09-18T00:19:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8556e-05 2024-09-18T00:19:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312105+0000", 2024-09-18T00:19:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021345199999999999 2024-09-18T00:19:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5714 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 80.280456404000006, 2024-09-18T00:19:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027448727999999999, 2024-09-18T00:19:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5714 2024-09-18T00:19:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310211+0000", 2024-09-18T00:19:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5639999999999998e-06 2024-09-18T00:19:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312678+0000", 2024-09-18T00:19:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002466761 2024-09-18T00:19:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312746+0000", 2024-09-18T00:19:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8046000000000005e-05 2024-09-18T00:19:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312774+0000", 2024-09-18T00:19:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7813e-05 2024-09-18T00:19:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312927+0000", 2024-09-18T00:19:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015328499999999999 2024-09-18T00:19:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313214+0000", 2024-09-18T00:19:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028683099999999998 2024-09-18T00:19:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313708+0000", 2024-09-18T00:19:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049379100000000004 2024-09-18T00:19:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336895+0000", 2024-09-18T00:19:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023186801 2024-09-18T00:19:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337454+0000", 2024-09-18T00:19:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055979399999999996 2024-09-18T00:19:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:45.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337571+0000", 2024-09-18T00:19:45.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116821 2024-09-18T00:19:45.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337656+0000", 2024-09-18T00:19:45.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4221000000000006e-05 2024-09-18T00:19:45.612 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.613 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.613 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.613 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6801 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:45.613 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:45.613 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 79.553619956000006, 2024-09-18T00:19:45.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051953602000000002, 2024-09-18T00:19:45.613 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.613 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.613 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:45.614 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.614 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.614 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.615 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6801 2024-09-18T00:19:45.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.616 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:45.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:45.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037046+0000", 2024-09-18T00:19:45.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.19e-06 2024-09-18T00:19:45.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037062+0000", 2024-09-18T00:19:45.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586999999999998e-05 2024-09-18T00:19:45.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037096+0000", 2024-09-18T00:19:45.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4112000000000001e-05 2024-09-18T00:19:45.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037120+0000", 2024-09-18T00:19:45.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.355e-05 2024-09-18T00:19:45.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037246+0000", 2024-09-18T00:19:45.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012652200000000001 2024-09-18T00:19:45.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037385+0000", 2024-09-18T00:19:45.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013924599999999999 2024-09-18T00:19:45.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:45.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.040881+0000", 2024-09-18T00:19:45.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034954830000000002 2024-09-18T00:19:45.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:45.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088794+0000", 2024-09-18T00:19:45.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047912897000000003 2024-09-18T00:19:45.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:45.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088828+0000", 2024-09-18T00:19:45.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3880000000000001e-05 2024-09-18T00:19:45.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:45.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088916+0000", 2024-09-18T00:19:45.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.865e-05 2024-09-18T00:19:45.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088997+0000", 2024-09-18T00:19:45.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0484999999999999e-05 2024-09-18T00:19:45.621 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.621 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.621 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.621 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:6801 3.10 e42/23 ondisk, result = 0)", 2024-09-18T00:19:45.621 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:45.621 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 79.539812545999993, 2024-09-18T00:19:45.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037960394000000001, 2024-09-18T00:19:45.621 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.621 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.621 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.622 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:45.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:45.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050852+0000", 2024-09-18T00:19:45.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:19:45.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050860+0000", 2024-09-18T00:19:45.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7729999999999999e-06 2024-09-18T00:19:45.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050891+0000", 2024-09-18T00:19:45.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0617000000000002e-05 2024-09-18T00:19:45.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050910+0000", 2024-09-18T00:19:45.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.944e-05 2024-09-18T00:19:45.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088766+0000", 2024-09-18T00:19:45.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037855633999999999 2024-09-18T00:19:45.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088784+0000", 2024-09-18T00:19:45.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7767000000000001e-05 2024-09-18T00:19:45.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088811+0000", 2024-09-18T00:19:45.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7382999999999999e-05 2024-09-18T00:19:45.625 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.625 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.625 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.625 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12594 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:45.625 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:45.625 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.199036007999993, 2024-09-18T00:19:45.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026492600000000002, 2024-09-18T00:19:45.626 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.626 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.626 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.626 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.626 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.626 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.626 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12594 2024-09-18T00:19:45.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.626 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:45.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:45.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391629+0000", 2024-09-18T00:19:45.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.86e-06 2024-09-18T00:19:45.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391637+0000", 2024-09-18T00:19:45.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3999999999999992e-06 2024-09-18T00:19:45.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391664+0000", 2024-09-18T00:19:45.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6536000000000001e-05 2024-09-18T00:19:45.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391681+0000", 2024-09-18T00:19:45.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6563e-05 2024-09-18T00:19:45.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417791+0000", 2024-09-18T00:19:45.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026110936000000001 2024-09-18T00:19:45.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417933+0000", 2024-09-18T00:19:45.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014112099999999999 2024-09-18T00:19:45.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418120+0000", 2024-09-18T00:19:45.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018718399999999999 2024-09-18T00:19:45.630 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.630 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.630 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.630 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12595 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:45.630 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:45.630 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.198959810000005, 2024-09-18T00:19:45.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026819428999999999, 2024-09-18T00:19:45.630 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.630 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.630 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.630 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.631 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.631 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.631 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12595 2024-09-18T00:19:45.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.631 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:45.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:45.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391705+0000", 2024-09-18T00:19:45.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3999999999999999e-06 2024-09-18T00:19:45.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391711+0000", 2024-09-18T00:19:45.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7459999999999997e-06 2024-09-18T00:19:45.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391733+0000", 2024-09-18T00:19:45.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2923000000000001e-05 2024-09-18T00:19:45.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391748+0000", 2024-09-18T00:19:45.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4790000000000001e-05 2024-09-18T00:19:45.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418180+0000", 2024-09-18T00:19:45.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026431888000000001 2024-09-18T00:19:45.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418328+0000", 2024-09-18T00:19:45.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000147725 2024-09-18T00:19:45.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418523+0000", 2024-09-18T00:19:45.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000194957 2024-09-18T00:19:45.635 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.635 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.635 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.635 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12596 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:45.635 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:45.635 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.198896841999996, 2024-09-18T00:19:45.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027220296000000001, 2024-09-18T00:19:45.636 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.636 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.636 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.636 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.636 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.636 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.636 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12596 2024-09-18T00:19:45.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.636 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:45.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:45.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391768+0000", 2024-09-18T00:19:45.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.283e-06 2024-09-18T00:19:45.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391773+0000", 2024-09-18T00:19:45.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3970000000000002e-06 2024-09-18T00:19:45.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391795+0000", 2024-09-18T00:19:45.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1835e-05 2024-09-18T00:19:45.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391809+0000", 2024-09-18T00:19:45.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4107000000000001e-05 2024-09-18T00:19:45.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418582+0000", 2024-09-18T00:19:45.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026772918999999999 2024-09-18T00:19:45.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418780+0000", 2024-09-18T00:19:45.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000198042 2024-09-18T00:19:45.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418987+0000", 2024-09-18T00:19:45.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000206713 2024-09-18T00:19:45.639 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.640 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.640 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.640 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12597 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:45.640 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:45.640 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.198836323999998, 2024-09-18T00:19:45.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027612714, 2024-09-18T00:19:45.640 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.640 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.640 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.640 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.640 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.640 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.641 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12597 2024-09-18T00:19:45.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.641 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:45.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:45.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391828+0000", 2024-09-18T00:19:45.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.203e-06 2024-09-18T00:19:45.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391834+0000", 2024-09-18T00:19:45.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6529999999999999e-06 2024-09-18T00:19:45.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391856+0000", 2024-09-18T00:19:45.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2195999999999999e-05 2024-09-18T00:19:45.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391870+0000", 2024-09-18T00:19:45.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4063000000000001e-05 2024-09-18T00:19:45.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419049+0000", 2024-09-18T00:19:45.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027178751000000001 2024-09-18T00:19:45.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419239+0000", 2024-09-18T00:19:45.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019039400000000001 2024-09-18T00:19:45.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419440+0000", 2024-09-18T00:19:45.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000200454 2024-09-18T00:19:45.644 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.644 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.644 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.644 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12598 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:45.644 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:45.644 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.198773273, 2024-09-18T00:19:45.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028269056000000001, 2024-09-18T00:19:45.645 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.645 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.645 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.645 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.645 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.645 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.645 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12598 2024-09-18T00:19:45.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.645 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:45.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:45.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391891+0000", 2024-09-18T00:19:45.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.257e-06 2024-09-18T00:19:45.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391897+0000", 2024-09-18T00:19:45.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9560000000000002e-06 2024-09-18T00:19:45.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391920+0000", 2024-09-18T00:19:45.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3025999999999999e-05 2024-09-18T00:19:45.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391935+0000", 2024-09-18T00:19:45.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4853e-05 2024-09-18T00:19:45.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419527+0000", 2024-09-18T00:19:45.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027592378000000001 2024-09-18T00:19:45.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419681+0000", 2024-09-18T00:19:45.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015316 2024-09-18T00:19:45.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420159+0000", 2024-09-18T00:19:45.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000478426 2024-09-18T00:19:45.648 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.648 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.649 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.649 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12599 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:45.649 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:45.649 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.198709418000007, 2024-09-18T00:19:45.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028649857000000001, 2024-09-18T00:19:45.649 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.649 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.649 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.649 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.649 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.649 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.649 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12599 2024-09-18T00:19:45.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.650 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:45.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:45.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391955+0000", 2024-09-18T00:19:45.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-06 2024-09-18T00:19:45.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391961+0000", 2024-09-18T00:19:45.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7100000000000004e-06 2024-09-18T00:19:45.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391982+0000", 2024-09-18T00:19:45.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1492e-05 2024-09-18T00:19:45.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391996+0000", 2024-09-18T00:19:45.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4194e-05 2024-09-18T00:19:45.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420237+0000", 2024-09-18T00:19:45.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028240818000000001 2024-09-18T00:19:45.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420430+0000", 2024-09-18T00:19:45.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019228299999999999 2024-09-18T00:19:45.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420604+0000", 2024-09-18T00:19:45.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000174117 2024-09-18T00:19:45.653 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.653 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.653 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.653 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12600 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 270336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:45.653 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:45.653 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.198648423999998, 2024-09-18T00:19:45.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028964586, 2024-09-18T00:19:45.653 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.654 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.654 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.654 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.654 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.654 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.654 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12600 2024-09-18T00:19:45.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.654 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:45.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:45.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392016+0000", 2024-09-18T00:19:45.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.31e-06 2024-09-18T00:19:45.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392022+0000", 2024-09-18T00:19:45.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5600000000000001e-06 2024-09-18T00:19:45.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392045+0000", 2024-09-18T00:19:45.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3595999999999999e-05 2024-09-18T00:19:45.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392060+0000", 2024-09-18T00:19:45.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4727e-05 2024-09-18T00:19:45.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420669+0000", 2024-09-18T00:19:45.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028608855999999998 2024-09-18T00:19:45.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420813+0000", 2024-09-18T00:19:45.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014429500000000001 2024-09-18T00:19:45.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420979+0000", 2024-09-18T00:19:45.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016624200000000001 2024-09-18T00:19:45.657 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.657 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.657 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12601 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.198584518999994, 2024-09-18T00:19:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029292832000000001, 2024-09-18T00:19:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12601 2024-09-18T00:19:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392080+0000", 2024-09-18T00:19:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392086+0000", 2024-09-18T00:19:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7570000000000001e-06 2024-09-18T00:19:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392105+0000", 2024-09-18T00:19:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9415999999999998e-05 2024-09-18T00:19:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392132+0000", 2024-09-18T00:19:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7019e-05 2024-09-18T00:19:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421049+0000", 2024-09-18T00:19:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028916659000000001 2024-09-18T00:19:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421204+0000", 2024-09-18T00:19:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155224 2024-09-18T00:19:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421372+0000", 2024-09-18T00:19:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016754699999999999 2024-09-18T00:19:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12602 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.198511887999999, 2024-09-18T00:19:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029612833000000002, 2024-09-18T00:19:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12602 2024-09-18T00:19:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392152+0000", 2024-09-18T00:19:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0470000000000001e-06 2024-09-18T00:19:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392158+0000", 2024-09-18T00:19:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:19:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392181+0000", 2024-09-18T00:19:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2629e-05 2024-09-18T00:19:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392222+0000", 2024-09-18T00:19:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0772000000000002e-05 2024-09-18T00:19:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421441+0000", 2024-09-18T00:19:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029219327 2024-09-18T00:19:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421608+0000", 2024-09-18T00:19:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000166813 2024-09-18T00:19:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421764+0000", 2024-09-18T00:19:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015628500000000001 2024-09-18T00:19:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12603 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.198421494000002, 2024-09-18T00:19:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029902524, 2024-09-18T00:19:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12603 2024-09-18T00:19:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392243+0000", 2024-09-18T00:19:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392260+0000", 2024-09-18T00:19:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6860000000000001e-05 2024-09-18T00:19:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392282+0000", 2024-09-18T00:19:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2713000000000001e-05 2024-09-18T00:19:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392297+0000", 2024-09-18T00:19:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4375e-05 2024-09-18T00:19:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421830+0000", 2024-09-18T00:19:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029532668000000002 2024-09-18T00:19:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421985+0000", 2024-09-18T00:19:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155613 2024-09-18T00:19:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422144+0000", 2024-09-18T00:19:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000159102 2024-09-18T00:19:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12604 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.198346443000005, 2024-09-18T00:19:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030228695, 2024-09-18T00:19:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12604 2024-09-18T00:19:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392318+0000", 2024-09-18T00:19:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392337+0000", 2024-09-18T00:19:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:19:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392364+0000", 2024-09-18T00:19:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7288999999999999e-05 2024-09-18T00:19:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392379+0000", 2024-09-18T00:19:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:19:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422209+0000", 2024-09-18T00:19:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029829050999999999 2024-09-18T00:19:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422388+0000", 2024-09-18T00:19:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017988699999999999 2024-09-18T00:19:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422545+0000", 2024-09-18T00:19:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015701199999999999 2024-09-18T00:19:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12605 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.198261881999997, 2024-09-18T00:19:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030488674, 2024-09-18T00:19:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12605 2024-09-18T00:19:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392403+0000", 2024-09-18T00:19:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.883e-06 2024-09-18T00:19:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392421+0000", 2024-09-18T00:19:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7526000000000001e-05 2024-09-18T00:19:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392450+0000", 2024-09-18T00:19:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8972e-05 2024-09-18T00:19:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392488+0000", 2024-09-18T00:19:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8522000000000001e-05 2024-09-18T00:19:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422614+0000", 2024-09-18T00:19:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030125945000000001 2024-09-18T00:19:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422752+0000", 2024-09-18T00:19:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000138178 2024-09-18T00:19:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422890+0000", 2024-09-18T00:19:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013764800000000001 2024-09-18T00:19:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12606 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.198153222000002, 2024-09-18T00:19:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033727073000000003, 2024-09-18T00:19:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12606 2024-09-18T00:19:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392511+0000", 2024-09-18T00:19:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3090000000000001e-06 2024-09-18T00:19:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392517+0000", 2024-09-18T00:19:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.993e-06 2024-09-18T00:19:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392543+0000", 2024-09-18T00:19:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6152999999999998e-05 2024-09-18T00:19:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392558+0000", 2024-09-18T00:19:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4610000000000001e-05 2024-09-18T00:19:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.425959+0000", 2024-09-18T00:19:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033400484000000001 2024-09-18T00:19:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426106+0000", 2024-09-18T00:19:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014779800000000001 2024-09-18T00:19:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426237+0000", 2024-09-18T00:19:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013072600000000001 2024-09-18T00:19:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17562 3.a 3:5578626c:::rbd_data.112a6efb7564.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T00:19:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.632688061000003, 2024-09-18T00:19:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052733453999999999, 2024-09-18T00:19:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17562 2024-09-18T00:19:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957977+0000", 2024-09-18T00:19:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.119e-06 2024-09-18T00:19:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957988+0000", 2024-09-18T00:19:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0557000000000001e-05 2024-09-18T00:19:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958017+0000", 2024-09-18T00:19:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9156e-05 2024-09-18T00:19:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958036+0000", 2024-09-18T00:19:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8919e-05 2024-09-18T00:19:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958144+0000", 2024-09-18T00:19:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107905 2024-09-18T00:19:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958284+0000", 2024-09-18T00:19:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140216 2024-09-18T00:19:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.960572+0000", 2024-09-18T00:19:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022881860000000002 2024-09-18T00:19:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010537+0000", 2024-09-18T00:19:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049964664999999998 2024-09-18T00:19:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010590+0000", 2024-09-18T00:19:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3094999999999999e-05 2024-09-18T00:19:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010642+0000", 2024-09-18T00:19:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2073000000000001e-05 2024-09-18T00:19:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010709+0000", 2024-09-18T00:19:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6563000000000006e-05 2024-09-18T00:19:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17562 3.a e60/23 ondisk, result = 0)", 2024-09-18T00:19:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.62248469, 2024-09-18T00:19:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042539307999999998, 2024-09-18T00:19:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968180+0000", 2024-09-18T00:19:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-18T00:19:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968188+0000", 2024-09-18T00:19:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7419999999999997e-06 2024-09-18T00:19:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968217+0000", 2024-09-18T00:19:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8926999999999999e-05 2024-09-18T00:19:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968231+0000", 2024-09-18T00:19:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4168999999999999e-05 2024-09-18T00:19:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010575+0000", 2024-09-18T00:19:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042343923999999998 2024-09-18T00:19:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010584+0000", 2024-09-18T00:19:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-18T00:19:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010718+0000", 2024-09-18T00:19:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013344499999999999 2024-09-18T00:19:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:45.694 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:45.695 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:19:45.697 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:19:45.697 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:45.693+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:45.713 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:19:45.796 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:45.793+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:45.870 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:45.871 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:45.871 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:45.871 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:45.885 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:19:45.902 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:45.903 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:45.903 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:45.920 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:45.920 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:45.930 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:45.929+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:45.936 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:45.936 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:45.954 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:45.954 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:45.972 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:45.972 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:45.992 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:45.992 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:45.998 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:45.997+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:46.007 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:46.007 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:46.007 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:46.008 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:46.008 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:46.012 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:46.013 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:46.016 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:46.016 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:46.022 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:19:46.099 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:46.097+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:46.190 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:46.190 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:46.200 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:46.197+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:46.205 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:19:46.301 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:46.297+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:46.367 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:46.367 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:46.381 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:19:46.401 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:46.401+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:46.502 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:46.501+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:46.506 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:46.506 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:46.520 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:19:46.606 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:46.605+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:46.620 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is unset 2024-09-18T00:19:46.666 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:19:46.676 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:19:46.673+0000 7f8bf65e57c0 -1 osd.1 45 log_to_monitors true 2024-09-18T00:19:46.689 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:46.690 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:46.704 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:46.701+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:46.706 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:19:46.801 INFO:teuthology.orchestra.run.smithi097.stderr:no valid command found; 10 closest matches: 2024-09-18T00:19:46.801 INFO:teuthology.orchestra.run.smithi097.stderr:0 2024-09-18T00:19:46.801 INFO:teuthology.orchestra.run.smithi097.stderr:1 2024-09-18T00:19:46.801 INFO:teuthology.orchestra.run.smithi097.stderr:2 2024-09-18T00:19:46.801 INFO:teuthology.orchestra.run.smithi097.stderr:abort 2024-09-18T00:19:46.801 INFO:teuthology.orchestra.run.smithi097.stderr:assert 2024-09-18T00:19:46.801 INFO:teuthology.orchestra.run.smithi097.stderr:bluefs debug_inject_read_zeros 2024-09-18T00:19:46.801 INFO:teuthology.orchestra.run.smithi097.stderr:bluefs files list 2024-09-18T00:19:46.801 INFO:teuthology.orchestra.run.smithi097.stderr:bluefs stats 2024-09-18T00:19:46.801 INFO:teuthology.orchestra.run.smithi097.stderr:bluestore allocator dump block 2024-09-18T00:19:46.801 INFO:teuthology.orchestra.run.smithi097.stderr:bluestore allocator fragmentation block 2024-09-18T00:19:46.802 INFO:teuthology.orchestra.run.smithi097.stderr:admin_socket: invalid command 2024-09-18T00:19:46.803 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T00:19:46.803 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-18T00:19:46.804 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:46.801+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:46.842 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:46.843 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:46.856 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:19:46.949 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:46.949+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:47.007 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:47.005+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:47.025 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:47.026 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:47.040 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:47.107 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:47.105+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:47.204 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:47.204 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:47.208 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:47.205+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:47.218 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:19:47.309 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:47.309+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:47.389 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:47.389 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:47.404 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:19:47.410 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:47.409+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:47.511 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:47.509+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:47.569 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:47.569 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:47.584 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:19:47.630 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:47.629+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:47.653 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:19:47.653+0000 7f8bea3c3640 -1 osd.1 105 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:19:47.713 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:47.709+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:47.722 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:47.722 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:47.737 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:19:47.814 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:47.813+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:47.910 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:47.910 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:47.915 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:47.912+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:47.924 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:19:48.016 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:48.013+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:48.017 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:19:48.086 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:48.086 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:48.101 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:19:48.145 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:48.144+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:48.158 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:48.158 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:48.158 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:19:48.176 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:48.176 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:48.192 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:48.192 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:48.213 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:48.213 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:48.218 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:48.217+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:48.225 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:48.226 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:48.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:48.226 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:19:48.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.226 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:19:48.226 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:48.226 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 95.424101148999995, 2024-09-18T00:19:48.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:19:48.226 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.226 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.226 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.226 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.226 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.226 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.227 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:19:48.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.227 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:48.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:19:48.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:19:48.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:19:48.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:19:48.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:19:48.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.228 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.228 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:19:48.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:19:48.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:19:48.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:19:48.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:19:48.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:19:48.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:19:48.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:19:48.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:48.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:19:48.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:19:48.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:48.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:19:48.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:19:48.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:48.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:19:48.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:19:48.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:48.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:19:48.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:19:48.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:19:48.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:19:48.232 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.232 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.232 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.232 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:19:48.232 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:48.232 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 95.407893275999996, 2024-09-18T00:19:48.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:19:48.233 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.233 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.233 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:48.233 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.233 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.233 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:48.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:19:48.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:19:48.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:48.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:19:48.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:19:48.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:19:48.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:19:48.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:19:48.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:19:48.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:19:48.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:19:48.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:19:48.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:19:48.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:19:48.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:19:48.236 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.237 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.237 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:48.237 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:48.237 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 82.380851956000001, 2024-09-18T00:19:48.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:19:48.237 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.237 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.237 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.237 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.237 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.237 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.238 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:19:48.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.238 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:48.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.238 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.238 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:19:48.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:19:48.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:19:48.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:19:48.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:19:48.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:19:48.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:19:48.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:19:48.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:19:48.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:19:48.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:19:48.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:19:48.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:19:48.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:48.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:19:48.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:19:48.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:48.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:19:48.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:19:48.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:48.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:19:48.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:19:48.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:48.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:19:48.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:19:48.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:19:48.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:19:48.242 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.243 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.243 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.243 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:19:48.243 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:48.243 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 82.359575403999997, 2024-09-18T00:19:48.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:19:48.243 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.243 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.243 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:48.243 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:48.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:19:48.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:19:48.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:19:48.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:19:48.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:19:48.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:19:48.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:19:48.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:19:48.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:19:48.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:19:48.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:19:48.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:19:48.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:19:48.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:19:48.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:19:48.247 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.247 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.247 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.247 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11460 3.19 3:9e578b6c:::rbd_data.112ab1fa102a.0000000000000153:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:19:48.247 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:48.247 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 72.957203952, 2024-09-18T00:19:48.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041989011999999999, 2024-09-18T00:19:48.247 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.247 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.247 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.247 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.247 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.247 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.248 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11460 2024-09-18T00:19:48.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.248 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:48.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:19:48.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267067+0000", 2024-09-18T00:19:48.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:19:48.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267088+0000", 2024-09-18T00:19:48.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0908999999999999e-05 2024-09-18T00:19:48.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267133+0000", 2024-09-18T00:19:48.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5232000000000003e-05 2024-09-18T00:19:48.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267157+0000", 2024-09-18T00:19:48.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3833e-05 2024-09-18T00:19:48.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267300+0000", 2024-09-18T00:19:48.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142282 2024-09-18T00:19:48.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267447+0000", 2024-09-18T00:19:48.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147851 2024-09-18T00:19:48.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:48.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308889+0000", 2024-09-18T00:19:48.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041441779999999998 2024-09-18T00:19:48.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:48.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308956+0000", 2024-09-18T00:19:48.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7136000000000002e-05 2024-09-18T00:19:48.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.309053+0000", 2024-09-18T00:19:48.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6552999999999995e-05 2024-09-18T00:19:48.252 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.252 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.252 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.252 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:16947 3.18 3:18b58c7b:::rbd_data.112a6efb7564.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:19:48.252 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:48.252 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.607426251, 2024-09-18T00:19:48.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.101692484, 2024-09-18T00:19:48.253 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.253 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.253 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.253 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.253 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.253 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.253 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 16947 2024-09-18T00:19:48.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.253 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:48.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:19:48.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616843+0000", 2024-09-18T00:19:48.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0780000000000001e-06 2024-09-18T00:19:48.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617785+0000", 2024-09-18T00:19:48.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094204100000000002 2024-09-18T00:19:48.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617801+0000", 2024-09-18T00:19:48.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6436999999999999e-05 2024-09-18T00:19:48.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617809+0000", 2024-09-18T00:19:48.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9049999999999997e-06 2024-09-18T00:19:48.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617904+0000", 2024-09-18T00:19:48.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5204999999999994e-05 2024-09-18T00:19:48.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.618071+0000", 2024-09-18T00:19:48.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016640000000000001 2024-09-18T00:19:48.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:48.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.624534+0000", 2024-09-18T00:19:48.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064634499999999999 2024-09-18T00:19:48.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:48.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718299+0000", 2024-09-18T00:19:48.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093764739 2024-09-18T00:19:48.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:48.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718375+0000", 2024-09-18T00:19:48.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6532999999999994e-05 2024-09-18T00:19:48.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:48.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718466+0000", 2024-09-18T00:19:48.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0581000000000006e-05 2024-09-18T00:19:48.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718534+0000", 2024-09-18T00:19:48.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8114999999999994e-05 2024-09-18T00:19:48.259 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.259 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.259 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.259 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:16947 3.18 e59/23 ondisk, result = 0)", 2024-09-18T00:19:48.259 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:48.259 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 59.582209556999999, 2024-09-18T00:19:48.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076488069000000006, 2024-09-18T00:19:48.259 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.259 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.259 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:48.259 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:48.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:19:48.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642060+0000", 2024-09-18T00:19:48.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6640000000000001e-06 2024-09-18T00:19:48.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642071+0000", 2024-09-18T00:19:48.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1204e-05 2024-09-18T00:19:48.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642091+0000", 2024-09-18T00:19:48.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0313999999999999e-05 2024-09-18T00:19:48.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642103+0000", 2024-09-18T00:19:48.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1214e-05 2024-09-18T00:19:48.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718350+0000", 2024-09-18T00:19:48.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076247681999999997 2024-09-18T00:19:48.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718366+0000", 2024-09-18T00:19:48.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5420000000000001e-05 2024-09-18T00:19:48.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718546+0000", 2024-09-18T00:19:48.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000180571 2024-09-18T00:19:48.263 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.263 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.263 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.263 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18686 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:48.263 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:48.263 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.275532187000003, 2024-09-18T00:19:48.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041132573999999998, 2024-09-18T00:19:48.263 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.263 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.264 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.264 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.264 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.264 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.264 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18686 2024-09-18T00:19:48.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.264 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:48.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:19:48.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948737+0000", 2024-09-18T00:19:48.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3769999999999999e-06 2024-09-18T00:19:48.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949968+0000", 2024-09-18T00:19:48.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001230493 2024-09-18T00:19:48.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949993+0000", 2024-09-18T00:19:48.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5282e-05 2024-09-18T00:19:48.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950011+0000", 2024-09-18T00:19:48.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8199999999999999e-05 2024-09-18T00:19:48.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950180+0000", 2024-09-18T00:19:48.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016867600000000001 2024-09-18T00:19:48.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950337+0000", 2024-09-18T00:19:48.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156902 2024-09-18T00:19:48.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:48.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950818+0000", 2024-09-18T00:19:48.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048192599999999998 2024-09-18T00:19:48.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:48.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978491+0000", 2024-09-18T00:19:48.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027672568000000002 2024-09-18T00:19:48.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:48.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989759+0000", 2024-09-18T00:19:48.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011268144000000001 2024-09-18T00:19:48.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:48.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989824+0000", 2024-09-18T00:19:48.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4556999999999999e-05 2024-09-18T00:19:48.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.269 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.269 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989868+0000", 2024-09-18T00:19:48.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4449000000000001e-05 2024-09-18T00:19:48.269 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.269 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.269 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.270 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18687 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:48.270 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:48.270 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.274204421999997, 2024-09-18T00:19:48.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040034156000000001, 2024-09-18T00:19:48.270 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.270 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.270 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.270 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.270 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.270 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.270 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18687 2024-09-18T00:19:48.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.271 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:48.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:19:48.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950065+0000", 2024-09-18T00:19:48.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5820000000000001e-06 2024-09-18T00:19:48.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950072+0000", 2024-09-18T00:19:48.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1029999999999996e-06 2024-09-18T00:19:48.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950092+0000", 2024-09-18T00:19:48.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0288e-05 2024-09-18T00:19:48.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950107+0000", 2024-09-18T00:19:48.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4290999999999999e-05 2024-09-18T00:19:48.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975820+0000", 2024-09-18T00:19:48.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025713573999999999 2024-09-18T00:19:48.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975980+0000", 2024-09-18T00:19:48.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016002799999999999 2024-09-18T00:19:48.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:48.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.976299+0000", 2024-09-18T00:19:48.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031853300000000002 2024-09-18T00:19:48.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:48.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989936+0000", 2024-09-18T00:19:48.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013637046999999999 2024-09-18T00:19:48.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:48.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990025+0000", 2024-09-18T00:19:48.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9040000000000001e-05 2024-09-18T00:19:48.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:48.275 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990064+0000", 2024-09-18T00:19:48.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9394000000000003e-05 2024-09-18T00:19:48.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.275 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990098+0000", 2024-09-18T00:19:48.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3275999999999998e-05 2024-09-18T00:19:48.276 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.276 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.276 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.276 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18688 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:48.276 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:48.276 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.274103070000002, 2024-09-18T00:19:48.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040041259000000003, 2024-09-18T00:19:48.276 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.276 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.276 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.277 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.277 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18688 2024-09-18T00:19:48.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.277 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:48.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:19:48.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950166+0000", 2024-09-18T00:19:48.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:19:48.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951295+0000", 2024-09-18T00:19:48.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011293480000000001 2024-09-18T00:19:48.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951328+0000", 2024-09-18T00:19:48.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2524999999999998e-05 2024-09-18T00:19:48.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951346+0000", 2024-09-18T00:19:48.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8170999999999999e-05 2024-09-18T00:19:48.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978546+0000", 2024-09-18T00:19:48.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027200418000000001 2024-09-18T00:19:48.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978666+0000", 2024-09-18T00:19:48.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119913 2024-09-18T00:19:48.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:48.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.979098+0000", 2024-09-18T00:19:48.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00043169399999999999 2024-09-18T00:19:48.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:48.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990000+0000", 2024-09-18T00:19:48.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010901719000000001 2024-09-18T00:19:48.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:48.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990109+0000", 2024-09-18T00:19:48.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010964299999999999 2024-09-18T00:19:48.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:48.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990152+0000", 2024-09-18T00:19:48.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2877000000000003e-05 2024-09-18T00:19:48.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990206+0000", 2024-09-18T00:19:48.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3717000000000001e-05 2024-09-18T00:19:48.282 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.282 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.282 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.282 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.282 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18689 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:48.282 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:48.282 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.272891371999997, 2024-09-18T00:19:48.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051022338, 2024-09-18T00:19:48.282 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.282 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.282 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.283 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.283 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18689 2024-09-18T00:19:48.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.283 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:48.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:19:48.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951378+0000", 2024-09-18T00:19:48.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:19:48.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951385+0000", 2024-09-18T00:19:48.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-18T00:19:48.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951402+0000", 2024-09-18T00:19:48.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6538999999999999e-05 2024-09-18T00:19:48.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951565+0000", 2024-09-18T00:19:48.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016307999999999999 2024-09-18T00:19:48.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988232+0000", 2024-09-18T00:19:48.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036667656999999999 2024-09-18T00:19:48.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988318+0000", 2024-09-18T00:19:48.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5348999999999999e-05 2024-09-18T00:19:48.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:48.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988574+0000", 2024-09-18T00:19:48.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025670800000000002 2024-09-18T00:19:48.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:48.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002254+0000", 2024-09-18T00:19:48.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013679274 2024-09-18T00:19:48.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:48.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002326+0000", 2024-09-18T00:19:48.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2720999999999997e-05 2024-09-18T00:19:48.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:48.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002364+0000", 2024-09-18T00:19:48.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7314000000000002e-05 2024-09-18T00:19:48.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002399+0000", 2024-09-18T00:19:48.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5162e-05 2024-09-18T00:19:48.287 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.287 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.287 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.288 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18690 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:48.288 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:48.288 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.272674354000003, 2024-09-18T00:19:48.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050872134999999999, 2024-09-18T00:19:48.288 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.288 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.288 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.288 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.288 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.288 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.288 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18690 2024-09-18T00:19:48.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.288 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:48.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:19:48.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951595+0000", 2024-09-18T00:19:48.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4479999999999999e-06 2024-09-18T00:19:48.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952167+0000", 2024-09-18T00:19:48.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00057188400000000004 2024-09-18T00:19:48.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952192+0000", 2024-09-18T00:19:48.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4936999999999999e-05 2024-09-18T00:19:48.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952206+0000", 2024-09-18T00:19:48.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4565e-05 2024-09-18T00:19:48.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990303+0000", 2024-09-18T00:19:48.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038096954000000002 2024-09-18T00:19:48.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990403+0000", 2024-09-18T00:19:48.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9486000000000003e-05 2024-09-18T00:19:48.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:48.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990773+0000", 2024-09-18T00:19:48.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037037799999999998 2024-09-18T00:19:48.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:48.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002316+0000", 2024-09-18T00:19:48.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011542581 2024-09-18T00:19:48.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:48.292 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002407+0000", 2024-09-18T00:19:48.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0968e-05 2024-09-18T00:19:48.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.292 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:48.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002442+0000", 2024-09-18T00:19:48.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4962000000000002e-05 2024-09-18T00:19:48.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002466+0000", 2024-09-18T00:19:48.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3972e-05 2024-09-18T00:19:48.293 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.293 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.293 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.293 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18691 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:48.293 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:48.293 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.272041477000002, 2024-09-18T00:19:48.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051495530999999997, 2024-09-18T00:19:48.294 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.294 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.294 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.294 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.294 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18691 2024-09-18T00:19:48.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.294 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:19:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952228+0000", 2024-09-18T00:19:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:19:48.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952234+0000", 2024-09-18T00:19:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0310000000000004e-06 2024-09-18T00:19:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952248+0000", 2024-09-18T00:19:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4577e-05 2024-09-18T00:19:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952260+0000", 2024-09-18T00:19:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1659e-05 2024-09-18T00:19:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.000989+0000", 2024-09-18T00:19:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048729445000000003 2024-09-18T00:19:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001076+0000", 2024-09-18T00:19:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6323999999999995e-05 2024-09-18T00:19:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:19:48.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001295+0000", 2024-09-18T00:19:48.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021944299999999999 2024-09-18T00:19:48.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:48.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003502+0000", 2024-09-18T00:19:48.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022065209999999999 2024-09-18T00:19:48.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:48.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003581+0000", 2024-09-18T00:19:48.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9404000000000001e-05 2024-09-18T00:19:48.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003652+0000", 2024-09-18T00:19:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0506999999999994e-05 2024-09-18T00:19:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003722+0000", 2024-09-18T00:19:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0300000000000001e-05 2024-09-18T00:19:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18692 3.7 3:e69124d0:::rbd_data.112a6efb7564.0000000000000001:f [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 55.271993416000001, 2024-09-18T00:19:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050418114999999999, 2024-09-18T00:19:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18692 2024-09-18T00:19:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.300 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:19:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952275+0000", 2024-09-18T00:19:48.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4e-07 2024-09-18T00:19:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952281+0000", 2024-09-18T00:19:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2000000000000002e-06 2024-09-18T00:19:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952295+0000", 2024-09-18T00:19:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4731e-05 2024-09-18T00:19:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952307+0000", 2024-09-18T00:19:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1877e-05 2024-09-18T00:19:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002506+0000", 2024-09-18T00:19:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050199230999999997 2024-09-18T00:19:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002574+0000", 2024-09-18T00:19:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8076000000000003e-05 2024-09-18T00:19:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002693+0000", 2024-09-18T00:19:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011806 2024-09-18T00:19:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19383 3.5 3:a1859bb0:::rbd_data.112a7fe29ab4.000000000000015b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:19:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 54.601981819000002, 2024-09-18T00:19:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056760192000000001, 2024-09-18T00:19:48.304 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19383 2024-09-18T00:19:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:19:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622287+0000", 2024-09-18T00:19:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-18T00:19:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622292+0000", 2024-09-18T00:19:48.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6110000000000004e-06 2024-09-18T00:19:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622304+0000", 2024-09-18T00:19:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2106000000000001e-05 2024-09-18T00:19:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622309+0000", 2024-09-18T00:19:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.818e-06 2024-09-18T00:19:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622360+0000", 2024-09-18T00:19:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0430999999999999e-05 2024-09-18T00:19:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622438+0000", 2024-09-18T00:19:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8362999999999995e-05 2024-09-18T00:19:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678811+0000", 2024-09-18T00:19:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056372894 2024-09-18T00:19:48.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678906+0000", 2024-09-18T00:19:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5078e-05 2024-09-18T00:19:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.679046+0000", 2024-09-18T00:19:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139987 2024-09-18T00:19:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22468 3.5 3:a250f1ef:::rbd_data.112a948d5956.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.452900413000002, 2024-09-18T00:19:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.055599787999999997, 2024-09-18T00:19:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.310 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22468 2024-09-18T00:19:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:19:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771370+0000", 2024-09-18T00:19:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9629999999999998e-06 2024-09-18T00:19:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771385+0000", 2024-09-18T00:19:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5053e-05 2024-09-18T00:19:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771429+0000", 2024-09-18T00:19:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3192999999999998e-05 2024-09-18T00:19:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771453+0000", 2024-09-18T00:19:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3995999999999999e-05 2024-09-18T00:19:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771623+0000", 2024-09-18T00:19:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000170846 2024-09-18T00:19:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771838+0000", 2024-09-18T00:19:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021416600000000001 2024-09-18T00:19:48.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826866+0000", 2024-09-18T00:19:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.05502866 2024-09-18T00:19:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826906+0000", 2024-09-18T00:19:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9805999999999998e-05 2024-09-18T00:19:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826967+0000", 2024-09-18T00:19:48.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1105000000000005e-05 2024-09-18T00:19:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22470 3.2 3:474595b6:::rbd_data.112a948d5956.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:19:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 49.396709428999998, 2024-09-18T00:19:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088321282000000001, 2024-09-18T00:19:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.315 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22470 2024-09-18T00:19:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:19:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827560+0000", 2024-09-18T00:19:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.762e-06 2024-09-18T00:19:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827569+0000", 2024-09-18T00:19:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8740000000000001e-06 2024-09-18T00:19:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827585+0000", 2024-09-18T00:19:48.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5962000000000001e-05 2024-09-18T00:19:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827595+0000", 2024-09-18T00:19:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5910000000000002e-06 2024-09-18T00:19:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827811+0000", 2024-09-18T00:19:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000216858 2024-09-18T00:19:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:19:48.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827818+0000", 2024-09-18T00:19:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.99e-06 2024-09-18T00:19:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829799+0000", 2024-09-18T00:19:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019803849999999999 2024-09-18T00:19:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829884+0000", 2024-09-18T00:19:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4852999999999995e-05 2024-09-18T00:19:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915616+0000", 2024-09-18T00:19:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085732808999999993 2024-09-18T00:19:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915711+0000", 2024-09-18T00:19:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4578000000000001e-05 2024-09-18T00:19:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915880+0000", 2024-09-18T00:19:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016861999999999999 2024-09-18T00:19:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.320 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:42788 3.7 3:e5a8432b:::rbd_data.112a3496573c.0000000000000027:17 [read 309248~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 27.505297629000001, 2024-09-18T00:19:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.039621847000000002, 2024-09-18T00:19:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:19:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 42788 2024-09-18T00:19:48.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718970+0000", 2024-09-18T00:19:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718971+0000", 2024-09-18T00:19:48.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2500000000000001e-07 2024-09-18T00:19:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718973+0000", 2024-09-18T00:19:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2510000000000001e-06 2024-09-18T00:19:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718979+0000", 2024-09-18T00:19:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1700000000000002e-06 2024-09-18T00:19:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.718984+0000", 2024-09-18T00:19:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9110000000000001e-06 2024-09-18T00:19:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.749126+0000", 2024-09-18T00:19:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.030142164999999999 2024-09-18T00:19:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.749199+0000", 2024-09-18T00:19:48.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2985999999999995e-05 2024-09-18T00:19:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.758592+0000", 2024-09-18T00:19:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0093929389999999995 2024-09-18T00:19:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:54153 3.b 3:d3f7b2f3:::rbd_data.112a859a01c9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,zero 4596947~21151] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:19:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:19:48.325 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 17.371496693000001, 2024-09-18T00:19:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062612886000000006, 2024-09-18T00:19:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 54153 2024-09-18T00:19:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:19:48.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:19:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852773+0000", 2024-09-18T00:19:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.426e-06 2024-09-18T00:19:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.327 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855337+0000", 2024-09-18T00:19:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025647830000000002 2024-09-18T00:19:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855369+0000", 2024-09-18T00:19:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1300999999999997e-05 2024-09-18T00:19:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855387+0000", 2024-09-18T00:19:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7949999999999999e-05 2024-09-18T00:19:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855487+0000", 2024-09-18T00:19:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010062 2024-09-18T00:19:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855686+0000", 2024-09-18T00:19:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019901399999999999 2024-09-18T00:19:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858037+0000", 2024-09-18T00:19:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002351264 2024-09-18T00:19:48.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.896302+0000", 2024-09-18T00:19:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038264864000000003 2024-09-18T00:19:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915250+0000", 2024-09-18T00:19:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018947538999999999 2024-09-18T00:19:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915318+0000", 2024-09-18T00:19:48.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7838999999999999e-05 2024-09-18T00:19:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915384+0000", 2024-09-18T00:19:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6285999999999995e-05 2024-09-18T00:19:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:55488 3.18 3:1d5573d8:::rbd_data.112a859a01c9.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T00:19:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:19:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 16.638666484000002, 2024-09-18T00:19:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045116205999999999, 2024-09-18T00:19:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:19:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:19:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:19:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:19:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 55488 2024-09-18T00:19:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:19:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:19:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:19:48.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:19:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:19:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:19:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:19:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585602+0000", 2024-09-18T00:19:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4000000000000001e-07 2024-09-18T00:19:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:19:48.333 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585607+0000", 2024-09-18T00:19:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5240000000000002e-06 2024-09-18T00:19:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:19:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585618+0000", 2024-09-18T00:19:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1493e-05 2024-09-18T00:19:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585624+0000", 2024-09-18T00:19:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7180000000000001e-06 2024-09-18T00:19:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:19:48.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585672+0000", 2024-09-18T00:19:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8158000000000002e-05 2024-09-18T00:19:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:19:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585780+0000", 2024-09-18T00:19:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010781399999999999 2024-09-18T00:19:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.588202+0000", 2024-09-18T00:19:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002422156 2024-09-18T00:19:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:48.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630375+0000", 2024-09-18T00:19:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042172820999999999 2024-09-18T00:19:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:19:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630426+0000", 2024-09-18T00:19:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1369000000000002e-05 2024-09-18T00:19:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:19:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630581+0000", 2024-09-18T00:19:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000155041 2024-09-18T00:19:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:19:48.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:19:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:19:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630718+0000", 2024-09-18T00:19:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000136372 2024-09-18T00:19:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:19:48.337 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:19:48.337 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:48.338 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:48.338 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:48.339 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:19:48.340 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:48.340 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:48.340 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:48.341 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:19:48.341 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:48.341 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:19:48.341 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:48.336+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:48.440 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:48.437+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:48.474 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:48.474 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:48.474 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:48.475 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:48.488 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:19:48.574 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:48.573+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:48.642 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:48.637+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:48.651 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:19:48.665 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:48.665 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:48.665 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:48.666 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:48.666 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:48.680 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:19:48.766 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:48.765+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:48.844 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:48.844 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:48.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:48.844 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:48.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.844 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2418 3.1a 3:59a8b40d:::rbd_data.112aa7ba4578.00000000000001b0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:19:48.844 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:48.844 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 88.106645254, 2024-09-18T00:19:48.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10488562999999999, 2024-09-18T00:19:48.845 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.845 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.845 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.845 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.845 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.845 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.845 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2418 2024-09-18T00:19:48.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.845 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:48.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:19:48.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735592+0000", 2024-09-18T00:19:48.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5229999999999998e-06 2024-09-18T00:19:48.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735604+0000", 2024-09-18T00:19:48.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2143e-05 2024-09-18T00:19:48.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735629+0000", 2024-09-18T00:19:48.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.497e-05 2024-09-18T00:19:48.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735648+0000", 2024-09-18T00:19:48.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8896999999999999e-05 2024-09-18T00:19:48.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735775+0000", 2024-09-18T00:19:48.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126935 2024-09-18T00:19:48.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.736118+0000", 2024-09-18T00:19:48.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034330800000000001 2024-09-18T00:19:48.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-18T00:19:48.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.741633+0000", 2024-09-18T00:19:48.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055144139999999996 2024-09-18T00:19:48.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:48.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840134+0000", 2024-09-18T00:19:48.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.098501618999999999 2024-09-18T00:19:48.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:48.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840184+0000", 2024-09-18T00:19:48.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9953000000000001e-05 2024-09-18T00:19:48.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:48.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840336+0000", 2024-09-18T00:19:48.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151476 2024-09-18T00:19:48.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840475+0000", 2024-09-18T00:19:48.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013939200000000001 2024-09-18T00:19:48.850 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.850 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.850 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.850 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:22272 3.3 3:c617f123:::rbd_data.112a948d5956.0000000000000009:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:19:48.851 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:48.851 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.851137055000002, 2024-09-18T00:19:48.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10752577300000001, 2024-09-18T00:19:48.851 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.851 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.851 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.851 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 22272 2024-09-18T00:19:48.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.851 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:48.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:19:48.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991102+0000", 2024-09-18T00:19:48.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9369999999999997e-06 2024-09-18T00:19:48.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991118+0000", 2024-09-18T00:19:48.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6396e-05 2024-09-18T00:19:48.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991150+0000", 2024-09-18T00:19:48.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2132999999999999e-05 2024-09-18T00:19:48.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991170+0000", 2024-09-18T00:19:48.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0085000000000002e-05 2024-09-18T00:19:48.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991370+0000", 2024-09-18T00:19:48.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019963700000000001 2024-09-18T00:19:48.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991518+0000", 2024-09-18T00:19:48.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000148424 2024-09-18T00:19:48.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:48.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.998850+0000", 2024-09-18T00:19:48.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073315480000000002 2024-09-18T00:19:48.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:48.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098178+0000", 2024-09-18T00:19:48.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.099328067000000006 2024-09-18T00:19:48.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:48.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098314+0000", 2024-09-18T00:19:48.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000135678 2024-09-18T00:19:48.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:48.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098498+0000", 2024-09-18T00:19:48.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018479600000000001 2024-09-18T00:19:48.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098623+0000", 2024-09-18T00:19:48.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000125072 2024-09-18T00:19:48.856 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.856 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.856 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.856 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:23689 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-18T00:19:48.856 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:48.856 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.989812276999999, 2024-09-18T00:19:48.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9562625809999998, 2024-09-18T00:19:48.856 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.857 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.857 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:48.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.857 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 23689 2024-09-18T00:19:48.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.857 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:48.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:19:48.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852423+0000", 2024-09-18T00:19:48.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9999999999999995e-07 2024-09-18T00:19:48.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852429+0000", 2024-09-18T00:19:48.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3279999999999996e-06 2024-09-18T00:19:48.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852442+0000", 2024-09-18T00:19:48.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3052e-05 2024-09-18T00:19:48.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852448+0000", 2024-09-18T00:19:48.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2779999999999999e-06 2024-09-18T00:19:48.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852523+0000", 2024-09-18T00:19:48.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4463999999999997e-05 2024-09-18T00:19:48.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:48.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852538+0000", 2024-09-18T00:19:48.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5466999999999999e-05 2024-09-18T00:19:48.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793478+0000", 2024-09-18T00:19:48.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.94094012699999996 2024-09-18T00:19:48.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:48.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793494+0000", 2024-09-18T00:19:48.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5807999999999998e-05 2024-09-18T00:19:48.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.804875+0000", 2024-09-18T00:19:48.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0113811939999999 2024-09-18T00:19:48.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805359+0000", 2024-09-18T00:19:48.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000484248 2024-09-18T00:19:48.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-18T00:19:48.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805872+0000", 2024-09-18T00:19:48.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00051218100000000001 2024-09-18T00:19:48.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:48.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.807790+0000", 2024-09-18T00:19:48.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001918763 2024-09-18T00:19:48.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:48.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808242+0000", 2024-09-18T00:19:48.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045126200000000002 2024-09-18T00:19:48.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:48.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808428+0000", 2024-09-18T00:19:48.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018633000000000001 2024-09-18T00:19:48.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808685+0000", 2024-09-18T00:19:48.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025707900000000001 2024-09-18T00:19:48.866 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.866 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.866 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.866 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47924 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7831552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.866 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032564+0000", 2024-09-18T00:19:48.866 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809670298, 2024-09-18T00:19:48.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086266190000000006, 2024-09-18T00:19:48.867 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.867 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.867 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.867 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.867 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.867 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.867 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47924 2024-09-18T00:19:48.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.867 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032564+0000", 2024-09-18T00:19:48.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032564+0000", 2024-09-18T00:19:48.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032565+0000", 2024-09-18T00:19:48.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9599999999999999e-07 2024-09-18T00:19:48.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032567+0000", 2024-09-18T00:19:48.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.142e-06 2024-09-18T00:19:48.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032573+0000", 2024-09-18T00:19:48.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2659999999999998e-06 2024-09-18T00:19:48.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032578+0000", 2024-09-18T00:19:48.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9019999999999998e-06 2024-09-18T00:19:48.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118632+0000", 2024-09-18T00:19:48.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086053942999999994 2024-09-18T00:19:48.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118706+0000", 2024-09-18T00:19:48.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4266999999999999e-05 2024-09-18T00:19:48.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118831+0000", 2024-09-18T00:19:48.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012417400000000001 2024-09-18T00:19:48.870 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.870 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.870 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.871 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47925 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7864320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.871 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:19:48.871 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809650176000002, 2024-09-18T00:19:48.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086484302999999998, 2024-09-18T00:19:48.871 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.871 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.871 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.871 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47925 2024-09-18T00:19:48.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.871 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:19:48.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:19:48.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:19:48.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.82e-07 2024-09-18T00:19:48.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032587+0000", 2024-09-18T00:19:48.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.058e-06 2024-09-18T00:19:48.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032593+0000", 2024-09-18T00:19:48.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1380000000000004e-06 2024-09-18T00:19:48.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032598+0000", 2024-09-18T00:19:48.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9819999999999999e-06 2024-09-18T00:19:48.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118873+0000", 2024-09-18T00:19:48.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086274609000000002 2024-09-18T00:19:48.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118950+0000", 2024-09-18T00:19:48.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6687999999999995e-05 2024-09-18T00:19:48.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119069+0000", 2024-09-18T00:19:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011934600000000001 2024-09-18T00:19:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47926 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7897088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032604+0000", 2024-09-18T00:19:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809630299999998, 2024-09-18T00:19:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086712168000000006, 2024-09-18T00:19:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.875 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47926 2024-09-18T00:19:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032604+0000", 2024-09-18T00:19:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032604+0000", 2024-09-18T00:19:48.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032605+0000", 2024-09-18T00:19:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:19:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032611+0000", 2024-09-18T00:19:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4860000000000002e-06 2024-09-18T00:19:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032618+0000", 2024-09-18T00:19:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6699999999999997e-06 2024-09-18T00:19:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032623+0000", 2024-09-18T00:19:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0860000000000001e-06 2024-09-18T00:19:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119111+0000", 2024-09-18T00:19:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086488067000000002 2024-09-18T00:19:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119185+0000", 2024-09-18T00:19:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3758999999999996e-05 2024-09-18T00:19:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119317+0000", 2024-09-18T00:19:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013159599999999999 2024-09-18T00:19:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47927 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7929856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.879 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:19:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809604959000001, 2024-09-18T00:19:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086929532000000004, 2024-09-18T00:19:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47927 2024-09-18T00:19:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:19:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:19:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:19:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-18T00:19:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032632+0000", 2024-09-18T00:19:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1320000000000001e-06 2024-09-18T00:19:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032639+0000", 2024-09-18T00:19:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1840000000000004e-06 2024-09-18T00:19:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032644+0000", 2024-09-18T00:19:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2519999999999999e-06 2024-09-18T00:19:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119358+0000", 2024-09-18T00:19:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086714075000000002 2024-09-18T00:19:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119438+0000", 2024-09-18T00:19:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9756e-05 2024-09-18T00:19:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119559+0000", 2024-09-18T00:19:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121579 2024-09-18T00:19:48.883 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47928 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7962624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032650+0000", 2024-09-18T00:19:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809584534999999, 2024-09-18T00:19:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087152732999999996, 2024-09-18T00:19:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.884 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47928 2024-09-18T00:19:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032650+0000", 2024-09-18T00:19:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032650+0000", 2024-09-18T00:19:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032651+0000", 2024-09-18T00:19:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:19:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032653+0000", 2024-09-18T00:19:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.182e-06 2024-09-18T00:19:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032659+0000", 2024-09-18T00:19:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2319999999999998e-06 2024-09-18T00:19:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032664+0000", 2024-09-18T00:19:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0420000000000002e-06 2024-09-18T00:19:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119602+0000", 2024-09-18T00:19:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086938213 2024-09-18T00:19:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119676+0000", 2024-09-18T00:19:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3708000000000006e-05 2024-09-18T00:19:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119803+0000", 2024-09-18T00:19:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126852 2024-09-18T00:19:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47929 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7995392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032670+0000", 2024-09-18T00:19:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809564277, 2024-09-18T00:19:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087373082000000005, 2024-09-18T00:19:48.888 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47929 2024-09-18T00:19:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032670+0000", 2024-09-18T00:19:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032670+0000", 2024-09-18T00:19:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032671+0000", 2024-09-18T00:19:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:19:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032673+0000", 2024-09-18T00:19:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.928e-06 2024-09-18T00:19:48.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032679+0000", 2024-09-18T00:19:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.02e-06 2024-09-18T00:19:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032684+0000", 2024-09-18T00:19:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1660000000000002e-06 2024-09-18T00:19:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119844+0000", 2024-09-18T00:19:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087160248999999995 2024-09-18T00:19:48.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119920+0000", 2024-09-18T00:19:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5181999999999999e-05 2024-09-18T00:19:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120044+0000", 2024-09-18T00:19:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012403300000000001 2024-09-18T00:19:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.892 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47930 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8028160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032690+0000", 2024-09-18T00:19:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809544424999999, 2024-09-18T00:19:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087595941999999996, 2024-09-18T00:19:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47930 2024-09-18T00:19:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032690+0000", 2024-09-18T00:19:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032690+0000", 2024-09-18T00:19:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032691+0000", 2024-09-18T00:19:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9399999999999995e-07 2024-09-18T00:19:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032693+0000", 2024-09-18T00:19:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1299999999999999e-06 2024-09-18T00:19:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032699+0000", 2024-09-18T00:19:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1879999999999999e-06 2024-09-18T00:19:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032704+0000", 2024-09-18T00:19:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0939999999999998e-06 2024-09-18T00:19:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120087+0000", 2024-09-18T00:19:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087383049000000004 2024-09-18T00:19:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120164+0000", 2024-09-18T00:19:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6679000000000002e-05 2024-09-18T00:19:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120286+0000", 2024-09-18T00:19:48.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012230800000000001 2024-09-18T00:19:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47931 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8060928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:19:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809524182000001, 2024-09-18T00:19:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087823688999999996, 2024-09-18T00:19:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.897 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47931 2024-09-18T00:19:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:19:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:19:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:19:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9200000000000001e-07 2024-09-18T00:19:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032713+0000", 2024-09-18T00:19:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1239999999999999e-06 2024-09-18T00:19:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032719+0000", 2024-09-18T00:19:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1659999999999999e-06 2024-09-18T00:19:48.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032724+0000", 2024-09-18T00:19:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9359999999999998e-06 2024-09-18T00:19:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120330+0000", 2024-09-18T00:19:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087605559 2024-09-18T00:19:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120403+0000", 2024-09-18T00:19:48.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-05 2024-09-18T00:19:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120534+0000", 2024-09-18T00:19:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013121199999999999 2024-09-18T00:19:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47932 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8093696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032730+0000", 2024-09-18T00:19:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809504283999999, 2024-09-18T00:19:48.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088052486999999999, 2024-09-18T00:19:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47932 2024-09-18T00:19:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032730+0000", 2024-09-18T00:19:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032730+0000", 2024-09-18T00:19:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032731+0000", 2024-09-18T00:19:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2200000000000004e-07 2024-09-18T00:19:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032743+0000", 2024-09-18T00:19:48.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1796e-05 2024-09-18T00:19:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032749+0000", 2024-09-18T00:19:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6959999999999999e-06 2024-09-18T00:19:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032755+0000", 2024-09-18T00:19:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.22e-06 2024-09-18T00:19:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120578+0000", 2024-09-18T00:19:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087823017000000003 2024-09-18T00:19:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120654+0000", 2024-09-18T00:19:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6476000000000004e-05 2024-09-18T00:19:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120783+0000", 2024-09-18T00:19:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012876000000000001 2024-09-18T00:19:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.905 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47933 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8126464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032761+0000", 2024-09-18T00:19:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809473266000001, 2024-09-18T00:19:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088263233999999996, 2024-09-18T00:19:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47933 2024-09-18T00:19:48.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032761+0000", 2024-09-18T00:19:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032761+0000", 2024-09-18T00:19:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032762+0000", 2024-09-18T00:19:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:19:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032764+0000", 2024-09-18T00:19:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1519999999999999e-06 2024-09-18T00:19:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032770+0000", 2024-09-18T00:19:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1700000000000002e-06 2024-09-18T00:19:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032778+0000", 2024-09-18T00:19:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2400000000000001e-06 2024-09-18T00:19:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120825+0000", 2024-09-18T00:19:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088047046000000004 2024-09-18T00:19:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120898+0000", 2024-09-18T00:19:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3011999999999998e-05 2024-09-18T00:19:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121025+0000", 2024-09-18T00:19:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012710999999999999 2024-09-18T00:19:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47934 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8159232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032784+0000", 2024-09-18T00:19:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809450715000001, 2024-09-18T00:19:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088479458999999996, 2024-09-18T00:19:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47934 2024-09-18T00:19:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032784+0000", 2024-09-18T00:19:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032784+0000", 2024-09-18T00:19:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032785+0000", 2024-09-18T00:19:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9599999999999999e-07 2024-09-18T00:19:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032787+0000", 2024-09-18T00:19:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.052e-06 2024-09-18T00:19:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032793+0000", 2024-09-18T00:19:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2439999999999998e-06 2024-09-18T00:19:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032798+0000", 2024-09-18T00:19:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9080000000000003e-06 2024-09-18T00:19:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121066+0000", 2024-09-18T00:19:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088267795999999996 2024-09-18T00:19:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121139+0000", 2024-09-18T00:19:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2910999999999994e-05 2024-09-18T00:19:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121263+0000", 2024-09-18T00:19:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000124852 2024-09-18T00:19:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47935 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8192000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.914 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032841+0000", 2024-09-18T00:19:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809393634999999, 2024-09-18T00:19:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088666069, 2024-09-18T00:19:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47935 2024-09-18T00:19:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032841+0000", 2024-09-18T00:19:48.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032841+0000", 2024-09-18T00:19:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032842+0000", 2024-09-18T00:19:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6400000000000002e-07 2024-09-18T00:19:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032845+0000", 2024-09-18T00:19:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8959999999999999e-06 2024-09-18T00:19:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032852+0000", 2024-09-18T00:19:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1640000000000004e-06 2024-09-18T00:19:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032857+0000", 2024-09-18T00:19:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3340000000000001e-06 2024-09-18T00:19:48.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121304+0000", 2024-09-18T00:19:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088447001999999997 2024-09-18T00:19:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121376+0000", 2024-09-18T00:19:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2121999999999998e-05 2024-09-18T00:19:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121507+0000", 2024-09-18T00:19:48.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130987 2024-09-18T00:19:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47936 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:19:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809370913999999, 2024-09-18T00:19:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088878349999999995, 2024-09-18T00:19:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.919 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47936 2024-09-18T00:19:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:19:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:19:48.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:19:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8999999999999997e-07 2024-09-18T00:19:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032866+0000", 2024-09-18T00:19:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.142e-06 2024-09-18T00:19:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032873+0000", 2024-09-18T00:19:48.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:19:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032878+0000", 2024-09-18T00:19:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1220000000000003e-06 2024-09-18T00:19:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121548+0000", 2024-09-18T00:19:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088670530999999997 2024-09-18T00:19:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121621+0000", 2024-09-18T00:19:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2941999999999994e-05 2024-09-18T00:19:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121742+0000", 2024-09-18T00:19:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120903 2024-09-18T00:19:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47937 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8257536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.923 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:19:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809350806000001, 2024-09-18T00:19:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089098572000000001, 2024-09-18T00:19:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47937 2024-09-18T00:19:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:19:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:19:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:19:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9200000000000001e-07 2024-09-18T00:19:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032886+0000", 2024-09-18T00:19:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9999999999999999e-06 2024-09-18T00:19:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032893+0000", 2024-09-18T00:19:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2020000000000001e-06 2024-09-18T00:19:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032898+0000", 2024-09-18T00:19:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9860000000000002e-06 2024-09-18T00:19:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121785+0000", 2024-09-18T00:19:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088887318000000007 2024-09-18T00:19:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121858+0000", 2024-09-18T00:19:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2900999999999999e-05 2024-09-18T00:19:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121983+0000", 2024-09-18T00:19:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000124673 2024-09-18T00:19:48.927 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47938 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8290304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:19:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809331021999999, 2024-09-18T00:19:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089335780000000004, 2024-09-18T00:19:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.928 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47938 2024-09-18T00:19:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:19:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:19:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:19:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2799999999999996e-07 2024-09-18T00:19:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032923+0000", 2024-09-18T00:19:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8218000000000001e-05 2024-09-18T00:19:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032930+0000", 2024-09-18T00:19:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5440000000000001e-06 2024-09-18T00:19:48.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032936+0000", 2024-09-18T00:19:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4720000000000003e-06 2024-09-18T00:19:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122023+0000", 2024-09-18T00:19:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089087728000000005 2024-09-18T00:19:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122117+0000", 2024-09-18T00:19:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3295999999999996e-05 2024-09-18T00:19:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122240+0000", 2024-09-18T00:19:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012289399999999999 2024-09-18T00:19:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.932 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47939 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8323072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032943+0000", 2024-09-18T00:19:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809291469000001, 2024-09-18T00:19:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089547929999999998, 2024-09-18T00:19:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.933 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47939 2024-09-18T00:19:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032943+0000", 2024-09-18T00:19:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032943+0000", 2024-09-18T00:19:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032944+0000", 2024-09-18T00:19:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6199999999999998e-07 2024-09-18T00:19:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032946+0000", 2024-09-18T00:19:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2840000000000001e-06 2024-09-18T00:19:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032952+0000", 2024-09-18T00:19:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2700000000000001e-06 2024-09-18T00:19:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032957+0000", 2024-09-18T00:19:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0200000000000002e-06 2024-09-18T00:19:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122280+0000", 2024-09-18T00:19:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089322330000000005 2024-09-18T00:19:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122354+0000", 2024-09-18T00:19:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3794000000000005e-05 2024-09-18T00:19:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122491+0000", 2024-09-18T00:19:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013767 2024-09-18T00:19:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.938 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47940 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8355840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:19:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.809271040999999, 2024-09-18T00:19:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089758663000000002, 2024-09-18T00:19:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:48.939 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47940 2024-09-18T00:19:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:19:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:19:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:19:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8599999999999998e-07 2024-09-18T00:19:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032966+0000", 2024-09-18T00:19:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1160000000000002e-06 2024-09-18T00:19:48.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032972+0000", 2024-09-18T00:19:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1639999999999998e-06 2024-09-18T00:19:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:48.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032978+0000", 2024-09-18T00:19:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1320000000000002e-06 2024-09-18T00:19:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122520+0000", 2024-09-18T00:19:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089541930000000006 2024-09-18T00:19:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122601+0000", 2024-09-18T00:19:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1207999999999999e-05 2024-09-18T00:19:48.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122722+0000", 2024-09-18T00:19:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012162700000000001 2024-09-18T00:19:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:48.944 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:48.944 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:48.945 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:48.841+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:48.946 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:19:48.947 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:48.944+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:49.051 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:49.049+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:49.123 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:49.124 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:49.124 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:49.124 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:49.138 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:19:49.184 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:49.181+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:49.250 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:49.248+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:49.309 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:49.309 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:49.309 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:49.309 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:49.309 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:49.323 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:19:49.350 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:49.348+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:49.452 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:49.448+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:49.452 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:49.452 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:49.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:49.452 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:49.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.452 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2229 3.10 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:49.452 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:49.453 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 89.185227732000001, 2024-09-18T00:19:49.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048730086999999998, 2024-09-18T00:19:49.453 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.453 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.453 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:49.453 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.453 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.453 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.453 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2229 2024-09-18T00:19:49.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.453 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:49.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:49.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263362+0000", 2024-09-18T00:19:49.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3869999999999999e-06 2024-09-18T00:19:49.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263381+0000", 2024-09-18T00:19:49.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9153000000000001e-05 2024-09-18T00:19:49.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263418+0000", 2024-09-18T00:19:49.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6346000000000001e-05 2024-09-18T00:19:49.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263435+0000", 2024-09-18T00:19:49.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7309999999999999e-05 2024-09-18T00:19:49.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263589+0000", 2024-09-18T00:19:49.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015451199999999999 2024-09-18T00:19:49.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263902+0000", 2024-09-18T00:19:49.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031291799999999997 2024-09-18T00:19:49.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:49.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.265995+0000", 2024-09-18T00:19:49.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020931809999999999 2024-09-18T00:19:49.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:49.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311788+0000", 2024-09-18T00:19:49.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045792078 2024-09-18T00:19:49.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:49.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311903+0000", 2024-09-18T00:19:49.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011510699999999999 2024-09-18T00:19:49.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:49.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312009+0000", 2024-09-18T00:19:49.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010630599999999999 2024-09-18T00:19:49.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312089+0000", 2024-09-18T00:19:49.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9789000000000005e-05 2024-09-18T00:19:49.458 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.458 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.459 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.459 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2229 3.10 e36/23 ondisk, result = 0)", 2024-09-18T00:19:49.459 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:49.459 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 89.166082072999998, 2024-09-18T00:19:49.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029600955000000002, 2024-09-18T00:19:49.459 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.459 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.459 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:49.460 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:49.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:49.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282507+0000", 2024-09-18T00:19:49.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9160000000000001e-06 2024-09-18T00:19:49.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282522+0000", 2024-09-18T00:19:49.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5e-05 2024-09-18T00:19:49.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282558+0000", 2024-09-18T00:19:49.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5694000000000001e-05 2024-09-18T00:19:49.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282580+0000", 2024-09-18T00:19:49.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2323e-05 2024-09-18T00:19:49.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311873+0000", 2024-09-18T00:19:49.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029293013999999999 2024-09-18T00:19:49.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311892+0000", 2024-09-18T00:19:49.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8556e-05 2024-09-18T00:19:49.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312105+0000", 2024-09-18T00:19:49.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021345199999999999 2024-09-18T00:19:49.463 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.463 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.463 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.463 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5714 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:49.463 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:49.464 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.138379569999998, 2024-09-18T00:19:49.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027448727999999999, 2024-09-18T00:19:49.464 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.464 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.464 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:49.464 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.464 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.464 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.464 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5714 2024-09-18T00:19:49.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.464 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:49.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:49.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310211+0000", 2024-09-18T00:19:49.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5639999999999998e-06 2024-09-18T00:19:49.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312678+0000", 2024-09-18T00:19:49.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002466761 2024-09-18T00:19:49.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312746+0000", 2024-09-18T00:19:49.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8046000000000005e-05 2024-09-18T00:19:49.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312774+0000", 2024-09-18T00:19:49.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7813e-05 2024-09-18T00:19:49.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312927+0000", 2024-09-18T00:19:49.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015328499999999999 2024-09-18T00:19:49.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313214+0000", 2024-09-18T00:19:49.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028683099999999998 2024-09-18T00:19:49.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:49.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313708+0000", 2024-09-18T00:19:49.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049379100000000004 2024-09-18T00:19:49.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:49.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336895+0000", 2024-09-18T00:19:49.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023186801 2024-09-18T00:19:49.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:49.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337454+0000", 2024-09-18T00:19:49.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055979399999999996 2024-09-18T00:19:49.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:49.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337571+0000", 2024-09-18T00:19:49.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116821 2024-09-18T00:19:49.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337656+0000", 2024-09-18T00:19:49.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4221000000000006e-05 2024-09-18T00:19:49.469 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.469 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.469 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.470 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6801 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:49.470 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:49.470 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 83.411543121999998, 2024-09-18T00:19:49.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051953602000000002, 2024-09-18T00:19:49.470 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.470 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.470 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:49.470 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.470 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.470 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.470 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6801 2024-09-18T00:19:49.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.471 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:49.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:49.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037046+0000", 2024-09-18T00:19:49.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.19e-06 2024-09-18T00:19:49.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037062+0000", 2024-09-18T00:19:49.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586999999999998e-05 2024-09-18T00:19:49.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037096+0000", 2024-09-18T00:19:49.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4112000000000001e-05 2024-09-18T00:19:49.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037120+0000", 2024-09-18T00:19:49.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.355e-05 2024-09-18T00:19:49.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037246+0000", 2024-09-18T00:19:49.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012652200000000001 2024-09-18T00:19:49.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037385+0000", 2024-09-18T00:19:49.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013924599999999999 2024-09-18T00:19:49.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:49.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.040881+0000", 2024-09-18T00:19:49.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034954830000000002 2024-09-18T00:19:49.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:49.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088794+0000", 2024-09-18T00:19:49.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047912897000000003 2024-09-18T00:19:49.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:49.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088828+0000", 2024-09-18T00:19:49.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3880000000000001e-05 2024-09-18T00:19:49.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:49.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088916+0000", 2024-09-18T00:19:49.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.865e-05 2024-09-18T00:19:49.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088997+0000", 2024-09-18T00:19:49.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0484999999999999e-05 2024-09-18T00:19:49.476 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.476 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.476 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.476 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:6801 3.10 e42/23 ondisk, result = 0)", 2024-09-18T00:19:49.476 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:49.476 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 83.397735711999999, 2024-09-18T00:19:49.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037960394000000001, 2024-09-18T00:19:49.476 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.476 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.476 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:49.477 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:49.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:49.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050852+0000", 2024-09-18T00:19:49.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:19:49.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050860+0000", 2024-09-18T00:19:49.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7729999999999999e-06 2024-09-18T00:19:49.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050891+0000", 2024-09-18T00:19:49.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0617000000000002e-05 2024-09-18T00:19:49.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050910+0000", 2024-09-18T00:19:49.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.944e-05 2024-09-18T00:19:49.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088766+0000", 2024-09-18T00:19:49.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037855633999999999 2024-09-18T00:19:49.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088784+0000", 2024-09-18T00:19:49.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7767000000000001e-05 2024-09-18T00:19:49.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088811+0000", 2024-09-18T00:19:49.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7382999999999999e-05 2024-09-18T00:19:49.480 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.481 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.481 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.481 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12594 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:49.481 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:49.481 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.056959173999999, 2024-09-18T00:19:49.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026492600000000002, 2024-09-18T00:19:49.481 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.481 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.481 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:49.481 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.481 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.482 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.482 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12594 2024-09-18T00:19:49.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.482 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:49.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:49.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391629+0000", 2024-09-18T00:19:49.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.86e-06 2024-09-18T00:19:49.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391637+0000", 2024-09-18T00:19:49.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3999999999999992e-06 2024-09-18T00:19:49.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391664+0000", 2024-09-18T00:19:49.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6536000000000001e-05 2024-09-18T00:19:49.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391681+0000", 2024-09-18T00:19:49.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6563e-05 2024-09-18T00:19:49.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417791+0000", 2024-09-18T00:19:49.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026110936000000001 2024-09-18T00:19:49.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417933+0000", 2024-09-18T00:19:49.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014112099999999999 2024-09-18T00:19:49.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418120+0000", 2024-09-18T00:19:49.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018718399999999999 2024-09-18T00:19:49.486 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.486 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.486 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.486 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12595 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:49.486 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:49.486 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.056882975999997, 2024-09-18T00:19:49.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026819428999999999, 2024-09-18T00:19:49.486 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.486 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.487 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:49.487 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.487 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.487 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.487 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12595 2024-09-18T00:19:49.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.487 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:49.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:49.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391705+0000", 2024-09-18T00:19:49.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3999999999999999e-06 2024-09-18T00:19:49.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391711+0000", 2024-09-18T00:19:49.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7459999999999997e-06 2024-09-18T00:19:49.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391733+0000", 2024-09-18T00:19:49.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2923000000000001e-05 2024-09-18T00:19:49.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391748+0000", 2024-09-18T00:19:49.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4790000000000001e-05 2024-09-18T00:19:49.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418180+0000", 2024-09-18T00:19:49.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026431888000000001 2024-09-18T00:19:49.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418328+0000", 2024-09-18T00:19:49.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000147725 2024-09-18T00:19:49.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418523+0000", 2024-09-18T00:19:49.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000194957 2024-09-18T00:19:49.491 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.491 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.491 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.491 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12596 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:49.491 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:49.491 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.056820008000003, 2024-09-18T00:19:49.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027220296000000001, 2024-09-18T00:19:49.491 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.491 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.491 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:49.491 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.491 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.492 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.492 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12596 2024-09-18T00:19:49.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.492 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:49.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:49.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391768+0000", 2024-09-18T00:19:49.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.283e-06 2024-09-18T00:19:49.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391773+0000", 2024-09-18T00:19:49.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3970000000000002e-06 2024-09-18T00:19:49.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391795+0000", 2024-09-18T00:19:49.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1835e-05 2024-09-18T00:19:49.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391809+0000", 2024-09-18T00:19:49.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4107000000000001e-05 2024-09-18T00:19:49.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418582+0000", 2024-09-18T00:19:49.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026772918999999999 2024-09-18T00:19:49.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418780+0000", 2024-09-18T00:19:49.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000198042 2024-09-18T00:19:49.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418987+0000", 2024-09-18T00:19:49.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000206713 2024-09-18T00:19:49.495 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.495 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.495 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.495 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12597 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:49.495 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:49.495 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.056759490000005, 2024-09-18T00:19:49.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027612714, 2024-09-18T00:19:49.495 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.496 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.496 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:49.496 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.496 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.496 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.496 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12597 2024-09-18T00:19:49.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.496 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:49.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:49.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391828+0000", 2024-09-18T00:19:49.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.203e-06 2024-09-18T00:19:49.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391834+0000", 2024-09-18T00:19:49.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6529999999999999e-06 2024-09-18T00:19:49.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391856+0000", 2024-09-18T00:19:49.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2195999999999999e-05 2024-09-18T00:19:49.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391870+0000", 2024-09-18T00:19:49.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4063000000000001e-05 2024-09-18T00:19:49.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419049+0000", 2024-09-18T00:19:49.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027178751000000001 2024-09-18T00:19:49.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419239+0000", 2024-09-18T00:19:49.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019039400000000001 2024-09-18T00:19:49.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419440+0000", 2024-09-18T00:19:49.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000200454 2024-09-18T00:19:49.499 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.499 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.499 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.499 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12598 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:49.500 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:49.500 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.056696439000007, 2024-09-18T00:19:49.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028269056000000001, 2024-09-18T00:19:49.500 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.500 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.500 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:49.500 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.500 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.500 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.500 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12598 2024-09-18T00:19:49.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.500 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:49.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:49.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391891+0000", 2024-09-18T00:19:49.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.257e-06 2024-09-18T00:19:49.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391897+0000", 2024-09-18T00:19:49.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9560000000000002e-06 2024-09-18T00:19:49.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391920+0000", 2024-09-18T00:19:49.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3025999999999999e-05 2024-09-18T00:19:49.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391935+0000", 2024-09-18T00:19:49.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4853e-05 2024-09-18T00:19:49.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419527+0000", 2024-09-18T00:19:49.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027592378000000001 2024-09-18T00:19:49.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419681+0000", 2024-09-18T00:19:49.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015316 2024-09-18T00:19:49.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420159+0000", 2024-09-18T00:19:49.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000478426 2024-09-18T00:19:49.504 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.504 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.504 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.504 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12599 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:49.504 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:49.504 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.056632583999999, 2024-09-18T00:19:49.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028649857000000001, 2024-09-18T00:19:49.505 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.505 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.505 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:49.505 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.505 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.505 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.505 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12599 2024-09-18T00:19:49.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.505 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:49.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:49.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391955+0000", 2024-09-18T00:19:49.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-06 2024-09-18T00:19:49.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391961+0000", 2024-09-18T00:19:49.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7100000000000004e-06 2024-09-18T00:19:49.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391982+0000", 2024-09-18T00:19:49.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1492e-05 2024-09-18T00:19:49.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391996+0000", 2024-09-18T00:19:49.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4194e-05 2024-09-18T00:19:49.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420237+0000", 2024-09-18T00:19:49.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028240818000000001 2024-09-18T00:19:49.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420430+0000", 2024-09-18T00:19:49.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019228299999999999 2024-09-18T00:19:49.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420604+0000", 2024-09-18T00:19:49.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000174117 2024-09-18T00:19:49.509 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.509 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.509 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.510 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12600 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 270336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:49.510 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:49.510 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.056571590000004, 2024-09-18T00:19:49.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028964586, 2024-09-18T00:19:49.510 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.510 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.510 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:49.510 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.510 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.510 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.510 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12600 2024-09-18T00:19:49.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.511 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:49.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:49.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392016+0000", 2024-09-18T00:19:49.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.31e-06 2024-09-18T00:19:49.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392022+0000", 2024-09-18T00:19:49.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5600000000000001e-06 2024-09-18T00:19:49.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392045+0000", 2024-09-18T00:19:49.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3595999999999999e-05 2024-09-18T00:19:49.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392060+0000", 2024-09-18T00:19:49.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4727e-05 2024-09-18T00:19:49.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420669+0000", 2024-09-18T00:19:49.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028608855999999998 2024-09-18T00:19:49.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420813+0000", 2024-09-18T00:19:49.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014429500000000001 2024-09-18T00:19:49.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420979+0000", 2024-09-18T00:19:49.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016624200000000001 2024-09-18T00:19:49.514 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.514 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.514 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.514 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12601 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:49.514 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:49.515 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.056507685, 2024-09-18T00:19:49.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029292832000000001, 2024-09-18T00:19:49.515 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.515 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.515 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:49.515 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.515 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.515 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.515 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12601 2024-09-18T00:19:49.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.515 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:49.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:49.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392080+0000", 2024-09-18T00:19:49.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:49.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392086+0000", 2024-09-18T00:19:49.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7570000000000001e-06 2024-09-18T00:19:49.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392105+0000", 2024-09-18T00:19:49.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9415999999999998e-05 2024-09-18T00:19:49.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392132+0000", 2024-09-18T00:19:49.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7019e-05 2024-09-18T00:19:49.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421049+0000", 2024-09-18T00:19:49.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028916659000000001 2024-09-18T00:19:49.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421204+0000", 2024-09-18T00:19:49.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155224 2024-09-18T00:19:49.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421372+0000", 2024-09-18T00:19:49.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016754699999999999 2024-09-18T00:19:49.519 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.519 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.519 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.519 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12602 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:49.519 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:49.519 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.056435054000005, 2024-09-18T00:19:49.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029612833000000002, 2024-09-18T00:19:49.520 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.520 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.520 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:49.520 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.520 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.520 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.520 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12602 2024-09-18T00:19:49.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.520 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:49.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:49.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392152+0000", 2024-09-18T00:19:49.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0470000000000001e-06 2024-09-18T00:19:49.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392158+0000", 2024-09-18T00:19:49.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:19:49.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392181+0000", 2024-09-18T00:19:49.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2629e-05 2024-09-18T00:19:49.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392222+0000", 2024-09-18T00:19:49.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0772000000000002e-05 2024-09-18T00:19:49.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421441+0000", 2024-09-18T00:19:49.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029219327 2024-09-18T00:19:49.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421608+0000", 2024-09-18T00:19:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000166813 2024-09-18T00:19:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421764+0000", 2024-09-18T00:19:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015628500000000001 2024-09-18T00:19:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.524 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12603 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.056344659999993, 2024-09-18T00:19:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029902524, 2024-09-18T00:19:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12603 2024-09-18T00:19:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:49.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392243+0000", 2024-09-18T00:19:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392260+0000", 2024-09-18T00:19:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6860000000000001e-05 2024-09-18T00:19:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392282+0000", 2024-09-18T00:19:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2713000000000001e-05 2024-09-18T00:19:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392297+0000", 2024-09-18T00:19:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4375e-05 2024-09-18T00:19:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421830+0000", 2024-09-18T00:19:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029532668000000002 2024-09-18T00:19:49.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421985+0000", 2024-09-18T00:19:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155613 2024-09-18T00:19:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422144+0000", 2024-09-18T00:19:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000159102 2024-09-18T00:19:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12604 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.056269608999997, 2024-09-18T00:19:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030228695, 2024-09-18T00:19:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12604 2024-09-18T00:19:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.530 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392318+0000", 2024-09-18T00:19:49.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392337+0000", 2024-09-18T00:19:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:19:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392364+0000", 2024-09-18T00:19:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7288999999999999e-05 2024-09-18T00:19:49.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392379+0000", 2024-09-18T00:19:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:19:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422209+0000", 2024-09-18T00:19:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029829050999999999 2024-09-18T00:19:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422388+0000", 2024-09-18T00:19:49.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017988699999999999 2024-09-18T00:19:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422545+0000", 2024-09-18T00:19:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015701199999999999 2024-09-18T00:19:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12605 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.056185048000003, 2024-09-18T00:19:49.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030488674, 2024-09-18T00:19:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12605 2024-09-18T00:19:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392403+0000", 2024-09-18T00:19:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.883e-06 2024-09-18T00:19:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392421+0000", 2024-09-18T00:19:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7526000000000001e-05 2024-09-18T00:19:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392450+0000", 2024-09-18T00:19:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8972e-05 2024-09-18T00:19:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392488+0000", 2024-09-18T00:19:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8522000000000001e-05 2024-09-18T00:19:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422614+0000", 2024-09-18T00:19:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030125945000000001 2024-09-18T00:19:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422752+0000", 2024-09-18T00:19:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000138178 2024-09-18T00:19:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422890+0000", 2024-09-18T00:19:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013764800000000001 2024-09-18T00:19:49.538 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12606 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.056076387999994, 2024-09-18T00:19:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033727073000000003, 2024-09-18T00:19:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12606 2024-09-18T00:19:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392511+0000", 2024-09-18T00:19:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3090000000000001e-06 2024-09-18T00:19:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392517+0000", 2024-09-18T00:19:49.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.993e-06 2024-09-18T00:19:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392543+0000", 2024-09-18T00:19:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6152999999999998e-05 2024-09-18T00:19:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392558+0000", 2024-09-18T00:19:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4610000000000001e-05 2024-09-18T00:19:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.425959+0000", 2024-09-18T00:19:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033400484000000001 2024-09-18T00:19:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426106+0000", 2024-09-18T00:19:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014779800000000001 2024-09-18T00:19:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426237+0000", 2024-09-18T00:19:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013072600000000001 2024-09-18T00:19:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17562 3.a 3:5578626c:::rbd_data.112a6efb7564.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T00:19:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 59.490611227000002, 2024-09-18T00:19:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052733453999999999, 2024-09-18T00:19:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:49.544 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17562 2024-09-18T00:19:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957977+0000", 2024-09-18T00:19:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.119e-06 2024-09-18T00:19:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957988+0000", 2024-09-18T00:19:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0557000000000001e-05 2024-09-18T00:19:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958017+0000", 2024-09-18T00:19:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9156e-05 2024-09-18T00:19:49.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958036+0000", 2024-09-18T00:19:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8919e-05 2024-09-18T00:19:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958144+0000", 2024-09-18T00:19:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107905 2024-09-18T00:19:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958284+0000", 2024-09-18T00:19:49.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140216 2024-09-18T00:19:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.960572+0000", 2024-09-18T00:19:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022881860000000002 2024-09-18T00:19:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:49.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010537+0000", 2024-09-18T00:19:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049964664999999998 2024-09-18T00:19:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010590+0000", 2024-09-18T00:19:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3094999999999999e-05 2024-09-18T00:19:49.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010642+0000", 2024-09-18T00:19:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2073000000000001e-05 2024-09-18T00:19:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010709+0000", 2024-09-18T00:19:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6563000000000006e-05 2024-09-18T00:19:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.550 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17562 3.a e60/23 ondisk, result = 0)", 2024-09-18T00:19:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 59.480407855999999, 2024-09-18T00:19:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042539307999999998, 2024-09-18T00:19:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:49.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968180+0000", 2024-09-18T00:19:49.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-18T00:19:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968188+0000", 2024-09-18T00:19:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7419999999999997e-06 2024-09-18T00:19:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968217+0000", 2024-09-18T00:19:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8926999999999999e-05 2024-09-18T00:19:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:49.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968231+0000", 2024-09-18T00:19:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4168999999999999e-05 2024-09-18T00:19:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010575+0000", 2024-09-18T00:19:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042343923999999998 2024-09-18T00:19:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010584+0000", 2024-09-18T00:19:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-18T00:19:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:49.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010718+0000", 2024-09-18T00:19:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013344499999999999 2024-09-18T00:19:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:49.555 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:49.555 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:49.557 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:19:49.557 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:49.553+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:49.657 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:49.656+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:49.664 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is unset 2024-09-18T00:19:49.733 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:49.733 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:49.734 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:49.734 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:49.750 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:19:49.758 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:49.756+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:49.859 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:49.857+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:49.875 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:49.875 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:49.875 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:49.875 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:49.875 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:49.889 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:19:49.974 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:49.973+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:50.055 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:50.055 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:50.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:50.055 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:50.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.056 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2228 3.15 3:aa2fc059:::rbd_data.112aa7ba4578.000000000000014a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:50.056 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:50.056 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 89.787828809000004, 2024-09-18T00:19:50.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.068966377999999995, 2024-09-18T00:19:50.057 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.057 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.058 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:50.058 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:50.058 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:50.058 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:50.058 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2228 2024-09-18T00:19:50.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.059 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:50.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:50.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263099+0000", 2024-09-18T00:19:50.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2359999999999999e-06 2024-09-18T00:19:50.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263115+0000", 2024-09-18T00:19:50.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5996999999999999e-05 2024-09-18T00:19:50.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263156+0000", 2024-09-18T00:19:50.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:19:50.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263183+0000", 2024-09-18T00:19:50.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7233000000000002e-05 2024-09-18T00:19:50.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263304+0000", 2024-09-18T00:19:50.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121209 2024-09-18T00:19:50.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263528+0000", 2024-09-18T00:19:50.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022401800000000001 2024-09-18T00:19:50.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:19:50.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.266458+0000", 2024-09-18T00:19:50.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029296980000000001 2024-09-18T00:19:50.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:50.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331628+0000", 2024-09-18T00:19:50.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.065170045999999995 2024-09-18T00:19:50.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:50.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331819+0000", 2024-09-18T00:19:50.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000190819 2024-09-18T00:19:50.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:50.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331971+0000", 2024-09-18T00:19:50.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151972 2024-09-18T00:19:50.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332063+0000", 2024-09-18T00:19:50.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2083999999999995e-05 2024-09-18T00:19:50.069 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.069 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.069 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.070 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2854 3.f 3:f02b0bca:::rbd_data.112aa7ba4578.0000000000000135:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:19:50.070 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:50.070 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 88.180856319, 2024-09-18T00:19:50.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.082582802999999996, 2024-09-18T00:19:50.070 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.070 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.070 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:50.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:50.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:50.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:50.071 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2854 2024-09-18T00:19:50.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.071 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:50.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:50.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870071+0000", 2024-09-18T00:19:50.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.227e-06 2024-09-18T00:19:50.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870083+0000", 2024-09-18T00:19:50.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1909999999999999e-05 2024-09-18T00:19:50.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870114+0000", 2024-09-18T00:19:50.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0576e-05 2024-09-18T00:19:50.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870129+0000", 2024-09-18T00:19:50.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.485e-05 2024-09-18T00:19:50.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870256+0000", 2024-09-18T00:19:50.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012691300000000001 2024-09-18T00:19:50.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870493+0000", 2024-09-18T00:19:50.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023737399999999999 2024-09-18T00:19:50.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:50.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.874933+0000", 2024-09-18T00:19:50.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044398299999999996 2024-09-18T00:19:50.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:50.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952452+0000", 2024-09-18T00:19:50.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.077518918000000006 2024-09-18T00:19:50.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:50.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952486+0000", 2024-09-18T00:19:50.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4035999999999997e-05 2024-09-18T00:19:50.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:50.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952575+0000", 2024-09-18T00:19:50.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.954e-05 2024-09-18T00:19:50.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952652+0000", 2024-09-18T00:19:50.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6629000000000001e-05 2024-09-18T00:19:50.082 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.082 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.082 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.082 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2854 3.f e38/32 ondisk, result = 0)", 2024-09-18T00:19:50.083 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:50.083 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 88.163325503999999, 2024-09-18T00:19:50.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064868625999999999, 2024-09-18T00:19:50.083 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.083 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.083 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:50.083 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:50.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:50.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887602+0000", 2024-09-18T00:19:50.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.677e-06 2024-09-18T00:19:50.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887609+0000", 2024-09-18T00:19:50.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6729999999999992e-06 2024-09-18T00:19:50.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887637+0000", 2024-09-18T00:19:50.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8206000000000001e-05 2024-09-18T00:19:50.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887651+0000", 2024-09-18T00:19:50.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3387000000000001e-05 2024-09-18T00:19:50.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952422+0000", 2024-09-18T00:19:50.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064771366999999996 2024-09-18T00:19:50.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952441+0000", 2024-09-18T00:19:50.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8737e-05 2024-09-18T00:19:50.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952469+0000", 2024-09-18T00:19:50.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7579000000000001e-05 2024-09-18T00:19:50.090 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.091 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.091 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.091 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15879 3.16 3:692b47b6:::rbd_object_map.112a13ec2d6c.000000000000000b:head [call] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:19:50.091 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:50.091 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.396161597000003, 2024-09-18T00:19:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.011184018, 2024-09-18T00:19:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:50.092 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15879 2024-09-18T00:19:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654768+0000", 2024-09-18T00:19:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9129999999999996e-06 2024-09-18T00:19:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654786+0000", 2024-09-18T00:19:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8695999999999999e-05 2024-09-18T00:19:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654832+0000", 2024-09-18T00:19:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5735000000000002e-05 2024-09-18T00:19:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654860+0000", 2024-09-18T00:19:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7789000000000001e-05 2024-09-18T00:19:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655048+0000", 2024-09-18T00:19:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018781900000000001 2024-09-18T00:19:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:19:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655070+0000", 2024-09-18T00:19:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1916000000000002e-05 2024-09-18T00:19:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663335+0000", 2024-09-18T00:19:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0082651210000000003 2024-09-18T00:19:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663358+0000", 2024-09-18T00:19:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2986e-05 2024-09-18T00:19:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663739+0000", 2024-09-18T00:19:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0003814820000001 2024-09-18T00:19:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663920+0000", 2024-09-18T00:19:50.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018116900000000001 2024-09-18T00:19:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.665948+0000", 2024-09-18T00:19:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020273919999999998 2024-09-18T00:19:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.101 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15881 3.16 3:6cc14920:::rbd_header.112a3285400b:head [watch reconnect cookie 140685950184608 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:19:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.396000602000001, 2024-09-18T00:19:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0138723599999999, 2024-09-18T00:19:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.102 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15881 2024-09-18T00:19:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654926+0000", 2024-09-18T00:19:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6470000000000001e-06 2024-09-18T00:19:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654935+0000", 2024-09-18T00:19:50.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8859999999999993e-06 2024-09-18T00:19:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654960+0000", 2024-09-18T00:19:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4689999999999999e-05 2024-09-18T00:19:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654976+0000", 2024-09-18T00:19:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6319000000000001e-05 2024-09-18T00:19:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655238+0000", 2024-09-18T00:19:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026145999999999998 2024-09-18T00:19:50.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655255+0000", 2024-09-18T00:19:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7249999999999999e-05 2024-09-18T00:19:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663416+0000", 2024-09-18T00:19:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081607959999999997 2024-09-18T00:19:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663436+0000", 2024-09-18T00:19:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0384e-05 2024-09-18T00:19:50.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666184+0000", 2024-09-18T00:19:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002748121 2024-09-18T00:19:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666452+0000", 2024-09-18T00:19:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026801700000000001 2024-09-18T00:19:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.668797+0000", 2024-09-18T00:19:50.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023447899999999998 2024-09-18T00:19:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17089 3.f 3:f7681386:::rbd_data.112a6efb7564.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:19:50.112 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.171073505999999, 2024-09-18T00:19:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.12573889399999999, 2024-09-18T00:19:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:50.113 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17089 2024-09-18T00:19:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879854+0000", 2024-09-18T00:19:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:19:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879867+0000", 2024-09-18T00:19:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3103e-05 2024-09-18T00:19:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879898+0000", 2024-09-18T00:19:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0103e-05 2024-09-18T00:19:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879912+0000", 2024-09-18T00:19:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4636e-05 2024-09-18T00:19:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879992+0000", 2024-09-18T00:19:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9814000000000005e-05 2024-09-18T00:19:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.880101+0000", 2024-09-18T00:19:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109172 2024-09-18T00:19:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.887269+0000", 2024-09-18T00:19:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071679300000000003 2024-09-18T00:19:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005269+0000", 2024-09-18T00:19:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.117999621 2024-09-18T00:19:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005317+0000", 2024-09-18T00:19:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8279000000000003e-05 2024-09-18T00:19:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005464+0000", 2024-09-18T00:19:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014661200000000001 2024-09-18T00:19:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005591+0000", 2024-09-18T00:19:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000127107 2024-09-18T00:19:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17089 3.f e59/32 ondisk, result = 0)", 2024-09-18T00:19:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.146966640999999, 2024-09-18T00:19:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101334403, 2024-09-18T00:19:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903960+0000", 2024-09-18T00:19:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.33e-06 2024-09-18T00:19:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903966+0000", 2024-09-18T00:19:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5929999999999996e-06 2024-09-18T00:19:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903986+0000", 2024-09-18T00:19:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.018e-05 2024-09-18T00:19:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903997+0000", 2024-09-18T00:19:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0988999999999999e-05 2024-09-18T00:19:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005171+0000", 2024-09-18T00:19:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101174267 2024-09-18T00:19:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005200+0000", 2024-09-18T00:19:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8540000000000001e-05 2024-09-18T00:19:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005293+0000", 2024-09-18T00:19:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504000000000005e-05 2024-09-18T00:19:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24529 3.16 3:6857ce1a:::rbd_data.112a990280ef.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 43.414155229999999, 2024-09-18T00:19:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10693805200000001, 2024-09-18T00:19:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24529 2024-09-18T00:19:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636772+0000", 2024-09-18T00:19:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1100000000000001e-06 2024-09-18T00:19:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636783+0000", 2024-09-18T00:19:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1005999999999999e-05 2024-09-18T00:19:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636814+0000", 2024-09-18T00:19:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1047000000000003e-05 2024-09-18T00:19:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636829+0000", 2024-09-18T00:19:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4363e-05 2024-09-18T00:19:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636980+0000", 2024-09-18T00:19:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015181099999999999 2024-09-18T00:19:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.637237+0000", 2024-09-18T00:19:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000256352 2024-09-18T00:19:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743536+0000", 2024-09-18T00:19:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.106298825 2024-09-18T00:19:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743599+0000", 2024-09-18T00:19:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3449999999999997e-05 2024-09-18T00:19:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743708+0000", 2024-09-18T00:19:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109088 2024-09-18T00:19:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24805 3.16 3:6de5e15d:::rbd_data.112a990280ef.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.828351998000002, 2024-09-18T00:19:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14775107700000001, 2024-09-18T00:19:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24805 2024-09-18T00:19:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222574+0000", 2024-09-18T00:19:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0210000000000001e-06 2024-09-18T00:19:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222581+0000", 2024-09-18T00:19:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.144e-06 2024-09-18T00:19:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222595+0000", 2024-09-18T00:19:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4567e-05 2024-09-18T00:19:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222603+0000", 2024-09-18T00:19:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6529999999999994e-06 2024-09-18T00:19:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222687+0000", 2024-09-18T00:19:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4587999999999997e-05 2024-09-18T00:19:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222828+0000", 2024-09-18T00:19:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140178 2024-09-18T00:19:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370231+0000", 2024-09-18T00:19:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14740378700000001 2024-09-18T00:19:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370269+0000", 2024-09-18T00:19:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7886999999999998e-05 2024-09-18T00:19:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370324+0000", 2024-09-18T00:19:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5251999999999999e-05 2024-09-18T00:19:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24852 3.13 3:cc64f0fd:::rbd_data.112a990280ef.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.595612527999997, 2024-09-18T00:19:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.066392943999999995, 2024-09-18T00:19:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24852 2024-09-18T00:19:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455315+0000", 2024-09-18T00:19:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.705e-06 2024-09-18T00:19:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455324+0000", 2024-09-18T00:19:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8120000000000002e-06 2024-09-18T00:19:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455349+0000", 2024-09-18T00:19:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4250999999999999e-05 2024-09-18T00:19:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455361+0000", 2024-09-18T00:19:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2805e-05 2024-09-18T00:19:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455425+0000", 2024-09-18T00:19:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3981999999999998e-05 2024-09-18T00:19:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455480+0000", 2024-09-18T00:19:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4792e-05 2024-09-18T00:19:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.460671+0000", 2024-09-18T00:19:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051907300000000002 2024-09-18T00:19:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521355+0000", 2024-09-18T00:19:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060684208000000003 2024-09-18T00:19:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521493+0000", 2024-09-18T00:19:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013803200000000001 2024-09-18T00:19:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521590+0000", 2024-09-18T00:19:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6339e-05 2024-09-18T00:19:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521706+0000", 2024-09-18T00:19:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116288 2024-09-18T00:19:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:25269 3.f 3:f6adfd48:::rbd_data.112aa34d8ec3.0000000000000145:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T00:19:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 41.446022567, 2024-09-18T00:19:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.075209085999999994, 2024-09-18T00:19:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 25269 2024-09-18T00:19:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604904+0000", 2024-09-18T00:19:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:19:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604910+0000", 2024-09-18T00:19:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.964e-06 2024-09-18T00:19:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604925+0000", 2024-09-18T00:19:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4894e-05 2024-09-18T00:19:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604932+0000", 2024-09-18T00:19:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-18T00:19:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604990+0000", 2024-09-18T00:19:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8143000000000002e-05 2024-09-18T00:19:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.605104+0000", 2024-09-18T00:19:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011404399999999999 2024-09-18T00:19:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.608755+0000", 2024-09-18T00:19:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003651081 2024-09-18T00:19:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679920+0000", 2024-09-18T00:19:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.071164543999999996 2024-09-18T00:19:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679978+0000", 2024-09-18T00:19:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8585000000000002e-05 2024-09-18T00:19:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680036+0000", 2024-09-18T00:19:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7978e-05 2024-09-18T00:19:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680112+0000", 2024-09-18T00:19:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5527999999999999e-05 2024-09-18T00:19:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:25269 3.f e72/32 ondisk, result = 0)", 2024-09-18T00:19:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 41.432812181999999, 2024-09-18T00:19:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062014761000000002, 2024-09-18T00:19:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618114+0000", 2024-09-18T00:19:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1650000000000001e-06 2024-09-18T00:19:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618119+0000", 2024-09-18T00:19:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-18T00:19:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618134+0000", 2024-09-18T00:19:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5770999999999999e-05 2024-09-18T00:19:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618143+0000", 2024-09-18T00:19:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.551e-06 2024-09-18T00:19:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679967+0000", 2024-09-18T00:19:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061823937000000002 2024-09-18T00:19:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679975+0000", 2024-09-18T00:19:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7340000000000008e-06 2024-09-18T00:19:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680128+0000", 2024-09-18T00:19:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015338299999999999 2024-09-18T00:19:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48484 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 15704064~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:19:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.683610073000001, 2024-09-18T00:19:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058450876999999998, 2024-09-18T00:19:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48484 2024-09-18T00:19:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:19:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:19:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370429+0000", 2024-09-18T00:19:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031138530000000002 2024-09-18T00:19:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370435+0000", 2024-09-18T00:19:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6269999999999997e-06 2024-09-18T00:19:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370455+0000", 2024-09-18T00:19:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0426999999999999e-05 2024-09-18T00:19:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370465+0000", 2024-09-18T00:19:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0118999999999999e-05 2024-09-18T00:19:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425525+0000", 2024-09-18T00:19:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055059943 2024-09-18T00:19:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425645+0000", 2024-09-18T00:19:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119392 2024-09-18T00:19:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425766+0000", 2024-09-18T00:19:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012151599999999999 2024-09-18T00:19:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48503 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16326656~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.679729672000001, 2024-09-18T00:19:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058310395000000001, 2024-09-18T00:19:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48503 2024-09-18T00:19:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0100000000000004e-07 2024-09-18T00:19:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371199+0000", 2024-09-18T00:19:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8899999999999999e-06 2024-09-18T00:19:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371211+0000", 2024-09-18T00:19:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1707e-05 2024-09-18T00:19:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371218+0000", 2024-09-18T00:19:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.131e-06 2024-09-18T00:19:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429426+0000", 2024-09-18T00:19:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058207771999999998 2024-09-18T00:19:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429467+0000", 2024-09-18T00:19:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1532999999999997e-05 2024-09-18T00:19:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429506+0000", 2024-09-18T00:19:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8661000000000001e-05 2024-09-18T00:19:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48504 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16359424~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.679697853, 2024-09-18T00:19:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058412775, 2024-09-18T00:19:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48504 2024-09-18T00:19:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6899999999999996e-07 2024-09-18T00:19:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371231+0000", 2024-09-18T00:19:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6709999999999999e-06 2024-09-18T00:19:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371240+0000", 2024-09-18T00:19:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7309999999999999e-06 2024-09-18T00:19:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371249+0000", 2024-09-18T00:19:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4280000000000002e-06 2024-09-18T00:19:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429533+0000", 2024-09-18T00:19:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058283852999999997 2024-09-18T00:19:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429590+0000", 2024-09-18T00:19:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6824999999999999e-05 2024-09-18T00:19:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429640+0000", 2024-09-18T00:19:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0498000000000003e-05 2024-09-18T00:19:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48505 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16392192~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.679664880000001, 2024-09-18T00:19:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058482777999999999, 2024-09-18T00:19:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48505 2024-09-18T00:19:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7800000000000001e-07 2024-09-18T00:19:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371264+0000", 2024-09-18T00:19:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-18T00:19:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371274+0000", 2024-09-18T00:19:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4690000000000003e-06 2024-09-18T00:19:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371281+0000", 2024-09-18T00:19:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4120000000000004e-06 2024-09-18T00:19:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429665+0000", 2024-09-18T00:19:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058384108999999997 2024-09-18T00:19:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429705+0000", 2024-09-18T00:19:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0299000000000002e-05 2024-09-18T00:19:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429743+0000", 2024-09-18T00:19:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7951000000000003e-05 2024-09-18T00:19:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48506 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16424960~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:19:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.679635668, 2024-09-18T00:19:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058604485999999997, 2024-09-18T00:19:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48506 2024-09-18T00:19:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:19:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:19:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371355+0000", 2024-09-18T00:19:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5549000000000005e-05 2024-09-18T00:19:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371359+0000", 2024-09-18T00:19:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1049999999999997e-06 2024-09-18T00:19:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371380+0000", 2024-09-18T00:19:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0205000000000001e-05 2024-09-18T00:19:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371388+0000", 2024-09-18T00:19:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3229999999999998e-06 2024-09-18T00:19:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429771+0000", 2024-09-18T00:19:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058382878999999999 2024-09-18T00:19:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429842+0000", 2024-09-18T00:19:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1110000000000002e-05 2024-09-18T00:19:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429894+0000", 2024-09-18T00:19:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2315000000000003e-05 2024-09-18T00:19:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48507 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16457728~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:19:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.679524308000001, 2024-09-18T00:19:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058597275999999997, 2024-09-18T00:19:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48507 2024-09-18T00:19:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:19:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:19:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374394+0000", 2024-09-18T00:19:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029924449999999998 2024-09-18T00:19:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374400+0000", 2024-09-18T00:19:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0229999999999998e-06 2024-09-18T00:19:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374419+0000", 2024-09-18T00:19:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9488e-05 2024-09-18T00:19:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374432+0000", 2024-09-18T00:19:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2492e-05 2024-09-18T00:19:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429917+0000", 2024-09-18T00:19:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055485087000000002 2024-09-18T00:19:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429958+0000", 2024-09-18T00:19:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1258999999999999e-05 2024-09-18T00:19:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429998+0000", 2024-09-18T00:19:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0482000000000003e-05 2024-09-18T00:19:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:58332 3.15 3:ac910418:::rbd_data.112a17f95ba0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T00:19:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:19:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.993020276999999, 2024-09-18T00:19:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.071238735999999997, 2024-09-18T00:19:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 58332 2024-09-18T00:19:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:19:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:19:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057907+0000", 2024-09-18T00:19:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1859999999999999e-06 2024-09-18T00:19:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057920+0000", 2024-09-18T00:19:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3013000000000001e-05 2024-09-18T00:19:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057949+0000", 2024-09-18T00:19:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8745999999999999e-05 2024-09-18T00:19:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057966+0000", 2024-09-18T00:19:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6477000000000002e-05 2024-09-18T00:19:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.058035+0000", 2024-09-18T00:19:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9560000000000005e-05 2024-09-18T00:19:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.058133+0000", 2024-09-18T00:19:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8051e-05 2024-09-18T00:19:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:19:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.061809+0000", 2024-09-18T00:19:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003675453 2024-09-18T00:19:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128667+0000", 2024-09-18T00:19:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.066857960999999994 2024-09-18T00:19:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128952+0000", 2024-09-18T00:19:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028583100000000001 2024-09-18T00:19:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129060+0000", 2024-09-18T00:19:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010806000000000001 2024-09-18T00:19:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129144+0000", 2024-09-18T00:19:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3398000000000003e-05 2024-09-18T00:19:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:58332 3.15 e96/32 ondisk, result = 0)", 2024-09-18T00:19:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:19:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.980827447999999, 2024-09-18T00:19:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059068112999999998, 2024-09-18T00:19:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:19:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:19:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070099+0000", 2024-09-18T00:19:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9899999999999999e-07 2024-09-18T00:19:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070103+0000", 2024-09-18T00:19:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6880000000000001e-06 2024-09-18T00:19:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070115+0000", 2024-09-18T00:19:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2088e-05 2024-09-18T00:19:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070122+0000", 2024-09-18T00:19:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3900000000000004e-06 2024-09-18T00:19:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128758+0000", 2024-09-18T00:19:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058635541999999999 2024-09-18T00:19:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128774+0000", 2024-09-18T00:19:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6858999999999999e-05 2024-09-18T00:19:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129166+0000", 2024-09-18T00:19:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039164700000000001 2024-09-18T00:19:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:50.193 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:50.195 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:19:50.195 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:50.193+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:50.295 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:50.292+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:50.342 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:19:50.369 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:50.369 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:50.369 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:50.369 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:50.383 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:19:50.426 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:50.425+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:50.497 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:50.497 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:50.498 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:50.493+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:50.499 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:50.499 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:50.499 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:50.499 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:50.499 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:50.514 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:19:50.514 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:19:50.514 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-18T00:19:50.530 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:50.530 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:50.547 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:50.547 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:50.566 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:50.567 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:50.586 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:50.586 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:50.598 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:50.596+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:50.604 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:50.605 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:50.624 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:50.686 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:50.686 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:50.698 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:50.696+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:50.703 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:19:50.799 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:50.797+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:50.856 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:50.856 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:50.871 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:19:50.921 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:50.920+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:50.997 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:50.997 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:51.001 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:50.997+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:51.011 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:51.101 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:51.097+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:51.185 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:51.185 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:51.200 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:19:51.202 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:51.201+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:51.303 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:51.300+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:51.322 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:51.322 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:51.337 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:19:51.403 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:51.401+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:51.501 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:51.501 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:51.504 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:51.501+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:51.515 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:19:51.605 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:51.604+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:51.634 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:51.634 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:51.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:19:51.697 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:19:51.706 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:51.701+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:51.804 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:19:51.806 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:51.804+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:51.823 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:51.823 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:51.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:19:51.907 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:51.905+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:51.937 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:51.937 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:52.006 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:52.006 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:52.007 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:52.004+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:52.020 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:19:52.108 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:52.104+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:52.178 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:52.178 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:52.192 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:19:52.238 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:52.236+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:52.309 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:52.305+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:52.324 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:52.324 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:52.338 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:19:52.410 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:52.408+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:52.507 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:52.507 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:52.511 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:52.508+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:52.521 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:19:52.612 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:52.609+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:52.641 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:19:52.644 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:52.644 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:52.658 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:19:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is set 2024-09-18T00:19:52.746 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:52.745+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:52.803 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:52.803 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:52.813 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:52.809+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:52.820 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:19:52.820 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:19:52.837 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:52.837 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:52.842 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:52.842 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:52.853 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:52.858 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:19:52.874 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:52.874 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:52.893 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:52.893 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:52.913 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:52.913 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:52.930 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:52.930 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:52.951 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:52.949+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:53.015 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:53.012+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:53.023 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:53.023 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:53.024 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:53.024 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:53.038 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:19:53.126 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:53.125+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:53.208 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:53.208 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:53.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:53.209 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:53.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.209 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2229 3.10 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:53.209 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:53.209 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 92.941652562000002, 2024-09-18T00:19:53.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048730086999999998, 2024-09-18T00:19:53.209 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.209 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.209 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:53.209 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.209 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.210 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.210 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2229 2024-09-18T00:19:53.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.210 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:53.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:53.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263362+0000", 2024-09-18T00:19:53.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3869999999999999e-06 2024-09-18T00:19:53.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263381+0000", 2024-09-18T00:19:53.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9153000000000001e-05 2024-09-18T00:19:53.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263418+0000", 2024-09-18T00:19:53.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6346000000000001e-05 2024-09-18T00:19:53.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263435+0000", 2024-09-18T00:19:53.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7309999999999999e-05 2024-09-18T00:19:53.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263589+0000", 2024-09-18T00:19:53.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015451199999999999 2024-09-18T00:19:53.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263902+0000", 2024-09-18T00:19:53.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031291799999999997 2024-09-18T00:19:53.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:53.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.265995+0000", 2024-09-18T00:19:53.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020931809999999999 2024-09-18T00:19:53.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:53.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311788+0000", 2024-09-18T00:19:53.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045792078 2024-09-18T00:19:53.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:53.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311903+0000", 2024-09-18T00:19:53.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011510699999999999 2024-09-18T00:19:53.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:53.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312009+0000", 2024-09-18T00:19:53.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010630599999999999 2024-09-18T00:19:53.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312089+0000", 2024-09-18T00:19:53.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9789000000000005e-05 2024-09-18T00:19:53.214 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.214 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.215 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.215 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2229 3.10 e36/23 ondisk, result = 0)", 2024-09-18T00:19:53.215 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:53.215 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 92.922506902999999, 2024-09-18T00:19:53.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029600955000000002, 2024-09-18T00:19:53.215 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.215 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.215 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.215 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:53.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:53.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282507+0000", 2024-09-18T00:19:53.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9160000000000001e-06 2024-09-18T00:19:53.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282522+0000", 2024-09-18T00:19:53.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5e-05 2024-09-18T00:19:53.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282558+0000", 2024-09-18T00:19:53.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5694000000000001e-05 2024-09-18T00:19:53.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282580+0000", 2024-09-18T00:19:53.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2323e-05 2024-09-18T00:19:53.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311873+0000", 2024-09-18T00:19:53.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029293013999999999 2024-09-18T00:19:53.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311892+0000", 2024-09-18T00:19:53.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8556e-05 2024-09-18T00:19:53.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312105+0000", 2024-09-18T00:19:53.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021345199999999999 2024-09-18T00:19:53.218 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.219 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.219 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.219 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5714 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:53.219 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:53.219 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.894804399999998, 2024-09-18T00:19:53.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027448727999999999, 2024-09-18T00:19:53.219 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.219 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.219 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:53.219 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.219 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.219 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.220 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5714 2024-09-18T00:19:53.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.220 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:53.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:53.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310211+0000", 2024-09-18T00:19:53.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5639999999999998e-06 2024-09-18T00:19:53.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312678+0000", 2024-09-18T00:19:53.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002466761 2024-09-18T00:19:53.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312746+0000", 2024-09-18T00:19:53.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8046000000000005e-05 2024-09-18T00:19:53.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312774+0000", 2024-09-18T00:19:53.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7813e-05 2024-09-18T00:19:53.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312927+0000", 2024-09-18T00:19:53.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015328499999999999 2024-09-18T00:19:53.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313214+0000", 2024-09-18T00:19:53.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028683099999999998 2024-09-18T00:19:53.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:53.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313708+0000", 2024-09-18T00:19:53.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049379100000000004 2024-09-18T00:19:53.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:53.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336895+0000", 2024-09-18T00:19:53.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023186801 2024-09-18T00:19:53.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:53.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337454+0000", 2024-09-18T00:19:53.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055979399999999996 2024-09-18T00:19:53.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:53.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337571+0000", 2024-09-18T00:19:53.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116821 2024-09-18T00:19:53.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337656+0000", 2024-09-18T00:19:53.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4221000000000006e-05 2024-09-18T00:19:53.225 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.225 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.225 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.225 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6801 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:53.226 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:53.226 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.167967951999998, 2024-09-18T00:19:53.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051953602000000002, 2024-09-18T00:19:53.226 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.226 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.226 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:53.226 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.226 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.226 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.226 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6801 2024-09-18T00:19:53.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.226 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:53.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:53.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037046+0000", 2024-09-18T00:19:53.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.19e-06 2024-09-18T00:19:53.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037062+0000", 2024-09-18T00:19:53.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586999999999998e-05 2024-09-18T00:19:53.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037096+0000", 2024-09-18T00:19:53.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4112000000000001e-05 2024-09-18T00:19:53.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037120+0000", 2024-09-18T00:19:53.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.355e-05 2024-09-18T00:19:53.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037246+0000", 2024-09-18T00:19:53.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012652200000000001 2024-09-18T00:19:53.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037385+0000", 2024-09-18T00:19:53.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013924599999999999 2024-09-18T00:19:53.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:53.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.040881+0000", 2024-09-18T00:19:53.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034954830000000002 2024-09-18T00:19:53.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:53.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088794+0000", 2024-09-18T00:19:53.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047912897000000003 2024-09-18T00:19:53.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:53.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088828+0000", 2024-09-18T00:19:53.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3880000000000001e-05 2024-09-18T00:19:53.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:53.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088916+0000", 2024-09-18T00:19:53.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.865e-05 2024-09-18T00:19:53.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088997+0000", 2024-09-18T00:19:53.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0484999999999999e-05 2024-09-18T00:19:53.231 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.232 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.232 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.232 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:6801 3.10 e42/23 ondisk, result = 0)", 2024-09-18T00:19:53.232 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:53.232 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.154160542, 2024-09-18T00:19:53.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037960394000000001, 2024-09-18T00:19:53.232 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.232 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.233 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.233 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:53.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:53.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050852+0000", 2024-09-18T00:19:53.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:19:53.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050860+0000", 2024-09-18T00:19:53.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7729999999999999e-06 2024-09-18T00:19:53.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050891+0000", 2024-09-18T00:19:53.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0617000000000002e-05 2024-09-18T00:19:53.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050910+0000", 2024-09-18T00:19:53.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.944e-05 2024-09-18T00:19:53.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088766+0000", 2024-09-18T00:19:53.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037855633999999999 2024-09-18T00:19:53.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088784+0000", 2024-09-18T00:19:53.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7767000000000001e-05 2024-09-18T00:19:53.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088811+0000", 2024-09-18T00:19:53.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7382999999999999e-05 2024-09-18T00:19:53.236 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.236 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.236 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.236 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12594 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:53.236 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:53.236 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.813384004, 2024-09-18T00:19:53.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026492600000000002, 2024-09-18T00:19:53.237 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.238 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.238 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.238 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.238 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.238 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.238 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12594 2024-09-18T00:19:53.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.238 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:53.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:53.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391629+0000", 2024-09-18T00:19:53.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.86e-06 2024-09-18T00:19:53.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391637+0000", 2024-09-18T00:19:53.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3999999999999992e-06 2024-09-18T00:19:53.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391664+0000", 2024-09-18T00:19:53.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6536000000000001e-05 2024-09-18T00:19:53.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391681+0000", 2024-09-18T00:19:53.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6563e-05 2024-09-18T00:19:53.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417791+0000", 2024-09-18T00:19:53.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026110936000000001 2024-09-18T00:19:53.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417933+0000", 2024-09-18T00:19:53.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014112099999999999 2024-09-18T00:19:53.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418120+0000", 2024-09-18T00:19:53.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018718399999999999 2024-09-18T00:19:53.241 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.241 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.242 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.242 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12595 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:53.242 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:53.242 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.813307805999997, 2024-09-18T00:19:53.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026819428999999999, 2024-09-18T00:19:53.242 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.242 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.243 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.243 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12595 2024-09-18T00:19:53.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.243 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:53.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:53.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391705+0000", 2024-09-18T00:19:53.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3999999999999999e-06 2024-09-18T00:19:53.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391711+0000", 2024-09-18T00:19:53.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7459999999999997e-06 2024-09-18T00:19:53.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391733+0000", 2024-09-18T00:19:53.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2923000000000001e-05 2024-09-18T00:19:53.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391748+0000", 2024-09-18T00:19:53.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4790000000000001e-05 2024-09-18T00:19:53.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418180+0000", 2024-09-18T00:19:53.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026431888000000001 2024-09-18T00:19:53.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418328+0000", 2024-09-18T00:19:53.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000147725 2024-09-18T00:19:53.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418523+0000", 2024-09-18T00:19:53.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000194957 2024-09-18T00:19:53.246 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.246 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.246 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.247 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12596 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:53.247 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:53.247 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.813244838000003, 2024-09-18T00:19:53.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027220296000000001, 2024-09-18T00:19:53.247 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.247 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.247 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.247 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.247 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.247 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.247 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12596 2024-09-18T00:19:53.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.247 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:53.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:53.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391768+0000", 2024-09-18T00:19:53.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.283e-06 2024-09-18T00:19:53.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391773+0000", 2024-09-18T00:19:53.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3970000000000002e-06 2024-09-18T00:19:53.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391795+0000", 2024-09-18T00:19:53.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1835e-05 2024-09-18T00:19:53.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391809+0000", 2024-09-18T00:19:53.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4107000000000001e-05 2024-09-18T00:19:53.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418582+0000", 2024-09-18T00:19:53.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026772918999999999 2024-09-18T00:19:53.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418780+0000", 2024-09-18T00:19:53.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000198042 2024-09-18T00:19:53.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418987+0000", 2024-09-18T00:19:53.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000206713 2024-09-18T00:19:53.254 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.254 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.254 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.255 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12597 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:53.255 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:53.255 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.813184320000005, 2024-09-18T00:19:53.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027612714, 2024-09-18T00:19:53.255 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.255 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.255 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.255 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.255 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.255 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.255 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12597 2024-09-18T00:19:53.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.256 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:53.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:53.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391828+0000", 2024-09-18T00:19:53.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.203e-06 2024-09-18T00:19:53.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391834+0000", 2024-09-18T00:19:53.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6529999999999999e-06 2024-09-18T00:19:53.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391856+0000", 2024-09-18T00:19:53.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2195999999999999e-05 2024-09-18T00:19:53.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391870+0000", 2024-09-18T00:19:53.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4063000000000001e-05 2024-09-18T00:19:53.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419049+0000", 2024-09-18T00:19:53.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027178751000000001 2024-09-18T00:19:53.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419239+0000", 2024-09-18T00:19:53.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019039400000000001 2024-09-18T00:19:53.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419440+0000", 2024-09-18T00:19:53.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000200454 2024-09-18T00:19:53.259 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.259 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.259 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.260 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12598 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:53.260 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:53.260 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.813121269000007, 2024-09-18T00:19:53.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028269056000000001, 2024-09-18T00:19:53.260 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.260 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.260 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.260 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.260 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.260 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.261 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12598 2024-09-18T00:19:53.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.261 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:53.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:53.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391891+0000", 2024-09-18T00:19:53.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.257e-06 2024-09-18T00:19:53.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391897+0000", 2024-09-18T00:19:53.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9560000000000002e-06 2024-09-18T00:19:53.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391920+0000", 2024-09-18T00:19:53.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3025999999999999e-05 2024-09-18T00:19:53.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391935+0000", 2024-09-18T00:19:53.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4853e-05 2024-09-18T00:19:53.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419527+0000", 2024-09-18T00:19:53.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027592378000000001 2024-09-18T00:19:53.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419681+0000", 2024-09-18T00:19:53.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015316 2024-09-18T00:19:53.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420159+0000", 2024-09-18T00:19:53.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000478426 2024-09-18T00:19:53.264 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.264 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.265 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.265 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12599 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:53.265 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:53.265 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.813057413999999, 2024-09-18T00:19:53.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028649857000000001, 2024-09-18T00:19:53.265 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.265 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.265 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.265 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.265 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.265 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.266 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12599 2024-09-18T00:19:53.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.266 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:53.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:53.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391955+0000", 2024-09-18T00:19:53.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-06 2024-09-18T00:19:53.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391961+0000", 2024-09-18T00:19:53.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7100000000000004e-06 2024-09-18T00:19:53.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391982+0000", 2024-09-18T00:19:53.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1492e-05 2024-09-18T00:19:53.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391996+0000", 2024-09-18T00:19:53.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4194e-05 2024-09-18T00:19:53.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420237+0000", 2024-09-18T00:19:53.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028240818000000001 2024-09-18T00:19:53.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420430+0000", 2024-09-18T00:19:53.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019228299999999999 2024-09-18T00:19:53.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420604+0000", 2024-09-18T00:19:53.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000174117 2024-09-18T00:19:53.269 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.269 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.269 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.269 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12600 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 270336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:53.269 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:53.269 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.812996420000005, 2024-09-18T00:19:53.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028964586, 2024-09-18T00:19:53.270 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.270 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.270 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.270 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.270 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.270 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.270 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12600 2024-09-18T00:19:53.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.270 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:53.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:53.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392016+0000", 2024-09-18T00:19:53.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.31e-06 2024-09-18T00:19:53.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392022+0000", 2024-09-18T00:19:53.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5600000000000001e-06 2024-09-18T00:19:53.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392045+0000", 2024-09-18T00:19:53.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3595999999999999e-05 2024-09-18T00:19:53.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392060+0000", 2024-09-18T00:19:53.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4727e-05 2024-09-18T00:19:53.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420669+0000", 2024-09-18T00:19:53.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028608855999999998 2024-09-18T00:19:53.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420813+0000", 2024-09-18T00:19:53.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014429500000000001 2024-09-18T00:19:53.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420979+0000", 2024-09-18T00:19:53.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016624200000000001 2024-09-18T00:19:53.273 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.274 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.274 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.274 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12601 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:53.274 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:53.274 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.812932515, 2024-09-18T00:19:53.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029292832000000001, 2024-09-18T00:19:53.274 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.274 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.274 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.274 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.274 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.274 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.274 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12601 2024-09-18T00:19:53.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.275 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:53.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:53.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392080+0000", 2024-09-18T00:19:53.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:53.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392086+0000", 2024-09-18T00:19:53.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7570000000000001e-06 2024-09-18T00:19:53.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392105+0000", 2024-09-18T00:19:53.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9415999999999998e-05 2024-09-18T00:19:53.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392132+0000", 2024-09-18T00:19:53.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7019e-05 2024-09-18T00:19:53.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421049+0000", 2024-09-18T00:19:53.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028916659000000001 2024-09-18T00:19:53.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421204+0000", 2024-09-18T00:19:53.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155224 2024-09-18T00:19:53.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421372+0000", 2024-09-18T00:19:53.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016754699999999999 2024-09-18T00:19:53.278 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.278 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.278 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.278 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12602 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:53.278 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:53.278 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.812859884000005, 2024-09-18T00:19:53.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029612833000000002, 2024-09-18T00:19:53.279 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.279 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.279 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.279 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.279 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.279 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.279 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12602 2024-09-18T00:19:53.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.279 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:53.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:53.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392152+0000", 2024-09-18T00:19:53.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0470000000000001e-06 2024-09-18T00:19:53.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392158+0000", 2024-09-18T00:19:53.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:19:53.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392181+0000", 2024-09-18T00:19:53.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2629e-05 2024-09-18T00:19:53.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392222+0000", 2024-09-18T00:19:53.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0772000000000002e-05 2024-09-18T00:19:53.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421441+0000", 2024-09-18T00:19:53.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029219327 2024-09-18T00:19:53.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421608+0000", 2024-09-18T00:19:53.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000166813 2024-09-18T00:19:53.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421764+0000", 2024-09-18T00:19:53.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015628500000000001 2024-09-18T00:19:53.282 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.282 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.283 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.283 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12603 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:53.283 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:53.283 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.812769489999994, 2024-09-18T00:19:53.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029902524, 2024-09-18T00:19:53.283 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.283 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.283 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.283 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12603 2024-09-18T00:19:53.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.284 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:53.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:53.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392243+0000", 2024-09-18T00:19:53.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:53.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392260+0000", 2024-09-18T00:19:53.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6860000000000001e-05 2024-09-18T00:19:53.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392282+0000", 2024-09-18T00:19:53.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2713000000000001e-05 2024-09-18T00:19:53.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392297+0000", 2024-09-18T00:19:53.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4375e-05 2024-09-18T00:19:53.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421830+0000", 2024-09-18T00:19:53.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029532668000000002 2024-09-18T00:19:53.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421985+0000", 2024-09-18T00:19:53.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155613 2024-09-18T00:19:53.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422144+0000", 2024-09-18T00:19:53.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000159102 2024-09-18T00:19:53.287 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.287 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.288 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.288 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12604 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:53.288 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:53.288 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.812694438999998, 2024-09-18T00:19:53.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030228695, 2024-09-18T00:19:53.288 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.288 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.288 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.288 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.288 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.288 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.288 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12604 2024-09-18T00:19:53.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.289 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:53.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:53.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392318+0000", 2024-09-18T00:19:53.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:53.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392337+0000", 2024-09-18T00:19:53.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:19:53.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392364+0000", 2024-09-18T00:19:53.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7288999999999999e-05 2024-09-18T00:19:53.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392379+0000", 2024-09-18T00:19:53.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:19:53.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422209+0000", 2024-09-18T00:19:53.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029829050999999999 2024-09-18T00:19:53.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422388+0000", 2024-09-18T00:19:53.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017988699999999999 2024-09-18T00:19:53.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422545+0000", 2024-09-18T00:19:53.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015701199999999999 2024-09-18T00:19:53.295 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.295 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.295 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.296 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12605 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:53.296 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:53.296 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.812609878000004, 2024-09-18T00:19:53.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030488674, 2024-09-18T00:19:53.296 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.296 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.297 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.297 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.297 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.297 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.297 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12605 2024-09-18T00:19:53.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.298 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:53.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:53.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392403+0000", 2024-09-18T00:19:53.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.883e-06 2024-09-18T00:19:53.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392421+0000", 2024-09-18T00:19:53.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7526000000000001e-05 2024-09-18T00:19:53.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392450+0000", 2024-09-18T00:19:53.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8972e-05 2024-09-18T00:19:53.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392488+0000", 2024-09-18T00:19:53.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8522000000000001e-05 2024-09-18T00:19:53.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422614+0000", 2024-09-18T00:19:53.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030125945000000001 2024-09-18T00:19:53.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422752+0000", 2024-09-18T00:19:53.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000138178 2024-09-18T00:19:53.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422890+0000", 2024-09-18T00:19:53.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013764800000000001 2024-09-18T00:19:53.305 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.305 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.305 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.306 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12606 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:53.306 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:53.306 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.812501217999994, 2024-09-18T00:19:53.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033727073000000003, 2024-09-18T00:19:53.306 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.306 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.307 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.307 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.307 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.307 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.307 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12606 2024-09-18T00:19:53.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.307 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:53.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:53.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392511+0000", 2024-09-18T00:19:53.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3090000000000001e-06 2024-09-18T00:19:53.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392517+0000", 2024-09-18T00:19:53.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.993e-06 2024-09-18T00:19:53.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392543+0000", 2024-09-18T00:19:53.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6152999999999998e-05 2024-09-18T00:19:53.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392558+0000", 2024-09-18T00:19:53.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4610000000000001e-05 2024-09-18T00:19:53.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.425959+0000", 2024-09-18T00:19:53.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033400484000000001 2024-09-18T00:19:53.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426106+0000", 2024-09-18T00:19:53.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014779800000000001 2024-09-18T00:19:53.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426237+0000", 2024-09-18T00:19:53.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013072600000000001 2024-09-18T00:19:53.313 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.313 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.314 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.314 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17562 3.a 3:5578626c:::rbd_data.112a6efb7564.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T00:19:53.314 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:53.314 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 63.247036057000003, 2024-09-18T00:19:53.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052733453999999999, 2024-09-18T00:19:53.314 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.314 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.314 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:53.314 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.315 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.315 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.315 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17562 2024-09-18T00:19:53.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.315 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:53.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:53.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957977+0000", 2024-09-18T00:19:53.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.119e-06 2024-09-18T00:19:53.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957988+0000", 2024-09-18T00:19:53.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0557000000000001e-05 2024-09-18T00:19:53.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958017+0000", 2024-09-18T00:19:53.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9156e-05 2024-09-18T00:19:53.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958036+0000", 2024-09-18T00:19:53.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8919e-05 2024-09-18T00:19:53.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958144+0000", 2024-09-18T00:19:53.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107905 2024-09-18T00:19:53.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958284+0000", 2024-09-18T00:19:53.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140216 2024-09-18T00:19:53.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:53.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.960572+0000", 2024-09-18T00:19:53.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022881860000000002 2024-09-18T00:19:53.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:53.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010537+0000", 2024-09-18T00:19:53.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049964664999999998 2024-09-18T00:19:53.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:53.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010590+0000", 2024-09-18T00:19:53.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3094999999999999e-05 2024-09-18T00:19:53.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:53.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010642+0000", 2024-09-18T00:19:53.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2073000000000001e-05 2024-09-18T00:19:53.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010709+0000", 2024-09-18T00:19:53.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6563000000000006e-05 2024-09-18T00:19:53.320 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.320 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.321 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.321 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17562 3.a e60/23 ondisk, result = 0)", 2024-09-18T00:19:53.321 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:53.321 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 63.236832686, 2024-09-18T00:19:53.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042539307999999998, 2024-09-18T00:19:53.321 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.322 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.322 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.322 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:53.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:53.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968180+0000", 2024-09-18T00:19:53.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-18T00:19:53.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968188+0000", 2024-09-18T00:19:53.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7419999999999997e-06 2024-09-18T00:19:53.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968217+0000", 2024-09-18T00:19:53.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8926999999999999e-05 2024-09-18T00:19:53.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968231+0000", 2024-09-18T00:19:53.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4168999999999999e-05 2024-09-18T00:19:53.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010575+0000", 2024-09-18T00:19:53.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042343923999999998 2024-09-18T00:19:53.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010584+0000", 2024-09-18T00:19:53.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-18T00:19:53.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010718+0000", 2024-09-18T00:19:53.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013344499999999999 2024-09-18T00:19:53.326 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.326 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.326 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.326 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.326 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.326 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:53.328 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:19:53.328 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:53.325+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:53.429 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:19:53.428+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:19:53.498 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:53.499 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:53.513 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:19:53.530 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:53.528+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:53.630 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:53.630 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:53.630 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:53.630 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:53.630 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:53.631 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:53.629+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:53.643 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:19:53.734 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:53.733+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:53.824 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:53.825 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:53.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:53.825 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:53.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.825 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2228 3.15 3:aa2fc059:::rbd_data.112aa7ba4578.000000000000014a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:53.825 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:53.825 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 93.557810171, 2024-09-18T00:19:53.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.068966377999999995, 2024-09-18T00:19:53.826 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.826 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.826 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:53.826 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.826 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.826 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.826 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2228 2024-09-18T00:19:53.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.827 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:53.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:53.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263099+0000", 2024-09-18T00:19:53.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2359999999999999e-06 2024-09-18T00:19:53.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263115+0000", 2024-09-18T00:19:53.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5996999999999999e-05 2024-09-18T00:19:53.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263156+0000", 2024-09-18T00:19:53.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:19:53.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263183+0000", 2024-09-18T00:19:53.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7233000000000002e-05 2024-09-18T00:19:53.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263304+0000", 2024-09-18T00:19:53.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121209 2024-09-18T00:19:53.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263528+0000", 2024-09-18T00:19:53.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022401800000000001 2024-09-18T00:19:53.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:19:53.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.266458+0000", 2024-09-18T00:19:53.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029296980000000001 2024-09-18T00:19:53.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:53.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331628+0000", 2024-09-18T00:19:53.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.065170045999999995 2024-09-18T00:19:53.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:53.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331819+0000", 2024-09-18T00:19:53.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000190819 2024-09-18T00:19:53.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:53.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331971+0000", 2024-09-18T00:19:53.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151972 2024-09-18T00:19:53.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332063+0000", 2024-09-18T00:19:53.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2083999999999995e-05 2024-09-18T00:19:53.832 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.833 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.833 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2854 3.f 3:f02b0bca:::rbd_data.112aa7ba4578.0000000000000135:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:19:53.833 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:53.833 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.950837680999996, 2024-09-18T00:19:53.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.082582802999999996, 2024-09-18T00:19:53.833 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.833 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.833 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:53.833 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.834 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.834 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.834 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2854 2024-09-18T00:19:53.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.834 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:53.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:53.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870071+0000", 2024-09-18T00:19:53.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.227e-06 2024-09-18T00:19:53.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870083+0000", 2024-09-18T00:19:53.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1909999999999999e-05 2024-09-18T00:19:53.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870114+0000", 2024-09-18T00:19:53.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0576e-05 2024-09-18T00:19:53.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870129+0000", 2024-09-18T00:19:53.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.485e-05 2024-09-18T00:19:53.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870256+0000", 2024-09-18T00:19:53.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012691300000000001 2024-09-18T00:19:53.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870493+0000", 2024-09-18T00:19:53.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023737399999999999 2024-09-18T00:19:53.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:53.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.874933+0000", 2024-09-18T00:19:53.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044398299999999996 2024-09-18T00:19:53.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:53.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952452+0000", 2024-09-18T00:19:53.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.077518918000000006 2024-09-18T00:19:53.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:53.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952486+0000", 2024-09-18T00:19:53.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4035999999999997e-05 2024-09-18T00:19:53.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:53.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952575+0000", 2024-09-18T00:19:53.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.954e-05 2024-09-18T00:19:53.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952652+0000", 2024-09-18T00:19:53.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6629000000000001e-05 2024-09-18T00:19:53.841 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.841 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.841 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.842 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2854 3.f e38/32 ondisk, result = 0)", 2024-09-18T00:19:53.842 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:53.842 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.933306865999995, 2024-09-18T00:19:53.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064868625999999999, 2024-09-18T00:19:53.842 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.842 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.842 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.843 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:53.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:53.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887602+0000", 2024-09-18T00:19:53.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.677e-06 2024-09-18T00:19:53.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887609+0000", 2024-09-18T00:19:53.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6729999999999992e-06 2024-09-18T00:19:53.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887637+0000", 2024-09-18T00:19:53.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8206000000000001e-05 2024-09-18T00:19:53.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887651+0000", 2024-09-18T00:19:53.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3387000000000001e-05 2024-09-18T00:19:53.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952422+0000", 2024-09-18T00:19:53.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064771366999999996 2024-09-18T00:19:53.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952441+0000", 2024-09-18T00:19:53.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8737e-05 2024-09-18T00:19:53.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952469+0000", 2024-09-18T00:19:53.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7579000000000001e-05 2024-09-18T00:19:53.848 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.848 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.849 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.849 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15879 3.16 3:692b47b6:::rbd_object_map.112a13ec2d6c.000000000000000b:head [call] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:19:53.849 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:53.849 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.166142958999998, 2024-09-18T00:19:53.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.011184018, 2024-09-18T00:19:53.849 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.849 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.850 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.850 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.850 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.850 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.850 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15879 2024-09-18T00:19:53.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.850 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:53.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:53.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654768+0000", 2024-09-18T00:19:53.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9129999999999996e-06 2024-09-18T00:19:53.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654786+0000", 2024-09-18T00:19:53.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8695999999999999e-05 2024-09-18T00:19:53.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654832+0000", 2024-09-18T00:19:53.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5735000000000002e-05 2024-09-18T00:19:53.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654860+0000", 2024-09-18T00:19:53.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7789000000000001e-05 2024-09-18T00:19:53.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655048+0000", 2024-09-18T00:19:53.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018781900000000001 2024-09-18T00:19:53.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:19:53.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655070+0000", 2024-09-18T00:19:53.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1916000000000002e-05 2024-09-18T00:19:53.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663335+0000", 2024-09-18T00:19:53.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0082651210000000003 2024-09-18T00:19:53.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:53.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663358+0000", 2024-09-18T00:19:53.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2986e-05 2024-09-18T00:19:53.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663739+0000", 2024-09-18T00:19:53.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0003814820000001 2024-09-18T00:19:53.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663920+0000", 2024-09-18T00:19:53.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018116900000000001 2024-09-18T00:19:53.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.665948+0000", 2024-09-18T00:19:53.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020273919999999998 2024-09-18T00:19:53.856 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.857 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.857 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.857 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15881 3.16 3:6cc14920:::rbd_header.112a3285400b:head [watch reconnect cookie 140685950184608 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:19:53.857 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:53.857 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.165981963999997, 2024-09-18T00:19:53.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0138723599999999, 2024-09-18T00:19:53.857 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.857 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.857 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.858 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.858 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.858 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15881 2024-09-18T00:19:53.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.858 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:53.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:53.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654926+0000", 2024-09-18T00:19:53.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6470000000000001e-06 2024-09-18T00:19:53.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654935+0000", 2024-09-18T00:19:53.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8859999999999993e-06 2024-09-18T00:19:53.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654960+0000", 2024-09-18T00:19:53.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4689999999999999e-05 2024-09-18T00:19:53.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654976+0000", 2024-09-18T00:19:53.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6319000000000001e-05 2024-09-18T00:19:53.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655238+0000", 2024-09-18T00:19:53.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026145999999999998 2024-09-18T00:19:53.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:53.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655255+0000", 2024-09-18T00:19:53.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7249999999999999e-05 2024-09-18T00:19:53.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663416+0000", 2024-09-18T00:19:53.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081607959999999997 2024-09-18T00:19:53.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:53.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663436+0000", 2024-09-18T00:19:53.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0384e-05 2024-09-18T00:19:53.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666184+0000", 2024-09-18T00:19:53.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002748121 2024-09-18T00:19:53.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666452+0000", 2024-09-18T00:19:53.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026801700000000001 2024-09-18T00:19:53.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.668797+0000", 2024-09-18T00:19:53.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023447899999999998 2024-09-18T00:19:53.863 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.863 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.863 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.863 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17089 3.f 3:f7681386:::rbd_data.112a6efb7564.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:19:53.863 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:53.864 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.941054867999995, 2024-09-18T00:19:53.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.12573889399999999, 2024-09-18T00:19:53.864 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.864 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.864 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:53.864 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.864 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.864 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.864 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17089 2024-09-18T00:19:53.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.864 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:53.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:53.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879854+0000", 2024-09-18T00:19:53.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:19:53.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879867+0000", 2024-09-18T00:19:53.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3103e-05 2024-09-18T00:19:53.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879898+0000", 2024-09-18T00:19:53.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0103e-05 2024-09-18T00:19:53.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879912+0000", 2024-09-18T00:19:53.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4636e-05 2024-09-18T00:19:53.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879992+0000", 2024-09-18T00:19:53.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9814000000000005e-05 2024-09-18T00:19:53.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.880101+0000", 2024-09-18T00:19:53.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109172 2024-09-18T00:19:53.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:53.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.887269+0000", 2024-09-18T00:19:53.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071679300000000003 2024-09-18T00:19:53.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:53.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005269+0000", 2024-09-18T00:19:53.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.117999621 2024-09-18T00:19:53.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:53.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005317+0000", 2024-09-18T00:19:53.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8279000000000003e-05 2024-09-18T00:19:53.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:53.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005464+0000", 2024-09-18T00:19:53.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014661200000000001 2024-09-18T00:19:53.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005591+0000", 2024-09-18T00:19:53.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000127107 2024-09-18T00:19:53.873 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.873 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.873 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.874 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17089 3.f e59/32 ondisk, result = 0)", 2024-09-18T00:19:53.874 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:53.874 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.916948003000002, 2024-09-18T00:19:53.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101334403, 2024-09-18T00:19:53.874 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.874 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.874 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.874 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:53.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:53.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903960+0000", 2024-09-18T00:19:53.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.33e-06 2024-09-18T00:19:53.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903966+0000", 2024-09-18T00:19:53.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5929999999999996e-06 2024-09-18T00:19:53.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903986+0000", 2024-09-18T00:19:53.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.018e-05 2024-09-18T00:19:53.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903997+0000", 2024-09-18T00:19:53.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0988999999999999e-05 2024-09-18T00:19:53.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005171+0000", 2024-09-18T00:19:53.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101174267 2024-09-18T00:19:53.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005200+0000", 2024-09-18T00:19:53.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8540000000000001e-05 2024-09-18T00:19:53.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005293+0000", 2024-09-18T00:19:53.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504000000000005e-05 2024-09-18T00:19:53.877 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.877 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.877 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.878 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24529 3.16 3:6857ce1a:::rbd_data.112a990280ef.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:53.878 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:53.878 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 47.184136592000002, 2024-09-18T00:19:53.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10693805200000001, 2024-09-18T00:19:53.878 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.878 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.878 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:53.878 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.878 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.878 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.878 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24529 2024-09-18T00:19:53.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.879 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:53.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:53.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636772+0000", 2024-09-18T00:19:53.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1100000000000001e-06 2024-09-18T00:19:53.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636783+0000", 2024-09-18T00:19:53.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1005999999999999e-05 2024-09-18T00:19:53.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636814+0000", 2024-09-18T00:19:53.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1047000000000003e-05 2024-09-18T00:19:53.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636829+0000", 2024-09-18T00:19:53.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4363e-05 2024-09-18T00:19:53.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636980+0000", 2024-09-18T00:19:53.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015181099999999999 2024-09-18T00:19:53.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.637237+0000", 2024-09-18T00:19:53.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000256352 2024-09-18T00:19:53.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:53.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743536+0000", 2024-09-18T00:19:53.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.106298825 2024-09-18T00:19:53.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:53.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743599+0000", 2024-09-18T00:19:53.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3449999999999997e-05 2024-09-18T00:19:53.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743708+0000", 2024-09-18T00:19:53.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109088 2024-09-18T00:19:53.883 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.883 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.883 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.884 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24805 3.16 3:6de5e15d:::rbd_data.112a990280ef.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:53.884 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:53.884 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.598333359999998, 2024-09-18T00:19:53.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14775107700000001, 2024-09-18T00:19:53.884 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.884 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.884 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:53.884 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.884 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.884 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.884 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24805 2024-09-18T00:19:53.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.885 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:53.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:53.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222574+0000", 2024-09-18T00:19:53.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0210000000000001e-06 2024-09-18T00:19:53.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222581+0000", 2024-09-18T00:19:53.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.144e-06 2024-09-18T00:19:53.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222595+0000", 2024-09-18T00:19:53.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4567e-05 2024-09-18T00:19:53.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222603+0000", 2024-09-18T00:19:53.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6529999999999994e-06 2024-09-18T00:19:53.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222687+0000", 2024-09-18T00:19:53.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4587999999999997e-05 2024-09-18T00:19:53.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222828+0000", 2024-09-18T00:19:53.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140178 2024-09-18T00:19:53.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:53.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370231+0000", 2024-09-18T00:19:53.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14740378700000001 2024-09-18T00:19:53.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:53.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370269+0000", 2024-09-18T00:19:53.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7886999999999998e-05 2024-09-18T00:19:53.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370324+0000", 2024-09-18T00:19:53.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5251999999999999e-05 2024-09-18T00:19:53.889 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.889 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.889 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.889 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24852 3.13 3:cc64f0fd:::rbd_data.112a990280ef.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:53.889 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:53.889 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.36559389, 2024-09-18T00:19:53.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.066392943999999995, 2024-09-18T00:19:53.890 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.890 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.890 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:53.890 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.890 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.890 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.890 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24852 2024-09-18T00:19:53.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.890 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:53.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:53.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455315+0000", 2024-09-18T00:19:53.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.705e-06 2024-09-18T00:19:53.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455324+0000", 2024-09-18T00:19:53.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8120000000000002e-06 2024-09-18T00:19:53.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455349+0000", 2024-09-18T00:19:53.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4250999999999999e-05 2024-09-18T00:19:53.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455361+0000", 2024-09-18T00:19:53.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2805e-05 2024-09-18T00:19:53.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455425+0000", 2024-09-18T00:19:53.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3981999999999998e-05 2024-09-18T00:19:53.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455480+0000", 2024-09-18T00:19:53.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4792e-05 2024-09-18T00:19:53.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:53.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.460671+0000", 2024-09-18T00:19:53.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051907300000000002 2024-09-18T00:19:53.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:53.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521355+0000", 2024-09-18T00:19:53.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060684208000000003 2024-09-18T00:19:53.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:53.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521493+0000", 2024-09-18T00:19:53.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013803200000000001 2024-09-18T00:19:53.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:53.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521590+0000", 2024-09-18T00:19:53.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6339e-05 2024-09-18T00:19:53.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521706+0000", 2024-09-18T00:19:53.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116288 2024-09-18T00:19:53.896 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.896 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.896 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.897 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:25269 3.f 3:f6adfd48:::rbd_data.112aa34d8ec3.0000000000000145:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T00:19:53.897 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:53.897 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.216003929000003, 2024-09-18T00:19:53.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.075209085999999994, 2024-09-18T00:19:53.897 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.897 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.897 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:53.897 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.897 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.897 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.897 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 25269 2024-09-18T00:19:53.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.897 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:53.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:53.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604904+0000", 2024-09-18T00:19:53.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:19:53.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604910+0000", 2024-09-18T00:19:53.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.964e-06 2024-09-18T00:19:53.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604925+0000", 2024-09-18T00:19:53.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4894e-05 2024-09-18T00:19:53.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604932+0000", 2024-09-18T00:19:53.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-18T00:19:53.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604990+0000", 2024-09-18T00:19:53.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8143000000000002e-05 2024-09-18T00:19:53.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.605104+0000", 2024-09-18T00:19:53.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011404399999999999 2024-09-18T00:19:53.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:53.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.608755+0000", 2024-09-18T00:19:53.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003651081 2024-09-18T00:19:53.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:53.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679920+0000", 2024-09-18T00:19:53.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.071164543999999996 2024-09-18T00:19:53.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:53.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679978+0000", 2024-09-18T00:19:53.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8585000000000002e-05 2024-09-18T00:19:53.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:53.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680036+0000", 2024-09-18T00:19:53.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7978e-05 2024-09-18T00:19:53.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680112+0000", 2024-09-18T00:19:53.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5527999999999999e-05 2024-09-18T00:19:53.903 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.903 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.904 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.904 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:25269 3.f e72/32 ondisk, result = 0)", 2024-09-18T00:19:53.904 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:53.904 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.202793544000002, 2024-09-18T00:19:53.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062014761000000002, 2024-09-18T00:19:53.904 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.904 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.904 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.904 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:53.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:53.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618114+0000", 2024-09-18T00:19:53.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1650000000000001e-06 2024-09-18T00:19:53.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618119+0000", 2024-09-18T00:19:53.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-18T00:19:53.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618134+0000", 2024-09-18T00:19:53.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5770999999999999e-05 2024-09-18T00:19:53.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618143+0000", 2024-09-18T00:19:53.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.551e-06 2024-09-18T00:19:53.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679967+0000", 2024-09-18T00:19:53.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061823937000000002 2024-09-18T00:19:53.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679975+0000", 2024-09-18T00:19:53.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7340000000000008e-06 2024-09-18T00:19:53.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680128+0000", 2024-09-18T00:19:53.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015338299999999999 2024-09-18T00:19:53.908 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.908 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.908 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.908 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48484 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 15704064~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:53.908 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:19:53.908 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.453591435, 2024-09-18T00:19:53.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058450876999999998, 2024-09-18T00:19:53.908 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.908 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48484 2024-09-18T00:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:19:53.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370429+0000", 2024-09-18T00:19:53.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031138530000000002 2024-09-18T00:19:53.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370435+0000", 2024-09-18T00:19:53.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6269999999999997e-06 2024-09-18T00:19:53.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370455+0000", 2024-09-18T00:19:53.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0426999999999999e-05 2024-09-18T00:19:53.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370465+0000", 2024-09-18T00:19:53.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0118999999999999e-05 2024-09-18T00:19:53.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425525+0000", 2024-09-18T00:19:53.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055059943 2024-09-18T00:19:53.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425645+0000", 2024-09-18T00:19:53.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119392 2024-09-18T00:19:53.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425766+0000", 2024-09-18T00:19:53.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012151599999999999 2024-09-18T00:19:53.912 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.912 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.912 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.913 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48503 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16326656~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:53.913 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:53.913 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.449711034, 2024-09-18T00:19:53.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058310395000000001, 2024-09-18T00:19:53.913 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.913 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.913 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.913 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.913 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.913 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.913 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48503 2024-09-18T00:19:53.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.913 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:53.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:53.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:53.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0100000000000004e-07 2024-09-18T00:19:53.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371199+0000", 2024-09-18T00:19:53.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8899999999999999e-06 2024-09-18T00:19:53.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371211+0000", 2024-09-18T00:19:53.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1707e-05 2024-09-18T00:19:53.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371218+0000", 2024-09-18T00:19:53.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.131e-06 2024-09-18T00:19:53.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429426+0000", 2024-09-18T00:19:53.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058207771999999998 2024-09-18T00:19:53.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429467+0000", 2024-09-18T00:19:53.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1532999999999997e-05 2024-09-18T00:19:53.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429506+0000", 2024-09-18T00:19:53.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8661000000000001e-05 2024-09-18T00:19:53.917 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.917 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.918 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.918 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48504 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16359424~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:53.918 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:53.918 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.449679215, 2024-09-18T00:19:53.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058412775, 2024-09-18T00:19:53.918 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.918 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.918 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.918 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.918 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.918 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.918 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48504 2024-09-18T00:19:53.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.919 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:53.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:53.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:53.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6899999999999996e-07 2024-09-18T00:19:53.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371231+0000", 2024-09-18T00:19:53.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6709999999999999e-06 2024-09-18T00:19:53.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371240+0000", 2024-09-18T00:19:53.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7309999999999999e-06 2024-09-18T00:19:53.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371249+0000", 2024-09-18T00:19:53.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4280000000000002e-06 2024-09-18T00:19:53.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429533+0000", 2024-09-18T00:19:53.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058283852999999997 2024-09-18T00:19:53.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429590+0000", 2024-09-18T00:19:53.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6824999999999999e-05 2024-09-18T00:19:53.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429640+0000", 2024-09-18T00:19:53.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0498000000000003e-05 2024-09-18T00:19:53.922 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.923 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.923 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.923 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48505 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16392192~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:53.923 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:53.923 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.449646242, 2024-09-18T00:19:53.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058482777999999999, 2024-09-18T00:19:53.923 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.923 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.923 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.924 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.924 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.924 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.924 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48505 2024-09-18T00:19:53.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.924 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:53.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:53.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:53.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7800000000000001e-07 2024-09-18T00:19:53.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371264+0000", 2024-09-18T00:19:53.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-18T00:19:53.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371274+0000", 2024-09-18T00:19:53.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4690000000000003e-06 2024-09-18T00:19:53.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371281+0000", 2024-09-18T00:19:53.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4120000000000004e-06 2024-09-18T00:19:53.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429665+0000", 2024-09-18T00:19:53.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058384108999999997 2024-09-18T00:19:53.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429705+0000", 2024-09-18T00:19:53.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0299000000000002e-05 2024-09-18T00:19:53.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429743+0000", 2024-09-18T00:19:53.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7951000000000003e-05 2024-09-18T00:19:53.928 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.928 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.928 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.928 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48506 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16424960~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:53.929 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:19:53.929 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.449617029999999, 2024-09-18T00:19:53.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058604485999999997, 2024-09-18T00:19:53.929 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.929 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.929 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.929 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.929 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.929 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.929 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48506 2024-09-18T00:19:53.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.930 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:19:53.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:19:53.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371355+0000", 2024-09-18T00:19:53.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5549000000000005e-05 2024-09-18T00:19:53.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371359+0000", 2024-09-18T00:19:53.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1049999999999997e-06 2024-09-18T00:19:53.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371380+0000", 2024-09-18T00:19:53.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0205000000000001e-05 2024-09-18T00:19:53.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371388+0000", 2024-09-18T00:19:53.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3229999999999998e-06 2024-09-18T00:19:53.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429771+0000", 2024-09-18T00:19:53.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058382878999999999 2024-09-18T00:19:53.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429842+0000", 2024-09-18T00:19:53.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1110000000000002e-05 2024-09-18T00:19:53.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429894+0000", 2024-09-18T00:19:53.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2315000000000003e-05 2024-09-18T00:19:53.933 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.933 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.933 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.933 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48507 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16457728~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:53.933 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:19:53.934 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.449505670000001, 2024-09-18T00:19:53.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058597275999999997, 2024-09-18T00:19:53.934 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.934 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.934 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.934 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.934 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.934 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.934 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48507 2024-09-18T00:19:53.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.934 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:19:53.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:19:53.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374394+0000", 2024-09-18T00:19:53.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029924449999999998 2024-09-18T00:19:53.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374400+0000", 2024-09-18T00:19:53.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0229999999999998e-06 2024-09-18T00:19:53.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374419+0000", 2024-09-18T00:19:53.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9488e-05 2024-09-18T00:19:53.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374432+0000", 2024-09-18T00:19:53.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2492e-05 2024-09-18T00:19:53.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429917+0000", 2024-09-18T00:19:53.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055485087000000002 2024-09-18T00:19:53.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429958+0000", 2024-09-18T00:19:53.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1258999999999999e-05 2024-09-18T00:19:53.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429998+0000", 2024-09-18T00:19:53.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0482000000000003e-05 2024-09-18T00:19:53.938 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.938 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.938 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.938 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:58332 3.15 3:ac910418:::rbd_data.112a17f95ba0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T00:19:53.938 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:19:53.938 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.763001639000001, 2024-09-18T00:19:53.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.071238735999999997, 2024-09-18T00:19:53.939 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.939 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.939 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:53.939 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:53.939 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:53.939 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:53.939 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 58332 2024-09-18T00:19:53.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.939 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:19:53.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:19:53.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057907+0000", 2024-09-18T00:19:53.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1859999999999999e-06 2024-09-18T00:19:53.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057920+0000", 2024-09-18T00:19:53.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3013000000000001e-05 2024-09-18T00:19:53.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057949+0000", 2024-09-18T00:19:53.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8745999999999999e-05 2024-09-18T00:19:53.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057966+0000", 2024-09-18T00:19:53.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6477000000000002e-05 2024-09-18T00:19:53.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.058035+0000", 2024-09-18T00:19:53.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9560000000000005e-05 2024-09-18T00:19:53.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.058133+0000", 2024-09-18T00:19:53.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8051e-05 2024-09-18T00:19:53.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:19:53.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.061809+0000", 2024-09-18T00:19:53.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003675453 2024-09-18T00:19:53.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:53.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128667+0000", 2024-09-18T00:19:53.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.066857960999999994 2024-09-18T00:19:53.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:53.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128952+0000", 2024-09-18T00:19:53.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028583100000000001 2024-09-18T00:19:53.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:53.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129060+0000", 2024-09-18T00:19:53.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010806000000000001 2024-09-18T00:19:53.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129144+0000", 2024-09-18T00:19:53.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3398000000000003e-05 2024-09-18T00:19:53.946 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.946 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.946 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.946 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:58332 3.15 e96/32 ondisk, result = 0)", 2024-09-18T00:19:53.946 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:19:53.946 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.750808810000001, 2024-09-18T00:19:53.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059068112999999998, 2024-09-18T00:19:53.946 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:53.946 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:53.947 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:53.947 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:53.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:53.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:19:53.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:53.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:19:53.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:53.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:53.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070099+0000", 2024-09-18T00:19:53.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9899999999999999e-07 2024-09-18T00:19:53.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:53.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070103+0000", 2024-09-18T00:19:53.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6880000000000001e-06 2024-09-18T00:19:53.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:53.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070115+0000", 2024-09-18T00:19:53.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2088e-05 2024-09-18T00:19:53.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:53.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070122+0000", 2024-09-18T00:19:53.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3900000000000004e-06 2024-09-18T00:19:53.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:53.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128758+0000", 2024-09-18T00:19:53.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058635541999999999 2024-09-18T00:19:53.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:53.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128774+0000", 2024-09-18T00:19:53.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6858999999999999e-05 2024-09-18T00:19:53.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:53.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:53.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:53.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129166+0000", 2024-09-18T00:19:53.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039164700000000001 2024-09-18T00:19:53.950 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.950 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.950 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.950 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:53.950 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:53.950 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:53.952 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:19:53.952 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:53.949+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:54.053 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:54.048+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:54.128 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:54.128 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:54.128 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:54.128 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:54.142 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:19:54.153 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:54.152+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:54.255 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:19:54.252+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:19:54.263 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:54.264 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:54.264 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:54.264 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:54.264 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:54.281 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:19:54.356 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:54.353+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:54.404 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:54.405 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:54.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:54.405 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:54.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.405 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34543 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:54.405 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:19:54.405 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 40.109969018999998, 2024-09-18T00:19:54.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023207593, 2024-09-18T00:19:54.405 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.406 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.406 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:54.406 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.406 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.406 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.406 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34543 2024-09-18T00:19:54.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.406 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.406 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:19:54.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:19:54.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290783+0000", 2024-09-18T00:19:54.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5240000000000001e-06 2024-09-18T00:19:54.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290799+0000", 2024-09-18T00:19:54.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:19:54.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290840+0000", 2024-09-18T00:19:54.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0926000000000001e-05 2024-09-18T00:19:54.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290868+0000", 2024-09-18T00:19:54.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8163000000000001e-05 2024-09-18T00:19:54.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291294+0000", 2024-09-18T00:19:54.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00042588700000000001 2024-09-18T00:19:54.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:19:54.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:19:54.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.986e-05 2024-09-18T00:19:54.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316617+0000", 2024-09-18T00:19:54.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025303583000000001 2024-09-18T00:19:54.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:54.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316637+0000", 2024-09-18T00:19:54.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9388000000000001e-05 2024-09-18T00:19:54.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313933+0000", 2024-09-18T00:19:54.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99729681000000003 2024-09-18T00:19:54.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313987+0000", 2024-09-18T00:19:54.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4069e-05 2024-09-18T00:19:54.411 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.411 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.411 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.411 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34544 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:54.411 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:19:54.411 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 40.109820663999997, 2024-09-18T00:19:54.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0232412120000001, 2024-09-18T00:19:54.411 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.411 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.411 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:54.411 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.412 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.412 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.412 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34544 2024-09-18T00:19:54.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.412 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:19:54.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:19:54.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290930+0000", 2024-09-18T00:19:54.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5030000000000001e-06 2024-09-18T00:19:54.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290940+0000", 2024-09-18T00:19:54.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9000000000000001e-06 2024-09-18T00:19:54.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290964+0000", 2024-09-18T00:19:54.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4273e-05 2024-09-18T00:19:54.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290982+0000", 2024-09-18T00:19:54.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8216e-05 2024-09-18T00:19:54.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291433+0000", 2024-09-18T00:19:54.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000451278 2024-09-18T00:19:54.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:54.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291446+0000", 2024-09-18T00:19:54.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3039e-05 2024-09-18T00:19:54.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316687+0000", 2024-09-18T00:19:54.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025240069 2024-09-18T00:19:54.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:54.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316701+0000", 2024-09-18T00:19:54.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4100000000000001e-05 2024-09-18T00:19:54.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314142+0000", 2024-09-18T00:19:54.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99744119799999997 2024-09-18T00:19:54.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314169+0000", 2024-09-18T00:19:54.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7636000000000001e-05 2024-09-18T00:19:54.416 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.416 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.416 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.417 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34545 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:54.417 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:19:54.417 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 40.109742142000002, 2024-09-18T00:19:54.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023257802, 2024-09-18T00:19:54.417 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.417 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.417 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:54.417 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.417 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.417 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.417 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34545 2024-09-18T00:19:54.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.417 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:19:54.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:19:54.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291008+0000", 2024-09-18T00:19:54.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.237e-06 2024-09-18T00:19:54.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291015+0000", 2024-09-18T00:19:54.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2030000000000003e-06 2024-09-18T00:19:54.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291035+0000", 2024-09-18T00:19:54.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.978e-05 2024-09-18T00:19:54.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291049+0000", 2024-09-18T00:19:54.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4429e-05 2024-09-18T00:19:54.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.421 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291630+0000", 2024-09-18T00:19:54.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00058103199999999999 2024-09-18T00:19:54.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.421 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:54.421 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291644+0000", 2024-09-18T00:19:54.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.342e-05 2024-09-18T00:19:54.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.421 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.421 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316753+0000", 2024-09-18T00:19:54.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025108703999999999 2024-09-18T00:19:54.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:54.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316766+0000", 2024-09-18T00:19:54.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3237e-05 2024-09-18T00:19:54.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314238+0000", 2024-09-18T00:19:54.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99747232699999999 2024-09-18T00:19:54.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314265+0000", 2024-09-18T00:19:54.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6432999999999999e-05 2024-09-18T00:19:54.423 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.423 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.423 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.423 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34546 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:54.423 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:19:54.423 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 40.109683803000003, 2024-09-18T00:19:54.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023324535, 2024-09-18T00:19:54.424 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.424 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.424 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:54.424 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.424 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.425 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.425 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34546 2024-09-18T00:19:54.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.425 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.425 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.425 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:19:54.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.425 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:19:54.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291066+0000", 2024-09-18T00:19:54.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.569999999999999e-07 2024-09-18T00:19:54.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291073+0000", 2024-09-18T00:19:54.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.063e-06 2024-09-18T00:19:54.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291090+0000", 2024-09-18T00:19:54.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6977e-05 2024-09-18T00:19:54.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291105+0000", 2024-09-18T00:19:54.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5166e-05 2024-09-18T00:19:54.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291771+0000", 2024-09-18T00:19:54.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00066599699999999999 2024-09-18T00:19:54.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:54.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291785+0000", 2024-09-18T00:19:54.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.327e-05 2024-09-18T00:19:54.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316826+0000", 2024-09-18T00:19:54.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025041955000000001 2024-09-18T00:19:54.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:54.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316840+0000", 2024-09-18T00:19:54.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3256e-05 2024-09-18T00:19:54.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314335+0000", 2024-09-18T00:19:54.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99749496100000001 2024-09-18T00:19:54.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.430 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.430 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314390+0000", 2024-09-18T00:19:54.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4932999999999997e-05 2024-09-18T00:19:54.430 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.430 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.430 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.431 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34547 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:54.431 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:19:54.431 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 40.109624504000003, 2024-09-18T00:19:54.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.028510467, 2024-09-18T00:19:54.431 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.431 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.431 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:54.431 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.431 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.432 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.432 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34547 2024-09-18T00:19:54.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.432 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:19:54.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:19:54.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291126+0000", 2024-09-18T00:19:54.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2699999999999999e-06 2024-09-18T00:19:54.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291150+0000", 2024-09-18T00:19:54.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4669999999999999e-05 2024-09-18T00:19:54.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291171+0000", 2024-09-18T00:19:54.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0706000000000001e-05 2024-09-18T00:19:54.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291186+0000", 2024-09-18T00:19:54.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5126999999999999e-05 2024-09-18T00:19:54.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291885+0000", 2024-09-18T00:19:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00069850299999999995 2024-09-18T00:19:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291897+0000", 2024-09-18T00:19:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-05 2024-09-18T00:19:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316887+0000", 2024-09-18T00:19:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024990155999999999 2024-09-18T00:19:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316900+0000", 2024-09-18T00:19:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.295e-05 2024-09-18T00:19:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319594+0000", 2024-09-18T00:19:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0026940719999999 2024-09-18T00:19:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319635+0000", 2024-09-18T00:19:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0583000000000001e-05 2024-09-18T00:19:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34548 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:19:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 40.109477046000002, 2024-09-18T00:19:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0285287400000001, 2024-09-18T00:19:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34548 2024-09-18T00:19:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:19:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:19:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291274+0000", 2024-09-18T00:19:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.12e-06 2024-09-18T00:19:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291288+0000", 2024-09-18T00:19:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4367e-05 2024-09-18T00:19:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:19:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5698999999999999e-05 2024-09-18T00:19:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291333+0000", 2024-09-18T00:19:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8783e-05 2024-09-18T00:19:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291987+0000", 2024-09-18T00:19:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00065417800000000005 2024-09-18T00:19:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.292000+0000", 2024-09-18T00:19:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2652e-05 2024-09-18T00:19:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316946+0000", 2024-09-18T00:19:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024946696000000001 2024-09-18T00:19:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316956+0000", 2024-09-18T00:19:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2760000000000006e-06 2024-09-18T00:19:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319752+0000", 2024-09-18T00:19:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002796083 2024-09-18T00:19:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319801+0000", 2024-09-18T00:19:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8885999999999999e-05 2024-09-18T00:19:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41950 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:19:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.706824337, 2024-09-18T00:19:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053368486, 2024-09-18T00:19:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41950 2024-09-18T00:19:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:19:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:19:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693926+0000", 2024-09-18T00:19:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4900000000000004e-07 2024-09-18T00:19:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693930+0000", 2024-09-18T00:19:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0130000000000004e-06 2024-09-18T00:19:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693940+0000", 2024-09-18T00:19:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0618000000000001e-05 2024-09-18T00:19:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693948+0000", 2024-09-18T00:19:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3140000000000004e-06 2024-09-18T00:19:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747133+0000", 2024-09-18T00:19:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053184689 2024-09-18T00:19:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747203+0000", 2024-09-18T00:19:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0339000000000005e-05 2024-09-18T00:19:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747293+0000", 2024-09-18T00:19:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9563999999999998e-05 2024-09-18T00:19:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41951 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:19:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.706789569999998, 2024-09-18T00:19:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053527695, 2024-09-18T00:19:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41951 2024-09-18T00:19:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:19:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:19:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693960+0000", 2024-09-18T00:19:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.16e-07 2024-09-18T00:19:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693971+0000", 2024-09-18T00:19:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1204e-05 2024-09-18T00:19:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693981+0000", 2024-09-18T00:19:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-18T00:19:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693989+0000", 2024-09-18T00:19:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0180000000000001e-06 2024-09-18T00:19:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747328+0000", 2024-09-18T00:19:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053338508 2024-09-18T00:19:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747400+0000", 2024-09-18T00:19:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2750999999999995e-05 2024-09-18T00:19:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747487+0000", 2024-09-18T00:19:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6527999999999996e-05 2024-09-18T00:19:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41952 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:19:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.706749162999998, 2024-09-18T00:19:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053728013999999998, 2024-09-18T00:19:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41952 2024-09-18T00:19:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:19:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:19:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694001+0000", 2024-09-18T00:19:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3799999999999996e-07 2024-09-18T00:19:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694004+0000", 2024-09-18T00:19:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.965e-06 2024-09-18T00:19:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694013+0000", 2024-09-18T00:19:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1689999999999998e-06 2024-09-18T00:19:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694022+0000", 2024-09-18T00:19:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7560000000000005e-06 2024-09-18T00:19:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747531+0000", 2024-09-18T00:19:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053508999000000002 2024-09-18T00:19:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747610+0000", 2024-09-18T00:19:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9212e-05 2024-09-18T00:19:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747728+0000", 2024-09-18T00:19:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011797499999999999 2024-09-18T00:19:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41953 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:19:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.706715533999997, 2024-09-18T00:19:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053936783000000002, 2024-09-18T00:19:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41953 2024-09-18T00:19:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:19:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:19:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694034+0000", 2024-09-18T00:19:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5199999999999996e-07 2024-09-18T00:19:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694037+0000", 2024-09-18T00:19:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.745e-06 2024-09-18T00:19:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694046+0000", 2024-09-18T00:19:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2490000000000007e-06 2024-09-18T00:19:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694054+0000", 2024-09-18T00:19:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1179999999999992e-06 2024-09-18T00:19:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747766+0000", 2024-09-18T00:19:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053711362999999998 2024-09-18T00:19:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747856+0000", 2024-09-18T00:19:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0779000000000006e-05 2024-09-18T00:19:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747970+0000", 2024-09-18T00:19:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011377700000000001 2024-09-18T00:19:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41954 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.706685381, 2024-09-18T00:19:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054104898999999998, 2024-09-18T00:19:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41954 2024-09-18T00:19:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4000000000000002e-07 2024-09-18T00:19:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694067+0000", 2024-09-18T00:19:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7329999999999999e-06 2024-09-18T00:19:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694076+0000", 2024-09-18T00:19:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2019999999999993e-06 2024-09-18T00:19:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694083+0000", 2024-09-18T00:19:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0029999999999997e-06 2024-09-18T00:19:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748003+0000", 2024-09-18T00:19:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053920463000000002 2024-09-18T00:19:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748077+0000", 2024-09-18T00:19:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4041000000000005e-05 2024-09-18T00:19:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748168+0000", 2024-09-18T00:19:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0916999999999997e-05 2024-09-18T00:19:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41955 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 231424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:19:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.706656543000001, 2024-09-18T00:19:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054272659000000001, 2024-09-18T00:19:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41955 2024-09-18T00:19:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:19:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:19:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694093+0000", 2024-09-18T00:19:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:19:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694109+0000", 2024-09-18T00:19:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6061000000000001e-05 2024-09-18T00:19:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694121+0000", 2024-09-18T00:19:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.216e-05 2024-09-18T00:19:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694129+0000", 2024-09-18T00:19:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0029999999999994e-06 2024-09-18T00:19:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748202+0000", 2024-09-18T00:19:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054072571 2024-09-18T00:19:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748274+0000", 2024-09-18T00:19:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2117000000000001e-05 2024-09-18T00:19:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748365+0000", 2024-09-18T00:19:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1139999999999998e-05 2024-09-18T00:19:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41956 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.706608334999999, 2024-09-18T00:19:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054420496999999998, 2024-09-18T00:19:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41956 2024-09-18T00:19:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8699999999999995e-07 2024-09-18T00:19:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694144+0000", 2024-09-18T00:19:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3510000000000002e-06 2024-09-18T00:19:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694154+0000", 2024-09-18T00:19:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0769999999999997e-06 2024-09-18T00:19:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694163+0000", 2024-09-18T00:19:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3300000000000005e-06 2024-09-18T00:19:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748397+0000", 2024-09-18T00:19:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054234497999999999 2024-09-18T00:19:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748471+0000", 2024-09-18T00:19:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3441000000000004e-05 2024-09-18T00:19:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748561+0000", 2024-09-18T00:19:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0212999999999998e-05 2024-09-18T00:19:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41957 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:19:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.706577082999999, 2024-09-18T00:19:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054590310000000003, 2024-09-18T00:19:54.484 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.484 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41957 2024-09-18T00:19:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:19:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:19:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694173+0000", 2024-09-18T00:19:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2500000000000005e-07 2024-09-18T00:19:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694175+0000", 2024-09-18T00:19:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.655e-06 2024-09-18T00:19:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694184+0000", 2024-09-18T00:19:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7630000000000006e-06 2024-09-18T00:19:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694191+0000", 2024-09-18T00:19:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8870000000000003e-06 2024-09-18T00:19:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748593+0000", 2024-09-18T00:19:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054402525 2024-09-18T00:19:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748669+0000", 2024-09-18T00:19:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5532999999999997e-05 2024-09-18T00:19:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748762+0000", 2024-09-18T00:19:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3221999999999996e-05 2024-09-18T00:19:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42392 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:19:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.693547643999999, 2024-09-18T00:19:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053378162, 2024-09-18T00:19:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42392 2024-09-18T00:19:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:19:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:19:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707430+0000", 2024-09-18T00:19:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022838599999999999 2024-09-18T00:19:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707434+0000", 2024-09-18T00:19:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5220000000000001e-06 2024-09-18T00:19:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707467+0000", 2024-09-18T00:19:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2570000000000002e-05 2024-09-18T00:19:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707475+0000", 2024-09-18T00:19:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:19:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760446+0000", 2024-09-18T00:19:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052971635000000003 2024-09-18T00:19:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760512+0000", 2024-09-18T00:19:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5679999999999995e-05 2024-09-18T00:19:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760579+0000", 2024-09-18T00:19:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7296000000000001e-05 2024-09-18T00:19:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42393 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 358400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:19:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.693258692999997, 2024-09-18T00:19:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053269618999999997, 2024-09-18T00:19:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42393 2024-09-18T00:19:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:19:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:19:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707491+0000", 2024-09-18T00:19:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1399999999999997e-07 2024-09-18T00:19:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707503+0000", 2024-09-18T00:19:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1912e-05 2024-09-18T00:19:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707512+0000", 2024-09-18T00:19:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1030000000000008e-06 2024-09-18T00:19:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707520+0000", 2024-09-18T00:19:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1249999999999993e-06 2024-09-18T00:19:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760605+0000", 2024-09-18T00:19:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053085495000000003 2024-09-18T00:19:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760680+0000", 2024-09-18T00:19:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5024000000000005e-05 2024-09-18T00:19:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760760+0000", 2024-09-18T00:19:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9345999999999996e-05 2024-09-18T00:19:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42394 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:19:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.693218705, 2024-09-18T00:19:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053403144999999999, 2024-09-18T00:19:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42394 2024-09-18T00:19:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:19:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:19:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711550+0000", 2024-09-18T00:19:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040200189999999997 2024-09-18T00:19:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711556+0000", 2024-09-18T00:19:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2339999999999999e-06 2024-09-18T00:19:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711569+0000", 2024-09-18T00:19:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2337999999999999e-05 2024-09-18T00:19:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711577+0000", 2024-09-18T00:19:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5660000000000007e-06 2024-09-18T00:19:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760790+0000", 2024-09-18T00:19:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049212196999999999 2024-09-18T00:19:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760857+0000", 2024-09-18T00:19:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7756000000000007e-05 2024-09-18T00:19:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760933+0000", 2024-09-18T00:19:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6035e-05 2024-09-18T00:19:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:55489 3.5 3:a5f1e468:::rbd_data.112a859a01c9.000000000000000c:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T00:19:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:19:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.769364579000001, 2024-09-18T00:19:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.05748731, 2024-09-18T00:19:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 55489 2024-09-18T00:19:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:19:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:19:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631388+0000", 2024-09-18T00:19:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6169999999999998e-06 2024-09-18T00:19:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631405+0000", 2024-09-18T00:19:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6643e-05 2024-09-18T00:19:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631444+0000", 2024-09-18T00:19:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9226e-05 2024-09-18T00:19:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631465+0000", 2024-09-18T00:19:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0826999999999999e-05 2024-09-18T00:19:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631578+0000", 2024-09-18T00:19:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011291899999999999 2024-09-18T00:19:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631651+0000", 2024-09-18T00:19:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3264999999999997e-05 2024-09-18T00:19:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:19:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.634073+0000", 2024-09-18T00:19:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0024221350000000002 2024-09-18T00:19:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688732+0000", 2024-09-18T00:19:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054658610000000003 2024-09-18T00:19:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688794+0000", 2024-09-18T00:19:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2255999999999995e-05 2024-09-18T00:19:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688834+0000", 2024-09-18T00:19:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9916999999999997e-05 2024-09-18T00:19:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688872+0000", 2024-09-18T00:19:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7895000000000002e-05 2024-09-18T00:19:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:58361 3.1d 3:bac90a3d:::rbd_data.112a17f95ba0.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T00:19:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:19:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.197019085000001, 2024-09-18T00:19:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101552963, 2024-09-18T00:19:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 58361 2024-09-18T00:19:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:19:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:19:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203733+0000", 2024-09-18T00:19:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.01e-06 2024-09-18T00:19:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203745+0000", 2024-09-18T00:19:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2513e-05 2024-09-18T00:19:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203777+0000", 2024-09-18T00:19:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1216000000000001e-05 2024-09-18T00:19:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203792+0000", 2024-09-18T00:19:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586e-05 2024-09-18T00:19:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203911+0000", 2024-09-18T00:19:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119234 2024-09-18T00:19:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.204056+0000", 2024-09-18T00:19:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000144943 2024-09-18T00:19:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.210219+0000", 2024-09-18T00:19:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061630649999999997 2024-09-18T00:19:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305071+0000", 2024-09-18T00:19:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.094852120999999998 2024-09-18T00:19:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305106+0000", 2024-09-18T00:19:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4848999999999998e-05 2024-09-18T00:19:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305208+0000", 2024-09-18T00:19:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101518 2024-09-18T00:19:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305283+0000", 2024-09-18T00:19:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4907999999999995e-05 2024-09-18T00:19:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:58361 3.1d e96/79 ondisk, result = 0)", 2024-09-18T00:19:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:19:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.169536044000001, 2024-09-18T00:19:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.073876681, 2024-09-18T00:19:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:19:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:19:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231215+0000", 2024-09-18T00:19:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9570000000000001e-06 2024-09-18T00:19:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231223+0000", 2024-09-18T00:19:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0900000000000005e-06 2024-09-18T00:19:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231252+0000", 2024-09-18T00:19:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9552000000000001e-05 2024-09-18T00:19:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231267+0000", 2024-09-18T00:19:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4263e-05 2024-09-18T00:19:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.304871+0000", 2024-09-18T00:19:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.073604257000000006 2024-09-18T00:19:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.304889+0000", 2024-09-18T00:19:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.808e-05 2024-09-18T00:19:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305090+0000", 2024-09-18T00:19:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020048200000000001 2024-09-18T00:19:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:54.522 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:54.523 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:19:54.524 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:54.520+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:54.624 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:54.620+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:54.655 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:54.655 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:54.656 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:54.656 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:54.669 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:19:54.723 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:19:54.725 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:54.721+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:54.826 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:54.825+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:54.842 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:54.842 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:54.842 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:54.860 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:19:54.927 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:54.925+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:54.948 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:19:54.979 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:54.979 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:54.979 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:54.979 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [] 2024-09-18T00:19:54.979 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:54.994 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:19:55.037 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:55.036+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:55.102 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:55.102 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:55.119 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:19:55.119 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:19:55.129 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:55.125+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:55.136 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:55.136 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:55.168 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:55.168 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:55.192 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:55.192 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:55.203 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:55.203 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:55.208 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:19:55.225 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:55.225 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:55.229 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:55.228+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:55.257 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:55.257 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:55.277 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:55.278 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:55.330 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:55.329+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:55.336 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:55.336 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:55.350 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:55.416 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is set 2024-09-18T00:19:55.431 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:55.429+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:55.524 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:55.525 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:55.532 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:19:55.528+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:19:55.540 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:19:55.633 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:55.632+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:55.714 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:55.715 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:55.729 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:19:55.773 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:55.772+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:55.835 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:55.833+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:55.897 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:55.897 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:55.911 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:19:55.957 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:55.956+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:56.034 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:56.034 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:56.037 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:56.033+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:56.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:19:56.138 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:56.136+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:56.213 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:56.213 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:56.226 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:19:56.273 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:19:56.272+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:19:56.339 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:56.337+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:56.398 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:56.398 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:56.412 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:19:56.440 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:56.437+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:56.541 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:56.540+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:56.544 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:56.544 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:56.560 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:19:56.642 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:56.641+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:56.697 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:56.697 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:56.712 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:19:56.758 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:56.757+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:56.844 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:56.841+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:56.878 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:56.878 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:56.892 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:19:56.945 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:19:56.940+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:19:56.954 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 7] out_osds: [6, 1] dead_osds: [] live_osds: [0, 2, 3, 4, 6, 7, 5, 1] 2024-09-18T00:19:56.954 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:19:56.955 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-18T00:19:56.955 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.000000 2024-09-18T00:19:56.955 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 0 2024-09-18T00:19:57.045 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:57.041+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:57.058 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:57.058 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:57.073 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:19:57.158 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:57.157+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:57.238 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:57.238 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:57.247 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:57.245+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:57.252 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:19:57.297 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:19:57.348 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:57.345+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:57.419 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:57.420 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:57.435 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:19:57.443 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:19:57.444 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:57.444 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:57.461 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:19:57.462 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:19:57.478 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:57.477+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:57.479 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:57.479 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:57.498 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:57.517 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:57.518 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:57.535 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:57.535 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:57.549 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:57.545+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:57.551 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:57.552 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:57.572 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:57.573 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:57.602 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:57.603 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:57.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:57.603 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:57.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.603 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2229 3.10 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:57.603 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:57.603 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 97.336469244, 2024-09-18T00:19:57.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048730086999999998, 2024-09-18T00:19:57.603 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.603 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.603 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:57.603 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.603 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.603 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.604 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2229 2024-09-18T00:19:57.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.604 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:57.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:19:57.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263362+0000", 2024-09-18T00:19:57.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3869999999999999e-06 2024-09-18T00:19:57.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263381+0000", 2024-09-18T00:19:57.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9153000000000001e-05 2024-09-18T00:19:57.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263418+0000", 2024-09-18T00:19:57.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6346000000000001e-05 2024-09-18T00:19:57.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263435+0000", 2024-09-18T00:19:57.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7309999999999999e-05 2024-09-18T00:19:57.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263589+0000", 2024-09-18T00:19:57.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015451199999999999 2024-09-18T00:19:57.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263902+0000", 2024-09-18T00:19:57.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031291799999999997 2024-09-18T00:19:57.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:57.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.265995+0000", 2024-09-18T00:19:57.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020931809999999999 2024-09-18T00:19:57.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:57.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311788+0000", 2024-09-18T00:19:57.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045792078 2024-09-18T00:19:57.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:57.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311903+0000", 2024-09-18T00:19:57.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011510699999999999 2024-09-18T00:19:57.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:57.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312009+0000", 2024-09-18T00:19:57.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010630599999999999 2024-09-18T00:19:57.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312089+0000", 2024-09-18T00:19:57.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9789000000000005e-05 2024-09-18T00:19:57.608 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.608 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.609 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.609 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2229 3.10 e36/23 ondisk, result = 0)", 2024-09-18T00:19:57.609 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:57.609 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 97.317323584999997, 2024-09-18T00:19:57.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029600955000000002, 2024-09-18T00:19:57.609 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.609 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.609 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:57.609 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:57.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:19:57.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282507+0000", 2024-09-18T00:19:57.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9160000000000001e-06 2024-09-18T00:19:57.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282522+0000", 2024-09-18T00:19:57.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5e-05 2024-09-18T00:19:57.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282558+0000", 2024-09-18T00:19:57.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5694000000000001e-05 2024-09-18T00:19:57.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282580+0000", 2024-09-18T00:19:57.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2323e-05 2024-09-18T00:19:57.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311873+0000", 2024-09-18T00:19:57.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029293013999999999 2024-09-18T00:19:57.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311892+0000", 2024-09-18T00:19:57.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8556e-05 2024-09-18T00:19:57.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312105+0000", 2024-09-18T00:19:57.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021345199999999999 2024-09-18T00:19:57.612 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.613 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.613 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.613 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5714 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:19:57.613 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:57.613 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 92.289621081999996, 2024-09-18T00:19:57.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027448727999999999, 2024-09-18T00:19:57.613 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.613 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.613 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:57.613 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.613 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.613 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.613 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5714 2024-09-18T00:19:57.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.614 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:57.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:19:57.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310211+0000", 2024-09-18T00:19:57.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5639999999999998e-06 2024-09-18T00:19:57.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312678+0000", 2024-09-18T00:19:57.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002466761 2024-09-18T00:19:57.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312746+0000", 2024-09-18T00:19:57.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8046000000000005e-05 2024-09-18T00:19:57.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312774+0000", 2024-09-18T00:19:57.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7813e-05 2024-09-18T00:19:57.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312927+0000", 2024-09-18T00:19:57.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015328499999999999 2024-09-18T00:19:57.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313214+0000", 2024-09-18T00:19:57.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028683099999999998 2024-09-18T00:19:57.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:57.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313708+0000", 2024-09-18T00:19:57.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049379100000000004 2024-09-18T00:19:57.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:57.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336895+0000", 2024-09-18T00:19:57.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023186801 2024-09-18T00:19:57.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:57.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337454+0000", 2024-09-18T00:19:57.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055979399999999996 2024-09-18T00:19:57.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:57.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337571+0000", 2024-09-18T00:19:57.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116821 2024-09-18T00:19:57.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337656+0000", 2024-09-18T00:19:57.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4221000000000006e-05 2024-09-18T00:19:57.618 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.619 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.619 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.619 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6801 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:19:57.619 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:57.619 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.562784633999996, 2024-09-18T00:19:57.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051953602000000002, 2024-09-18T00:19:57.619 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.619 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.619 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:57.619 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.619 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.619 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.620 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6801 2024-09-18T00:19:57.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.620 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:57.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:19:57.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037046+0000", 2024-09-18T00:19:57.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.19e-06 2024-09-18T00:19:57.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037062+0000", 2024-09-18T00:19:57.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586999999999998e-05 2024-09-18T00:19:57.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037096+0000", 2024-09-18T00:19:57.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4112000000000001e-05 2024-09-18T00:19:57.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037120+0000", 2024-09-18T00:19:57.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.355e-05 2024-09-18T00:19:57.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037246+0000", 2024-09-18T00:19:57.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012652200000000001 2024-09-18T00:19:57.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037385+0000", 2024-09-18T00:19:57.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013924599999999999 2024-09-18T00:19:57.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:19:57.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.040881+0000", 2024-09-18T00:19:57.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034954830000000002 2024-09-18T00:19:57.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:57.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088794+0000", 2024-09-18T00:19:57.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047912897000000003 2024-09-18T00:19:57.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:57.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088828+0000", 2024-09-18T00:19:57.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3880000000000001e-05 2024-09-18T00:19:57.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:57.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088916+0000", 2024-09-18T00:19:57.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.865e-05 2024-09-18T00:19:57.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088997+0000", 2024-09-18T00:19:57.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0484999999999999e-05 2024-09-18T00:19:57.624 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.625 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.625 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.625 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:6801 3.10 e42/23 ondisk, result = 0)", 2024-09-18T00:19:57.625 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:57.625 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.548977223999998, 2024-09-18T00:19:57.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037960394000000001, 2024-09-18T00:19:57.625 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.625 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.625 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:57.625 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:57.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:19:57.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050852+0000", 2024-09-18T00:19:57.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:19:57.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050860+0000", 2024-09-18T00:19:57.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7729999999999999e-06 2024-09-18T00:19:57.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050891+0000", 2024-09-18T00:19:57.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0617000000000002e-05 2024-09-18T00:19:57.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050910+0000", 2024-09-18T00:19:57.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.944e-05 2024-09-18T00:19:57.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088766+0000", 2024-09-18T00:19:57.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037855633999999999 2024-09-18T00:19:57.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088784+0000", 2024-09-18T00:19:57.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7767000000000001e-05 2024-09-18T00:19:57.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088811+0000", 2024-09-18T00:19:57.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7382999999999999e-05 2024-09-18T00:19:57.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.628 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.629 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12594 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:57.629 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:57.629 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 81.208200685999998, 2024-09-18T00:19:57.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026492600000000002, 2024-09-18T00:19:57.629 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.629 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.629 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:57.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.629 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12594 2024-09-18T00:19:57.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.629 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:57.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:19:57.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391629+0000", 2024-09-18T00:19:57.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.86e-06 2024-09-18T00:19:57.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391637+0000", 2024-09-18T00:19:57.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3999999999999992e-06 2024-09-18T00:19:57.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391664+0000", 2024-09-18T00:19:57.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6536000000000001e-05 2024-09-18T00:19:57.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391681+0000", 2024-09-18T00:19:57.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6563e-05 2024-09-18T00:19:57.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417791+0000", 2024-09-18T00:19:57.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026110936000000001 2024-09-18T00:19:57.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417933+0000", 2024-09-18T00:19:57.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014112099999999999 2024-09-18T00:19:57.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418120+0000", 2024-09-18T00:19:57.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018718399999999999 2024-09-18T00:19:57.633 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.633 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.633 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.633 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12595 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:57.633 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:57.633 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 81.208124487999996, 2024-09-18T00:19:57.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026819428999999999, 2024-09-18T00:19:57.633 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.633 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.634 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:57.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.634 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12595 2024-09-18T00:19:57.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.634 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:57.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:19:57.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391705+0000", 2024-09-18T00:19:57.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3999999999999999e-06 2024-09-18T00:19:57.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391711+0000", 2024-09-18T00:19:57.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7459999999999997e-06 2024-09-18T00:19:57.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391733+0000", 2024-09-18T00:19:57.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2923000000000001e-05 2024-09-18T00:19:57.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391748+0000", 2024-09-18T00:19:57.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4790000000000001e-05 2024-09-18T00:19:57.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418180+0000", 2024-09-18T00:19:57.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026431888000000001 2024-09-18T00:19:57.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418328+0000", 2024-09-18T00:19:57.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000147725 2024-09-18T00:19:57.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418523+0000", 2024-09-18T00:19:57.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000194957 2024-09-18T00:19:57.637 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.637 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.637 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.638 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12596 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:57.638 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:57.638 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 81.208061520000001, 2024-09-18T00:19:57.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027220296000000001, 2024-09-18T00:19:57.638 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.638 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.638 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:57.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.638 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12596 2024-09-18T00:19:57.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.638 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:57.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:19:57.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391768+0000", 2024-09-18T00:19:57.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.283e-06 2024-09-18T00:19:57.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391773+0000", 2024-09-18T00:19:57.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3970000000000002e-06 2024-09-18T00:19:57.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391795+0000", 2024-09-18T00:19:57.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1835e-05 2024-09-18T00:19:57.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391809+0000", 2024-09-18T00:19:57.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4107000000000001e-05 2024-09-18T00:19:57.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418582+0000", 2024-09-18T00:19:57.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026772918999999999 2024-09-18T00:19:57.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418780+0000", 2024-09-18T00:19:57.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000198042 2024-09-18T00:19:57.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418987+0000", 2024-09-18T00:19:57.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000206713 2024-09-18T00:19:57.642 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.642 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.642 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.642 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12597 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:57.642 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:57.642 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 81.208001002000003, 2024-09-18T00:19:57.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027612714, 2024-09-18T00:19:57.643 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.643 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.643 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:57.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.643 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12597 2024-09-18T00:19:57.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.643 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:57.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:19:57.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391828+0000", 2024-09-18T00:19:57.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.203e-06 2024-09-18T00:19:57.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391834+0000", 2024-09-18T00:19:57.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6529999999999999e-06 2024-09-18T00:19:57.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391856+0000", 2024-09-18T00:19:57.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2195999999999999e-05 2024-09-18T00:19:57.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391870+0000", 2024-09-18T00:19:57.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4063000000000001e-05 2024-09-18T00:19:57.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419049+0000", 2024-09-18T00:19:57.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027178751000000001 2024-09-18T00:19:57.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419239+0000", 2024-09-18T00:19:57.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019039400000000001 2024-09-18T00:19:57.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419440+0000", 2024-09-18T00:19:57.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000200454 2024-09-18T00:19:57.647 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.647 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.647 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.647 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12598 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:57.647 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:57.647 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 81.207937951000005, 2024-09-18T00:19:57.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028269056000000001, 2024-09-18T00:19:57.647 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.647 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.647 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:57.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12598 2024-09-18T00:19:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:19:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391891+0000", 2024-09-18T00:19:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.257e-06 2024-09-18T00:19:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391897+0000", 2024-09-18T00:19:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9560000000000002e-06 2024-09-18T00:19:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391920+0000", 2024-09-18T00:19:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3025999999999999e-05 2024-09-18T00:19:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391935+0000", 2024-09-18T00:19:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4853e-05 2024-09-18T00:19:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419527+0000", 2024-09-18T00:19:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027592378000000001 2024-09-18T00:19:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419681+0000", 2024-09-18T00:19:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015316 2024-09-18T00:19:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420159+0000", 2024-09-18T00:19:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000478426 2024-09-18T00:19:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12599 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 81.207874095999998, 2024-09-18T00:19:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028649857000000001, 2024-09-18T00:19:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12599 2024-09-18T00:19:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:19:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391955+0000", 2024-09-18T00:19:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-06 2024-09-18T00:19:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391961+0000", 2024-09-18T00:19:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7100000000000004e-06 2024-09-18T00:19:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391982+0000", 2024-09-18T00:19:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1492e-05 2024-09-18T00:19:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391996+0000", 2024-09-18T00:19:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4194e-05 2024-09-18T00:19:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420237+0000", 2024-09-18T00:19:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028240818000000001 2024-09-18T00:19:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420430+0000", 2024-09-18T00:19:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019228299999999999 2024-09-18T00:19:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420604+0000", 2024-09-18T00:19:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000174117 2024-09-18T00:19:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12600 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 270336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 81.207813102000003, 2024-09-18T00:19:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028964586, 2024-09-18T00:19:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12600 2024-09-18T00:19:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:19:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392016+0000", 2024-09-18T00:19:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.31e-06 2024-09-18T00:19:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392022+0000", 2024-09-18T00:19:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5600000000000001e-06 2024-09-18T00:19:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392045+0000", 2024-09-18T00:19:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3595999999999999e-05 2024-09-18T00:19:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392060+0000", 2024-09-18T00:19:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4727e-05 2024-09-18T00:19:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420669+0000", 2024-09-18T00:19:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028608855999999998 2024-09-18T00:19:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420813+0000", 2024-09-18T00:19:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014429500000000001 2024-09-18T00:19:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420979+0000", 2024-09-18T00:19:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016624200000000001 2024-09-18T00:19:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12601 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 81.207749196999998, 2024-09-18T00:19:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029292832000000001, 2024-09-18T00:19:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12601 2024-09-18T00:19:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:57.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:19:57.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392080+0000", 2024-09-18T00:19:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:19:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392086+0000", 2024-09-18T00:19:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7570000000000001e-06 2024-09-18T00:19:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392105+0000", 2024-09-18T00:19:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9415999999999998e-05 2024-09-18T00:19:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392132+0000", 2024-09-18T00:19:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7019e-05 2024-09-18T00:19:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421049+0000", 2024-09-18T00:19:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028916659000000001 2024-09-18T00:19:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421204+0000", 2024-09-18T00:19:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155224 2024-09-18T00:19:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421372+0000", 2024-09-18T00:19:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016754699999999999 2024-09-18T00:19:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12602 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 81.207676566000004, 2024-09-18T00:19:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029612833000000002, 2024-09-18T00:19:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12602 2024-09-18T00:19:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:19:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392152+0000", 2024-09-18T00:19:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0470000000000001e-06 2024-09-18T00:19:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392158+0000", 2024-09-18T00:19:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:19:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392181+0000", 2024-09-18T00:19:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2629e-05 2024-09-18T00:19:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392222+0000", 2024-09-18T00:19:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0772000000000002e-05 2024-09-18T00:19:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421441+0000", 2024-09-18T00:19:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029219327 2024-09-18T00:19:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421608+0000", 2024-09-18T00:19:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000166813 2024-09-18T00:19:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421764+0000", 2024-09-18T00:19:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015628500000000001 2024-09-18T00:19:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12603 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 81.207586172000006, 2024-09-18T00:19:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029902524, 2024-09-18T00:19:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12603 2024-09-18T00:19:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:19:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392243+0000", 2024-09-18T00:19:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392260+0000", 2024-09-18T00:19:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6860000000000001e-05 2024-09-18T00:19:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392282+0000", 2024-09-18T00:19:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2713000000000001e-05 2024-09-18T00:19:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392297+0000", 2024-09-18T00:19:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4375e-05 2024-09-18T00:19:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421830+0000", 2024-09-18T00:19:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029532668000000002 2024-09-18T00:19:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421985+0000", 2024-09-18T00:19:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155613 2024-09-18T00:19:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422144+0000", 2024-09-18T00:19:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000159102 2024-09-18T00:19:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12604 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 81.207511120999996, 2024-09-18T00:19:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030228695, 2024-09-18T00:19:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12604 2024-09-18T00:19:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:19:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392318+0000", 2024-09-18T00:19:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:19:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392337+0000", 2024-09-18T00:19:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:19:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392364+0000", 2024-09-18T00:19:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7288999999999999e-05 2024-09-18T00:19:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392379+0000", 2024-09-18T00:19:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:19:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422209+0000", 2024-09-18T00:19:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029829050999999999 2024-09-18T00:19:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422388+0000", 2024-09-18T00:19:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017988699999999999 2024-09-18T00:19:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422545+0000", 2024-09-18T00:19:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015701199999999999 2024-09-18T00:19:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12605 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 81.207426560000002, 2024-09-18T00:19:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030488674, 2024-09-18T00:19:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12605 2024-09-18T00:19:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:19:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392403+0000", 2024-09-18T00:19:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.883e-06 2024-09-18T00:19:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392421+0000", 2024-09-18T00:19:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7526000000000001e-05 2024-09-18T00:19:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392450+0000", 2024-09-18T00:19:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8972e-05 2024-09-18T00:19:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392488+0000", 2024-09-18T00:19:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8522000000000001e-05 2024-09-18T00:19:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422614+0000", 2024-09-18T00:19:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030125945000000001 2024-09-18T00:19:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422752+0000", 2024-09-18T00:19:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000138178 2024-09-18T00:19:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422890+0000", 2024-09-18T00:19:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013764800000000001 2024-09-18T00:19:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12606 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:19:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 81.207317900000007, 2024-09-18T00:19:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033727073000000003, 2024-09-18T00:19:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12606 2024-09-18T00:19:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:19:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392511+0000", 2024-09-18T00:19:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3090000000000001e-06 2024-09-18T00:19:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392517+0000", 2024-09-18T00:19:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.993e-06 2024-09-18T00:19:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392543+0000", 2024-09-18T00:19:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6152999999999998e-05 2024-09-18T00:19:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392558+0000", 2024-09-18T00:19:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4610000000000001e-05 2024-09-18T00:19:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.425959+0000", 2024-09-18T00:19:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033400484000000001 2024-09-18T00:19:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426106+0000", 2024-09-18T00:19:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014779800000000001 2024-09-18T00:19:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426237+0000", 2024-09-18T00:19:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013072600000000001 2024-09-18T00:19:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17562 3.a 3:5578626c:::rbd_data.112a6efb7564.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T00:19:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 67.641852739000001, 2024-09-18T00:19:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052733453999999999, 2024-09-18T00:19:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17562 2024-09-18T00:19:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:19:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957977+0000", 2024-09-18T00:19:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.119e-06 2024-09-18T00:19:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957988+0000", 2024-09-18T00:19:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0557000000000001e-05 2024-09-18T00:19:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958017+0000", 2024-09-18T00:19:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9156e-05 2024-09-18T00:19:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958036+0000", 2024-09-18T00:19:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8919e-05 2024-09-18T00:19:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958144+0000", 2024-09-18T00:19:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107905 2024-09-18T00:19:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958284+0000", 2024-09-18T00:19:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140216 2024-09-18T00:19:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.960572+0000", 2024-09-18T00:19:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022881860000000002 2024-09-18T00:19:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010537+0000", 2024-09-18T00:19:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049964664999999998 2024-09-18T00:19:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010590+0000", 2024-09-18T00:19:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3094999999999999e-05 2024-09-18T00:19:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010642+0000", 2024-09-18T00:19:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2073000000000001e-05 2024-09-18T00:19:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010709+0000", 2024-09-18T00:19:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6563000000000006e-05 2024-09-18T00:19:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17562 3.a e60/23 ondisk, result = 0)", 2024-09-18T00:19:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 67.631649367999998, 2024-09-18T00:19:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042539307999999998, 2024-09-18T00:19:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:19:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968180+0000", 2024-09-18T00:19:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-18T00:19:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968188+0000", 2024-09-18T00:19:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7419999999999997e-06 2024-09-18T00:19:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968217+0000", 2024-09-18T00:19:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8926999999999999e-05 2024-09-18T00:19:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968231+0000", 2024-09-18T00:19:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4168999999999999e-05 2024-09-18T00:19:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010575+0000", 2024-09-18T00:19:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042343923999999998 2024-09-18T00:19:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010584+0000", 2024-09-18T00:19:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-18T00:19:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010718+0000", 2024-09-18T00:19:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013344499999999999 2024-09-18T00:19:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:57.700 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:57.700 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:57.701 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:19:57.702 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:57.696+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:57.714 INFO:teuthology.orchestra.run.smithi097.stderr:set osd.5 primary-affinity to 0 (802) 2024-09-18T00:19:57.802 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:57.797+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:57.867 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:57.867 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:57.867 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:57.868 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:57.882 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:19:57.926 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:57.925+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:58.003 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:58.001+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:58.048 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:58.048 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:58.048 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:58.048 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:58.049 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:58.063 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:19:58.106 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:58.105+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:58.205 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:19:58.204+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:19:58.232 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2228 3.15 3:aa2fc059:::rbd_data.112aa7ba4578.000000000000014a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:19:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 97.965375795, 2024-09-18T00:19:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.068966377999999995, 2024-09-18T00:19:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2228 2024-09-18T00:19:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:19:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263099+0000", 2024-09-18T00:19:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2359999999999999e-06 2024-09-18T00:19:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263115+0000", 2024-09-18T00:19:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5996999999999999e-05 2024-09-18T00:19:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263156+0000", 2024-09-18T00:19:58.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:19:58.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263183+0000", 2024-09-18T00:19:58.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7233000000000002e-05 2024-09-18T00:19:58.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263304+0000", 2024-09-18T00:19:58.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121209 2024-09-18T00:19:58.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263528+0000", 2024-09-18T00:19:58.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022401800000000001 2024-09-18T00:19:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:19:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.266458+0000", 2024-09-18T00:19:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029296980000000001 2024-09-18T00:19:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331628+0000", 2024-09-18T00:19:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.065170045999999995 2024-09-18T00:19:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331819+0000", 2024-09-18T00:19:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000190819 2024-09-18T00:19:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331971+0000", 2024-09-18T00:19:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151972 2024-09-18T00:19:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332063+0000", 2024-09-18T00:19:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2083999999999995e-05 2024-09-18T00:19:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2854 3.f 3:f02b0bca:::rbd_data.112aa7ba4578.0000000000000135:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:19:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.358403304999996, 2024-09-18T00:19:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.082582802999999996, 2024-09-18T00:19:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2854 2024-09-18T00:19:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:19:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870071+0000", 2024-09-18T00:19:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.227e-06 2024-09-18T00:19:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870083+0000", 2024-09-18T00:19:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1909999999999999e-05 2024-09-18T00:19:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870114+0000", 2024-09-18T00:19:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0576e-05 2024-09-18T00:19:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870129+0000", 2024-09-18T00:19:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.485e-05 2024-09-18T00:19:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870256+0000", 2024-09-18T00:19:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012691300000000001 2024-09-18T00:19:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870493+0000", 2024-09-18T00:19:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023737399999999999 2024-09-18T00:19:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.874933+0000", 2024-09-18T00:19:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044398299999999996 2024-09-18T00:19:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952452+0000", 2024-09-18T00:19:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.077518918000000006 2024-09-18T00:19:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952486+0000", 2024-09-18T00:19:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4035999999999997e-05 2024-09-18T00:19:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952575+0000", 2024-09-18T00:19:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.954e-05 2024-09-18T00:19:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952652+0000", 2024-09-18T00:19:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6629000000000001e-05 2024-09-18T00:19:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2854 3.f e38/32 ondisk, result = 0)", 2024-09-18T00:19:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.340872489999995, 2024-09-18T00:19:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064868625999999999, 2024-09-18T00:19:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:19:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887602+0000", 2024-09-18T00:19:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.677e-06 2024-09-18T00:19:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887609+0000", 2024-09-18T00:19:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6729999999999992e-06 2024-09-18T00:19:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887637+0000", 2024-09-18T00:19:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8206000000000001e-05 2024-09-18T00:19:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887651+0000", 2024-09-18T00:19:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3387000000000001e-05 2024-09-18T00:19:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952422+0000", 2024-09-18T00:19:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064771366999999996 2024-09-18T00:19:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952441+0000", 2024-09-18T00:19:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8737e-05 2024-09-18T00:19:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952469+0000", 2024-09-18T00:19:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7579000000000001e-05 2024-09-18T00:19:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15879 3.16 3:692b47b6:::rbd_object_map.112a13ec2d6c.000000000000000b:head [call] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:19:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.573708582999998, 2024-09-18T00:19:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.011184018, 2024-09-18T00:19:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15879 2024-09-18T00:19:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:19:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654768+0000", 2024-09-18T00:19:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9129999999999996e-06 2024-09-18T00:19:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654786+0000", 2024-09-18T00:19:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8695999999999999e-05 2024-09-18T00:19:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654832+0000", 2024-09-18T00:19:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5735000000000002e-05 2024-09-18T00:19:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654860+0000", 2024-09-18T00:19:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7789000000000001e-05 2024-09-18T00:19:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655048+0000", 2024-09-18T00:19:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018781900000000001 2024-09-18T00:19:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:19:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655070+0000", 2024-09-18T00:19:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1916000000000002e-05 2024-09-18T00:19:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663335+0000", 2024-09-18T00:19:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0082651210000000003 2024-09-18T00:19:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663358+0000", 2024-09-18T00:19:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2986e-05 2024-09-18T00:19:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663739+0000", 2024-09-18T00:19:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0003814820000001 2024-09-18T00:19:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663920+0000", 2024-09-18T00:19:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018116900000000001 2024-09-18T00:19:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.665948+0000", 2024-09-18T00:19:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020273919999999998 2024-09-18T00:19:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15881 3.16 3:6cc14920:::rbd_header.112a3285400b:head [watch reconnect cookie 140685950184608 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:19:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.573547587999997, 2024-09-18T00:19:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0138723599999999, 2024-09-18T00:19:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15881 2024-09-18T00:19:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:19:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654926+0000", 2024-09-18T00:19:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6470000000000001e-06 2024-09-18T00:19:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654935+0000", 2024-09-18T00:19:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8859999999999993e-06 2024-09-18T00:19:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654960+0000", 2024-09-18T00:19:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4689999999999999e-05 2024-09-18T00:19:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654976+0000", 2024-09-18T00:19:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6319000000000001e-05 2024-09-18T00:19:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655238+0000", 2024-09-18T00:19:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026145999999999998 2024-09-18T00:19:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655255+0000", 2024-09-18T00:19:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7249999999999999e-05 2024-09-18T00:19:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663416+0000", 2024-09-18T00:19:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081607959999999997 2024-09-18T00:19:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663436+0000", 2024-09-18T00:19:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0384e-05 2024-09-18T00:19:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666184+0000", 2024-09-18T00:19:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002748121 2024-09-18T00:19:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666452+0000", 2024-09-18T00:19:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026801700000000001 2024-09-18T00:19:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.668797+0000", 2024-09-18T00:19:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023447899999999998 2024-09-18T00:19:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17089 3.f 3:f7681386:::rbd_data.112a6efb7564.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:19:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.348620491999995, 2024-09-18T00:19:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.12573889399999999, 2024-09-18T00:19:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17089 2024-09-18T00:19:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:19:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879854+0000", 2024-09-18T00:19:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:19:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879867+0000", 2024-09-18T00:19:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3103e-05 2024-09-18T00:19:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879898+0000", 2024-09-18T00:19:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0103e-05 2024-09-18T00:19:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879912+0000", 2024-09-18T00:19:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4636e-05 2024-09-18T00:19:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879992+0000", 2024-09-18T00:19:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9814000000000005e-05 2024-09-18T00:19:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.880101+0000", 2024-09-18T00:19:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109172 2024-09-18T00:19:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.887269+0000", 2024-09-18T00:19:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071679300000000003 2024-09-18T00:19:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005269+0000", 2024-09-18T00:19:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.117999621 2024-09-18T00:19:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005317+0000", 2024-09-18T00:19:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8279000000000003e-05 2024-09-18T00:19:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005464+0000", 2024-09-18T00:19:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014661200000000001 2024-09-18T00:19:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005591+0000", 2024-09-18T00:19:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000127107 2024-09-18T00:19:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17089 3.f e59/32 ondisk, result = 0)", 2024-09-18T00:19:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.324513627000002, 2024-09-18T00:19:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101334403, 2024-09-18T00:19:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:19:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903960+0000", 2024-09-18T00:19:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.33e-06 2024-09-18T00:19:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903966+0000", 2024-09-18T00:19:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5929999999999996e-06 2024-09-18T00:19:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903986+0000", 2024-09-18T00:19:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.018e-05 2024-09-18T00:19:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903997+0000", 2024-09-18T00:19:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0988999999999999e-05 2024-09-18T00:19:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005171+0000", 2024-09-18T00:19:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101174267 2024-09-18T00:19:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005200+0000", 2024-09-18T00:19:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8540000000000001e-05 2024-09-18T00:19:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005293+0000", 2024-09-18T00:19:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504000000000005e-05 2024-09-18T00:19:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24529 3.16 3:6857ce1a:::rbd_data.112a990280ef.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.591702216000002, 2024-09-18T00:19:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10693805200000001, 2024-09-18T00:19:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24529 2024-09-18T00:19:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:19:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636772+0000", 2024-09-18T00:19:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1100000000000001e-06 2024-09-18T00:19:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636783+0000", 2024-09-18T00:19:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1005999999999999e-05 2024-09-18T00:19:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636814+0000", 2024-09-18T00:19:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1047000000000003e-05 2024-09-18T00:19:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636829+0000", 2024-09-18T00:19:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4363e-05 2024-09-18T00:19:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636980+0000", 2024-09-18T00:19:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015181099999999999 2024-09-18T00:19:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.637237+0000", 2024-09-18T00:19:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000256352 2024-09-18T00:19:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743536+0000", 2024-09-18T00:19:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.106298825 2024-09-18T00:19:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743599+0000", 2024-09-18T00:19:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3449999999999997e-05 2024-09-18T00:19:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743708+0000", 2024-09-18T00:19:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109088 2024-09-18T00:19:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24805 3.16 3:6de5e15d:::rbd_data.112a990280ef.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.005898983999998, 2024-09-18T00:19:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14775107700000001, 2024-09-18T00:19:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24805 2024-09-18T00:19:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:19:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222574+0000", 2024-09-18T00:19:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0210000000000001e-06 2024-09-18T00:19:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222581+0000", 2024-09-18T00:19:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.144e-06 2024-09-18T00:19:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222595+0000", 2024-09-18T00:19:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4567e-05 2024-09-18T00:19:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222603+0000", 2024-09-18T00:19:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6529999999999994e-06 2024-09-18T00:19:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222687+0000", 2024-09-18T00:19:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4587999999999997e-05 2024-09-18T00:19:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222828+0000", 2024-09-18T00:19:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140178 2024-09-18T00:19:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370231+0000", 2024-09-18T00:19:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14740378700000001 2024-09-18T00:19:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370269+0000", 2024-09-18T00:19:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7886999999999998e-05 2024-09-18T00:19:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370324+0000", 2024-09-18T00:19:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5251999999999999e-05 2024-09-18T00:19:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.283 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.283 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24852 3.13 3:cc64f0fd:::rbd_data.112a990280ef.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:19:58.283 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:58.283 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.773159514, 2024-09-18T00:19:58.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.066392943999999995, 2024-09-18T00:19:58.283 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.283 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.283 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:58.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.283 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24852 2024-09-18T00:19:58.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.284 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:58.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:19:58.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455315+0000", 2024-09-18T00:19:58.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.705e-06 2024-09-18T00:19:58.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455324+0000", 2024-09-18T00:19:58.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8120000000000002e-06 2024-09-18T00:19:58.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455349+0000", 2024-09-18T00:19:58.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4250999999999999e-05 2024-09-18T00:19:58.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455361+0000", 2024-09-18T00:19:58.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2805e-05 2024-09-18T00:19:58.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455425+0000", 2024-09-18T00:19:58.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3981999999999998e-05 2024-09-18T00:19:58.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455480+0000", 2024-09-18T00:19:58.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4792e-05 2024-09-18T00:19:58.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:58.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.460671+0000", 2024-09-18T00:19:58.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051907300000000002 2024-09-18T00:19:58.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:58.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521355+0000", 2024-09-18T00:19:58.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060684208000000003 2024-09-18T00:19:58.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:58.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521493+0000", 2024-09-18T00:19:58.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013803200000000001 2024-09-18T00:19:58.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:58.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521590+0000", 2024-09-18T00:19:58.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6339e-05 2024-09-18T00:19:58.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521706+0000", 2024-09-18T00:19:58.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116288 2024-09-18T00:19:58.288 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.288 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.288 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.289 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:25269 3.f 3:f6adfd48:::rbd_data.112aa34d8ec3.0000000000000145:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T00:19:58.289 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:58.289 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.623569553000003, 2024-09-18T00:19:58.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.075209085999999994, 2024-09-18T00:19:58.289 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.289 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.289 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:58.289 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.289 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.289 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.289 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 25269 2024-09-18T00:19:58.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.290 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:58.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:19:58.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604904+0000", 2024-09-18T00:19:58.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:19:58.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604910+0000", 2024-09-18T00:19:58.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.964e-06 2024-09-18T00:19:58.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604925+0000", 2024-09-18T00:19:58.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4894e-05 2024-09-18T00:19:58.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604932+0000", 2024-09-18T00:19:58.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-18T00:19:58.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604990+0000", 2024-09-18T00:19:58.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8143000000000002e-05 2024-09-18T00:19:58.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.605104+0000", 2024-09-18T00:19:58.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011404399999999999 2024-09-18T00:19:58.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:58.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.608755+0000", 2024-09-18T00:19:58.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003651081 2024-09-18T00:19:58.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:58.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679920+0000", 2024-09-18T00:19:58.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.071164543999999996 2024-09-18T00:19:58.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:58.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679978+0000", 2024-09-18T00:19:58.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8585000000000002e-05 2024-09-18T00:19:58.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:58.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680036+0000", 2024-09-18T00:19:58.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7978e-05 2024-09-18T00:19:58.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680112+0000", 2024-09-18T00:19:58.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5527999999999999e-05 2024-09-18T00:19:58.295 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.295 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.295 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.295 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:25269 3.f e72/32 ondisk, result = 0)", 2024-09-18T00:19:58.295 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:58.295 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.610359168000002, 2024-09-18T00:19:58.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062014761000000002, 2024-09-18T00:19:58.295 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.295 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.295 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.295 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:58.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:19:58.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618114+0000", 2024-09-18T00:19:58.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1650000000000001e-06 2024-09-18T00:19:58.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618119+0000", 2024-09-18T00:19:58.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-18T00:19:58.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618134+0000", 2024-09-18T00:19:58.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5770999999999999e-05 2024-09-18T00:19:58.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618143+0000", 2024-09-18T00:19:58.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.551e-06 2024-09-18T00:19:58.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679967+0000", 2024-09-18T00:19:58.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061823937000000002 2024-09-18T00:19:58.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679975+0000", 2024-09-18T00:19:58.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7340000000000008e-06 2024-09-18T00:19:58.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680128+0000", 2024-09-18T00:19:58.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015338299999999999 2024-09-18T00:19:58.299 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.299 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.299 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.299 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48484 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 15704064~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:58.299 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:19:58.299 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.861157059, 2024-09-18T00:19:58.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058450876999999998, 2024-09-18T00:19:58.299 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.299 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.300 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.300 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.300 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.300 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.300 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48484 2024-09-18T00:19:58.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.300 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:19:58.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:19:58.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370429+0000", 2024-09-18T00:19:58.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031138530000000002 2024-09-18T00:19:58.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370435+0000", 2024-09-18T00:19:58.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6269999999999997e-06 2024-09-18T00:19:58.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370455+0000", 2024-09-18T00:19:58.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0426999999999999e-05 2024-09-18T00:19:58.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370465+0000", 2024-09-18T00:19:58.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0118999999999999e-05 2024-09-18T00:19:58.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425525+0000", 2024-09-18T00:19:58.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055059943 2024-09-18T00:19:58.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425645+0000", 2024-09-18T00:19:58.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119392 2024-09-18T00:19:58.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425766+0000", 2024-09-18T00:19:58.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012151599999999999 2024-09-18T00:19:58.303 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.303 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.304 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.304 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48503 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16326656~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:58.304 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:58.304 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.857276658, 2024-09-18T00:19:58.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058310395000000001, 2024-09-18T00:19:58.304 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.304 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.304 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.304 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.304 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.304 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.305 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48503 2024-09-18T00:19:58.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.305 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:58.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:58.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:19:58.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0100000000000004e-07 2024-09-18T00:19:58.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371199+0000", 2024-09-18T00:19:58.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8899999999999999e-06 2024-09-18T00:19:58.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371211+0000", 2024-09-18T00:19:58.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1707e-05 2024-09-18T00:19:58.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371218+0000", 2024-09-18T00:19:58.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.131e-06 2024-09-18T00:19:58.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429426+0000", 2024-09-18T00:19:58.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058207771999999998 2024-09-18T00:19:58.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429467+0000", 2024-09-18T00:19:58.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1532999999999997e-05 2024-09-18T00:19:58.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429506+0000", 2024-09-18T00:19:58.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8661000000000001e-05 2024-09-18T00:19:58.308 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.308 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.308 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.308 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48504 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16359424~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:58.308 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:58.308 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.857244839, 2024-09-18T00:19:58.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058412775, 2024-09-18T00:19:58.309 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.309 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.309 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.309 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.309 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.309 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.309 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48504 2024-09-18T00:19:58.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.309 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:58.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:58.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:19:58.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6899999999999996e-07 2024-09-18T00:19:58.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371231+0000", 2024-09-18T00:19:58.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6709999999999999e-06 2024-09-18T00:19:58.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371240+0000", 2024-09-18T00:19:58.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7309999999999999e-06 2024-09-18T00:19:58.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371249+0000", 2024-09-18T00:19:58.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4280000000000002e-06 2024-09-18T00:19:58.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429533+0000", 2024-09-18T00:19:58.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058283852999999997 2024-09-18T00:19:58.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429590+0000", 2024-09-18T00:19:58.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6824999999999999e-05 2024-09-18T00:19:58.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429640+0000", 2024-09-18T00:19:58.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0498000000000003e-05 2024-09-18T00:19:58.312 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.312 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.313 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.313 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48505 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16392192~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:58.313 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:58.313 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.857211866, 2024-09-18T00:19:58.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058482777999999999, 2024-09-18T00:19:58.313 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.313 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.313 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.313 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.313 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.313 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.313 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48505 2024-09-18T00:19:58.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.314 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:58.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:58.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:19:58.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7800000000000001e-07 2024-09-18T00:19:58.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371264+0000", 2024-09-18T00:19:58.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-18T00:19:58.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371274+0000", 2024-09-18T00:19:58.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4690000000000003e-06 2024-09-18T00:19:58.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371281+0000", 2024-09-18T00:19:58.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4120000000000004e-06 2024-09-18T00:19:58.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429665+0000", 2024-09-18T00:19:58.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058384108999999997 2024-09-18T00:19:58.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429705+0000", 2024-09-18T00:19:58.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0299000000000002e-05 2024-09-18T00:19:58.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429743+0000", 2024-09-18T00:19:58.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7951000000000003e-05 2024-09-18T00:19:58.317 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.317 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.317 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.317 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48506 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16424960~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:58.317 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:19:58.317 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.857182653999999, 2024-09-18T00:19:58.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058604485999999997, 2024-09-18T00:19:58.318 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.318 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.318 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.318 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.318 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.318 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.318 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48506 2024-09-18T00:19:58.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.318 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:19:58.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:19:58.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371355+0000", 2024-09-18T00:19:58.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5549000000000005e-05 2024-09-18T00:19:58.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371359+0000", 2024-09-18T00:19:58.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1049999999999997e-06 2024-09-18T00:19:58.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371380+0000", 2024-09-18T00:19:58.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0205000000000001e-05 2024-09-18T00:19:58.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371388+0000", 2024-09-18T00:19:58.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3229999999999998e-06 2024-09-18T00:19:58.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429771+0000", 2024-09-18T00:19:58.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058382878999999999 2024-09-18T00:19:58.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429842+0000", 2024-09-18T00:19:58.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1110000000000002e-05 2024-09-18T00:19:58.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429894+0000", 2024-09-18T00:19:58.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2315000000000003e-05 2024-09-18T00:19:58.321 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.321 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.322 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.322 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48507 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16457728~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:19:58.322 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:19:58.322 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.857071294000001, 2024-09-18T00:19:58.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058597275999999997, 2024-09-18T00:19:58.322 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.322 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.322 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.322 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.322 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.322 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.322 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48507 2024-09-18T00:19:58.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.323 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:19:58.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:19:58.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374394+0000", 2024-09-18T00:19:58.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029924449999999998 2024-09-18T00:19:58.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374400+0000", 2024-09-18T00:19:58.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0229999999999998e-06 2024-09-18T00:19:58.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374419+0000", 2024-09-18T00:19:58.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9488e-05 2024-09-18T00:19:58.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374432+0000", 2024-09-18T00:19:58.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2492e-05 2024-09-18T00:19:58.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429917+0000", 2024-09-18T00:19:58.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055485087000000002 2024-09-18T00:19:58.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429958+0000", 2024-09-18T00:19:58.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1258999999999999e-05 2024-09-18T00:19:58.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429998+0000", 2024-09-18T00:19:58.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0482000000000003e-05 2024-09-18T00:19:58.326 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.326 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.326 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.326 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:58332 3.15 3:ac910418:::rbd_data.112a17f95ba0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T00:19:58.326 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:19:58.326 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.170567262999999, 2024-09-18T00:19:58.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.071238735999999997, 2024-09-18T00:19:58.327 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.327 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.327 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:58.327 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.327 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.327 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.327 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 58332 2024-09-18T00:19:58.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.327 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:19:58.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:19:58.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057907+0000", 2024-09-18T00:19:58.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1859999999999999e-06 2024-09-18T00:19:58.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057920+0000", 2024-09-18T00:19:58.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3013000000000001e-05 2024-09-18T00:19:58.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057949+0000", 2024-09-18T00:19:58.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8745999999999999e-05 2024-09-18T00:19:58.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057966+0000", 2024-09-18T00:19:58.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6477000000000002e-05 2024-09-18T00:19:58.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.058035+0000", 2024-09-18T00:19:58.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9560000000000005e-05 2024-09-18T00:19:58.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.058133+0000", 2024-09-18T00:19:58.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8051e-05 2024-09-18T00:19:58.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:19:58.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.061809+0000", 2024-09-18T00:19:58.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003675453 2024-09-18T00:19:58.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:58.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128667+0000", 2024-09-18T00:19:58.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.066857960999999994 2024-09-18T00:19:58.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:58.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128952+0000", 2024-09-18T00:19:58.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028583100000000001 2024-09-18T00:19:58.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:58.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129060+0000", 2024-09-18T00:19:58.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010806000000000001 2024-09-18T00:19:58.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129144+0000", 2024-09-18T00:19:58.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3398000000000003e-05 2024-09-18T00:19:58.332 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.332 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.332 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.332 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:58332 3.15 e96/32 ondisk, result = 0)", 2024-09-18T00:19:58.332 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:19:58.332 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.158374433999999, 2024-09-18T00:19:58.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059068112999999998, 2024-09-18T00:19:58.333 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.333 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.333 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.333 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:19:58.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:19:58.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070099+0000", 2024-09-18T00:19:58.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9899999999999999e-07 2024-09-18T00:19:58.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070103+0000", 2024-09-18T00:19:58.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6880000000000001e-06 2024-09-18T00:19:58.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070115+0000", 2024-09-18T00:19:58.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2088e-05 2024-09-18T00:19:58.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070122+0000", 2024-09-18T00:19:58.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3900000000000004e-06 2024-09-18T00:19:58.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128758+0000", 2024-09-18T00:19:58.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058635541999999999 2024-09-18T00:19:58.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128774+0000", 2024-09-18T00:19:58.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6858999999999999e-05 2024-09-18T00:19:58.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129166+0000", 2024-09-18T00:19:58.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039164700000000001 2024-09-18T00:19:58.336 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.336 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.336 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.336 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.336 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.336 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:58.338 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:19:58.338 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:58.336+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:58.439 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:58.436+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:58.465 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:58.465 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:58.465 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:19:58.465 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:58.479 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:19:58.539 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:58.537+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:58.597 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:58.597 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:19:58.597 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:19:58.597 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:19:58.597 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:58.611 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:19:58.654 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:58.653+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:58.723 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is unset 2024-09-18T00:19:58.741 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:58.740+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:58.790 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:19:58.790 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:19:58.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:19:58.791 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:19:58.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.791 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34543 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:58.791 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:19:58.791 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.496448700000002, 2024-09-18T00:19:58.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023207593, 2024-09-18T00:19:58.791 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.791 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.791 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:58.791 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.791 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.791 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.792 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34543 2024-09-18T00:19:58.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.792 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:19:58.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:19:58.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290783+0000", 2024-09-18T00:19:58.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5240000000000001e-06 2024-09-18T00:19:58.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290799+0000", 2024-09-18T00:19:58.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:19:58.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290840+0000", 2024-09-18T00:19:58.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0926000000000001e-05 2024-09-18T00:19:58.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290868+0000", 2024-09-18T00:19:58.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8163000000000001e-05 2024-09-18T00:19:58.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291294+0000", 2024-09-18T00:19:58.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00042588700000000001 2024-09-18T00:19:58.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:19:58.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:19:58.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.986e-05 2024-09-18T00:19:58.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316617+0000", 2024-09-18T00:19:58.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025303583000000001 2024-09-18T00:19:58.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:58.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316637+0000", 2024-09-18T00:19:58.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9388000000000001e-05 2024-09-18T00:19:58.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313933+0000", 2024-09-18T00:19:58.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99729681000000003 2024-09-18T00:19:58.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313987+0000", 2024-09-18T00:19:58.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4069e-05 2024-09-18T00:19:58.796 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.796 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.796 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.796 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34544 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:58.796 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:19:58.796 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.496300345000002, 2024-09-18T00:19:58.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0232412120000001, 2024-09-18T00:19:58.797 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.797 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.797 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:58.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.797 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34544 2024-09-18T00:19:58.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.797 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:19:58.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:19:58.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290930+0000", 2024-09-18T00:19:58.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5030000000000001e-06 2024-09-18T00:19:58.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290940+0000", 2024-09-18T00:19:58.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9000000000000001e-06 2024-09-18T00:19:58.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290964+0000", 2024-09-18T00:19:58.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4273e-05 2024-09-18T00:19:58.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290982+0000", 2024-09-18T00:19:58.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8216e-05 2024-09-18T00:19:58.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291433+0000", 2024-09-18T00:19:58.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000451278 2024-09-18T00:19:58.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:58.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291446+0000", 2024-09-18T00:19:58.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3039e-05 2024-09-18T00:19:58.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316687+0000", 2024-09-18T00:19:58.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025240069 2024-09-18T00:19:58.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:58.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316701+0000", 2024-09-18T00:19:58.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4100000000000001e-05 2024-09-18T00:19:58.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314142+0000", 2024-09-18T00:19:58.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99744119799999997 2024-09-18T00:19:58.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314169+0000", 2024-09-18T00:19:58.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7636000000000001e-05 2024-09-18T00:19:58.802 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.802 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.802 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.802 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34545 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:58.802 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:19:58.802 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.496221822999999, 2024-09-18T00:19:58.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023257802, 2024-09-18T00:19:58.802 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.803 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.803 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:58.803 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.803 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.803 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.803 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34545 2024-09-18T00:19:58.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.803 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:19:58.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:19:58.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291008+0000", 2024-09-18T00:19:58.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.237e-06 2024-09-18T00:19:58.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291015+0000", 2024-09-18T00:19:58.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2030000000000003e-06 2024-09-18T00:19:58.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291035+0000", 2024-09-18T00:19:58.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.978e-05 2024-09-18T00:19:58.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291049+0000", 2024-09-18T00:19:58.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4429e-05 2024-09-18T00:19:58.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291630+0000", 2024-09-18T00:19:58.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00058103199999999999 2024-09-18T00:19:58.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:58.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291644+0000", 2024-09-18T00:19:58.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.342e-05 2024-09-18T00:19:58.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316753+0000", 2024-09-18T00:19:58.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025108703999999999 2024-09-18T00:19:58.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:58.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316766+0000", 2024-09-18T00:19:58.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3237e-05 2024-09-18T00:19:58.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314238+0000", 2024-09-18T00:19:58.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99747232699999999 2024-09-18T00:19:58.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314265+0000", 2024-09-18T00:19:58.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6432999999999999e-05 2024-09-18T00:19:58.808 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.808 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.808 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.808 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34546 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:58.808 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:19:58.808 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.496163484, 2024-09-18T00:19:58.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023324535, 2024-09-18T00:19:58.808 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.808 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.808 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:58.809 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.809 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.809 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.809 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34546 2024-09-18T00:19:58.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.809 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:19:58.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:19:58.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291066+0000", 2024-09-18T00:19:58.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.569999999999999e-07 2024-09-18T00:19:58.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291073+0000", 2024-09-18T00:19:58.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.063e-06 2024-09-18T00:19:58.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291090+0000", 2024-09-18T00:19:58.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6977e-05 2024-09-18T00:19:58.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291105+0000", 2024-09-18T00:19:58.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5166e-05 2024-09-18T00:19:58.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291771+0000", 2024-09-18T00:19:58.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00066599699999999999 2024-09-18T00:19:58.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:58.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291785+0000", 2024-09-18T00:19:58.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.327e-05 2024-09-18T00:19:58.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316826+0000", 2024-09-18T00:19:58.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025041955000000001 2024-09-18T00:19:58.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:58.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316840+0000", 2024-09-18T00:19:58.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3256e-05 2024-09-18T00:19:58.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314335+0000", 2024-09-18T00:19:58.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99749496100000001 2024-09-18T00:19:58.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314390+0000", 2024-09-18T00:19:58.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4932999999999997e-05 2024-09-18T00:19:58.813 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.813 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.813 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.813 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34547 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:58.813 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:19:58.813 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.496104185, 2024-09-18T00:19:58.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.028510467, 2024-09-18T00:19:58.814 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.814 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.814 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:58.814 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.814 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.814 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.814 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34547 2024-09-18T00:19:58.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.814 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:19:58.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:19:58.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291126+0000", 2024-09-18T00:19:58.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2699999999999999e-06 2024-09-18T00:19:58.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291150+0000", 2024-09-18T00:19:58.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4669999999999999e-05 2024-09-18T00:19:58.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291171+0000", 2024-09-18T00:19:58.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0706000000000001e-05 2024-09-18T00:19:58.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291186+0000", 2024-09-18T00:19:58.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5126999999999999e-05 2024-09-18T00:19:58.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291885+0000", 2024-09-18T00:19:58.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00069850299999999995 2024-09-18T00:19:58.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:58.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291897+0000", 2024-09-18T00:19:58.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-05 2024-09-18T00:19:58.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316887+0000", 2024-09-18T00:19:58.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024990155999999999 2024-09-18T00:19:58.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:58.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316900+0000", 2024-09-18T00:19:58.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.295e-05 2024-09-18T00:19:58.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319594+0000", 2024-09-18T00:19:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0026940719999999 2024-09-18T00:19:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319635+0000", 2024-09-18T00:19:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0583000000000001e-05 2024-09-18T00:19:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.818 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34548 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:19:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:19:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.495956726999999, 2024-09-18T00:19:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0285287400000001, 2024-09-18T00:19:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:19:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34548 2024-09-18T00:19:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:19:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:19:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291274+0000", 2024-09-18T00:19:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.12e-06 2024-09-18T00:19:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291288+0000", 2024-09-18T00:19:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4367e-05 2024-09-18T00:19:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:19:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5698999999999999e-05 2024-09-18T00:19:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291333+0000", 2024-09-18T00:19:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8783e-05 2024-09-18T00:19:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291987+0000", 2024-09-18T00:19:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00065417800000000005 2024-09-18T00:19:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:19:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.292000+0000", 2024-09-18T00:19:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2652e-05 2024-09-18T00:19:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316946+0000", 2024-09-18T00:19:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024946696000000001 2024-09-18T00:19:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:19:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316956+0000", 2024-09-18T00:19:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2760000000000006e-06 2024-09-18T00:19:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319752+0000", 2024-09-18T00:19:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002796083 2024-09-18T00:19:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319801+0000", 2024-09-18T00:19:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8885999999999999e-05 2024-09-18T00:19:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41950 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:19:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.093304017999998, 2024-09-18T00:19:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053368486, 2024-09-18T00:19:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41950 2024-09-18T00:19:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:19:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:19:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693926+0000", 2024-09-18T00:19:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4900000000000004e-07 2024-09-18T00:19:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693930+0000", 2024-09-18T00:19:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0130000000000004e-06 2024-09-18T00:19:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693940+0000", 2024-09-18T00:19:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0618000000000001e-05 2024-09-18T00:19:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693948+0000", 2024-09-18T00:19:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3140000000000004e-06 2024-09-18T00:19:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747133+0000", 2024-09-18T00:19:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053184689 2024-09-18T00:19:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747203+0000", 2024-09-18T00:19:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0339000000000005e-05 2024-09-18T00:19:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747293+0000", 2024-09-18T00:19:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9563999999999998e-05 2024-09-18T00:19:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41951 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:19:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.093269251000002, 2024-09-18T00:19:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053527695, 2024-09-18T00:19:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41951 2024-09-18T00:19:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:19:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:19:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693960+0000", 2024-09-18T00:19:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.16e-07 2024-09-18T00:19:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693971+0000", 2024-09-18T00:19:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1204e-05 2024-09-18T00:19:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693981+0000", 2024-09-18T00:19:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-18T00:19:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693989+0000", 2024-09-18T00:19:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0180000000000001e-06 2024-09-18T00:19:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747328+0000", 2024-09-18T00:19:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053338508 2024-09-18T00:19:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747400+0000", 2024-09-18T00:19:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2750999999999995e-05 2024-09-18T00:19:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747487+0000", 2024-09-18T00:19:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6527999999999996e-05 2024-09-18T00:19:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41952 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:19:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.093228844000002, 2024-09-18T00:19:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053728013999999998, 2024-09-18T00:19:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41952 2024-09-18T00:19:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:19:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:19:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694001+0000", 2024-09-18T00:19:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3799999999999996e-07 2024-09-18T00:19:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694004+0000", 2024-09-18T00:19:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.965e-06 2024-09-18T00:19:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694013+0000", 2024-09-18T00:19:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1689999999999998e-06 2024-09-18T00:19:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694022+0000", 2024-09-18T00:19:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7560000000000005e-06 2024-09-18T00:19:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747531+0000", 2024-09-18T00:19:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053508999000000002 2024-09-18T00:19:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747610+0000", 2024-09-18T00:19:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9212e-05 2024-09-18T00:19:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747728+0000", 2024-09-18T00:19:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011797499999999999 2024-09-18T00:19:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41953 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:19:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.093195215000001, 2024-09-18T00:19:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053936783000000002, 2024-09-18T00:19:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41953 2024-09-18T00:19:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:19:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:19:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694034+0000", 2024-09-18T00:19:58.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5199999999999996e-07 2024-09-18T00:19:58.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694037+0000", 2024-09-18T00:19:58.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.745e-06 2024-09-18T00:19:58.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694046+0000", 2024-09-18T00:19:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2490000000000007e-06 2024-09-18T00:19:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694054+0000", 2024-09-18T00:19:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1179999999999992e-06 2024-09-18T00:19:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747766+0000", 2024-09-18T00:19:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053711362999999998 2024-09-18T00:19:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747856+0000", 2024-09-18T00:19:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0779000000000006e-05 2024-09-18T00:19:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747970+0000", 2024-09-18T00:19:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011377700000000001 2024-09-18T00:19:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41954 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.093165061999997, 2024-09-18T00:19:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054104898999999998, 2024-09-18T00:19:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41954 2024-09-18T00:19:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:19:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4000000000000002e-07 2024-09-18T00:19:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694067+0000", 2024-09-18T00:19:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7329999999999999e-06 2024-09-18T00:19:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694076+0000", 2024-09-18T00:19:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2019999999999993e-06 2024-09-18T00:19:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694083+0000", 2024-09-18T00:19:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0029999999999997e-06 2024-09-18T00:19:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748003+0000", 2024-09-18T00:19:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053920463000000002 2024-09-18T00:19:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748077+0000", 2024-09-18T00:19:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4041000000000005e-05 2024-09-18T00:19:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748168+0000", 2024-09-18T00:19:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0916999999999997e-05 2024-09-18T00:19:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41955 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 231424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:19:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.093136223999998, 2024-09-18T00:19:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054272659000000001, 2024-09-18T00:19:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41955 2024-09-18T00:19:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:19:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:19:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694093+0000", 2024-09-18T00:19:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:19:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694109+0000", 2024-09-18T00:19:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6061000000000001e-05 2024-09-18T00:19:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694121+0000", 2024-09-18T00:19:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.216e-05 2024-09-18T00:19:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694129+0000", 2024-09-18T00:19:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0029999999999994e-06 2024-09-18T00:19:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748202+0000", 2024-09-18T00:19:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054072571 2024-09-18T00:19:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748274+0000", 2024-09-18T00:19:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2117000000000001e-05 2024-09-18T00:19:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748365+0000", 2024-09-18T00:19:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1139999999999998e-05 2024-09-18T00:19:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41956 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.093088016000003, 2024-09-18T00:19:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054420496999999998, 2024-09-18T00:19:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41956 2024-09-18T00:19:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:19:58.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8699999999999995e-07 2024-09-18T00:19:58.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694144+0000", 2024-09-18T00:19:58.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3510000000000002e-06 2024-09-18T00:19:58.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694154+0000", 2024-09-18T00:19:58.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0769999999999997e-06 2024-09-18T00:19:58.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694163+0000", 2024-09-18T00:19:58.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3300000000000005e-06 2024-09-18T00:19:58.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748397+0000", 2024-09-18T00:19:58.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054234497999999999 2024-09-18T00:19:58.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748471+0000", 2024-09-18T00:19:58.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3441000000000004e-05 2024-09-18T00:19:58.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748561+0000", 2024-09-18T00:19:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0212999999999998e-05 2024-09-18T00:19:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41957 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:19:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.093056764000004, 2024-09-18T00:19:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054590310000000003, 2024-09-18T00:19:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41957 2024-09-18T00:19:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:19:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:19:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694173+0000", 2024-09-18T00:19:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2500000000000005e-07 2024-09-18T00:19:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694175+0000", 2024-09-18T00:19:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.655e-06 2024-09-18T00:19:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694184+0000", 2024-09-18T00:19:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7630000000000006e-06 2024-09-18T00:19:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694191+0000", 2024-09-18T00:19:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8870000000000003e-06 2024-09-18T00:19:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748593+0000", 2024-09-18T00:19:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054402525 2024-09-18T00:19:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748669+0000", 2024-09-18T00:19:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5532999999999997e-05 2024-09-18T00:19:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748762+0000", 2024-09-18T00:19:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3221999999999996e-05 2024-09-18T00:19:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42392 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:19:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.080027325000003, 2024-09-18T00:19:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053378162, 2024-09-18T00:19:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42392 2024-09-18T00:19:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:19:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:19:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707430+0000", 2024-09-18T00:19:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022838599999999999 2024-09-18T00:19:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707434+0000", 2024-09-18T00:19:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5220000000000001e-06 2024-09-18T00:19:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707467+0000", 2024-09-18T00:19:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2570000000000002e-05 2024-09-18T00:19:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707475+0000", 2024-09-18T00:19:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:19:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760446+0000", 2024-09-18T00:19:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052971635000000003 2024-09-18T00:19:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760512+0000", 2024-09-18T00:19:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5679999999999995e-05 2024-09-18T00:19:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760579+0000", 2024-09-18T00:19:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7296000000000001e-05 2024-09-18T00:19:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42393 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 358400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:19:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.079738374000001, 2024-09-18T00:19:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053269618999999997, 2024-09-18T00:19:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42393 2024-09-18T00:19:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:19:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:19:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707491+0000", 2024-09-18T00:19:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1399999999999997e-07 2024-09-18T00:19:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707503+0000", 2024-09-18T00:19:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1912e-05 2024-09-18T00:19:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707512+0000", 2024-09-18T00:19:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1030000000000008e-06 2024-09-18T00:19:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707520+0000", 2024-09-18T00:19:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1249999999999993e-06 2024-09-18T00:19:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760605+0000", 2024-09-18T00:19:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053085495000000003 2024-09-18T00:19:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760680+0000", 2024-09-18T00:19:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5024000000000005e-05 2024-09-18T00:19:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760760+0000", 2024-09-18T00:19:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9345999999999996e-05 2024-09-18T00:19:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42394 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:19:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:19:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.079698385999997, 2024-09-18T00:19:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053403144999999999, 2024-09-18T00:19:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42394 2024-09-18T00:19:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:19:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:19:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711550+0000", 2024-09-18T00:19:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040200189999999997 2024-09-18T00:19:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711556+0000", 2024-09-18T00:19:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2339999999999999e-06 2024-09-18T00:19:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711569+0000", 2024-09-18T00:19:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2337999999999999e-05 2024-09-18T00:19:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711577+0000", 2024-09-18T00:19:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5660000000000007e-06 2024-09-18T00:19:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760790+0000", 2024-09-18T00:19:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049212196999999999 2024-09-18T00:19:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760857+0000", 2024-09-18T00:19:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7756000000000007e-05 2024-09-18T00:19:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760933+0000", 2024-09-18T00:19:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6035e-05 2024-09-18T00:19:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:55489 3.5 3:a5f1e468:::rbd_data.112a859a01c9.000000000000000c:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T00:19:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:19:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.155844259999999, 2024-09-18T00:19:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.05748731, 2024-09-18T00:19:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 55489 2024-09-18T00:19:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:19:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:19:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631388+0000", 2024-09-18T00:19:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6169999999999998e-06 2024-09-18T00:19:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631405+0000", 2024-09-18T00:19:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6643e-05 2024-09-18T00:19:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631444+0000", 2024-09-18T00:19:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9226e-05 2024-09-18T00:19:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631465+0000", 2024-09-18T00:19:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0826999999999999e-05 2024-09-18T00:19:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631578+0000", 2024-09-18T00:19:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011291899999999999 2024-09-18T00:19:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631651+0000", 2024-09-18T00:19:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3264999999999997e-05 2024-09-18T00:19:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:19:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.634073+0000", 2024-09-18T00:19:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0024221350000000002 2024-09-18T00:19:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688732+0000", 2024-09-18T00:19:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054658610000000003 2024-09-18T00:19:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688794+0000", 2024-09-18T00:19:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2255999999999995e-05 2024-09-18T00:19:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688834+0000", 2024-09-18T00:19:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9916999999999997e-05 2024-09-18T00:19:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688872+0000", 2024-09-18T00:19:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7895000000000002e-05 2024-09-18T00:19:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:58361 3.1d 3:bac90a3d:::rbd_data.112a17f95ba0.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T00:19:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:19:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.583498765999998, 2024-09-18T00:19:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101552963, 2024-09-18T00:19:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:19:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:19:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:19:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:19:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 58361 2024-09-18T00:19:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:19:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:19:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203733+0000", 2024-09-18T00:19:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.01e-06 2024-09-18T00:19:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203745+0000", 2024-09-18T00:19:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2513e-05 2024-09-18T00:19:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203777+0000", 2024-09-18T00:19:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1216000000000001e-05 2024-09-18T00:19:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203792+0000", 2024-09-18T00:19:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586e-05 2024-09-18T00:19:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203911+0000", 2024-09-18T00:19:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119234 2024-09-18T00:19:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.204056+0000", 2024-09-18T00:19:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000144943 2024-09-18T00:19:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:19:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.210219+0000", 2024-09-18T00:19:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061630649999999997 2024-09-18T00:19:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:19:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305071+0000", 2024-09-18T00:19:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.094852120999999998 2024-09-18T00:19:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:19:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305106+0000", 2024-09-18T00:19:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4848999999999998e-05 2024-09-18T00:19:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:19:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305208+0000", 2024-09-18T00:19:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101518 2024-09-18T00:19:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305283+0000", 2024-09-18T00:19:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4907999999999995e-05 2024-09-18T00:19:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:58361 3.1d e96/79 ondisk, result = 0)", 2024-09-18T00:19:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:19:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.556015725000002, 2024-09-18T00:19:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.073876681, 2024-09-18T00:19:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:19:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:19:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:19:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:19:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:19:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:19:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:19:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:19:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:19:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:19:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231215+0000", 2024-09-18T00:19:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9570000000000001e-06 2024-09-18T00:19:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:19:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231223+0000", 2024-09-18T00:19:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0900000000000005e-06 2024-09-18T00:19:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:19:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231252+0000", 2024-09-18T00:19:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9552000000000001e-05 2024-09-18T00:19:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:19:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231267+0000", 2024-09-18T00:19:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4263e-05 2024-09-18T00:19:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:19:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.304871+0000", 2024-09-18T00:19:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.073604257000000006 2024-09-18T00:19:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:19:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.304889+0000", 2024-09-18T00:19:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.808e-05 2024-09-18T00:19:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:19:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:19:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:19:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305090+0000", 2024-09-18T00:19:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020048200000000001 2024-09-18T00:19:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:19:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:19:58.891 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:19:58.892 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:19:58.893 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:19:58.888+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:19:58.993 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:19:58.992+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:19:59.015 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:59.015 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:59.015 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:59.015 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:59.029 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:19:59.094 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:59.093+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:59.195 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:59.192+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:59.196 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:19:59.196 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:19:59.196 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:59.210 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:19:59.297 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:19:59.293+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:19:59.387 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:59.388 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:19:59.388 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:19:59.388 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [] 2024-09-18T00:19:59.388 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:59.398 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:19:59.396+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:19:59.405 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:19:59.498 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:19:59.496+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:19:59.576 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:19:59.577 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:19:59.590 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:19:59.592 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:19:59.634 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:19:59.632+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:19:59.700 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:19:59.697+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:19:59.750 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:19:59.750 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:19:59.768 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:59.768 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:59.772 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:19:59.772 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:19:59.784 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:19:59.790 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:19:59.791 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:19:59.801 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:19:59.797+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:19:59.808 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:19:59.808 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:19:59.829 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:19:59.829 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:19:59.865 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:19:59.865 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:19:59.889 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:19:59.889 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:19:59.902 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:19:59.901+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:19:59.912 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:19:59.913 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:19:59.914 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:19:59.914 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:19:59.937 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:20:00.003 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:00.001+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:00.099 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:00.099 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:00.104 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:00.100+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:00.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:20:00.204 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:00.201+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:00.280 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:00.280 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:00.294 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:20:00.341 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:00.340+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:00.406 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:00.405+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:00.418 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:00.418 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:00.438 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:20:00.506 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:20:00.505+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:20:00.607 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:20:00.605+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:20:00.615 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:00.615 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:00.630 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:20:00.708 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:20:00.704+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:20:00.751 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:20:00.755 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:00.755 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:00.770 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:20:00.817 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:00.816+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:00.910 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:00.909+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:00.959 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:20:01.011 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:20:01.009+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:20:01.090 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:01.091 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:01.104 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:20:01.146 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:01.145+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:01.213 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:01.209+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:01.267 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:01.267 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:01.281 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:20:01.322 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:01.321+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:01.415 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:20:01.412+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:20:01.439 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is unset 2024-09-18T00:20:01.451 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:01.451 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:01.467 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:20:01.515 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:01.513+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:01.599 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:01.599 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:01.614 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:20:01.616 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:01.612+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:01.717 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:01.713+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:01.784 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:01.784 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:01.798 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:20:01.818 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:01.816+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:01.919 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:01.917+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:01.939 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:20:01.967 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:01.968 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:01.982 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:20:02.020 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:20:02.016+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:20:02.094 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:20:02.095 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:20:02.111 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:20:02.111 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:20:02.121 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:02.117+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:02.127 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:20:02.127 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:20:02.144 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:20:02.144 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:20:02.157 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:02.157 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:02.163 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:20:02.163 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:20:02.172 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:20:02.184 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:20:02.184 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:20:02.204 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:20:02.205 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:20:02.222 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:02.220+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:02.222 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:20:02.223 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:20:02.323 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:02.320+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:02.336 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:02.336 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:02.336 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:20:02.337 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:20:02.337 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:02.350 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:20:02.424 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:20:02.421+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:20:02.477 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:20:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:20:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:20:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2228 3.15 3:aa2fc059:::rbd_data.112aa7ba4578.000000000000014a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:20:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:20:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 102.210105918, 2024-09-18T00:20:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.068966377999999995, 2024-09-18T00:20:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2228 2024-09-18T00:20:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:20:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:20:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263099+0000", 2024-09-18T00:20:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2359999999999999e-06 2024-09-18T00:20:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263115+0000", 2024-09-18T00:20:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5996999999999999e-05 2024-09-18T00:20:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263156+0000", 2024-09-18T00:20:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:20:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263183+0000", 2024-09-18T00:20:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7233000000000002e-05 2024-09-18T00:20:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263304+0000", 2024-09-18T00:20:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121209 2024-09-18T00:20:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263528+0000", 2024-09-18T00:20:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022401800000000001 2024-09-18T00:20:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:20:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.266458+0000", 2024-09-18T00:20:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029296980000000001 2024-09-18T00:20:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331628+0000", 2024-09-18T00:20:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.065170045999999995 2024-09-18T00:20:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331819+0000", 2024-09-18T00:20:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000190819 2024-09-18T00:20:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331971+0000", 2024-09-18T00:20:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151972 2024-09-18T00:20:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332063+0000", 2024-09-18T00:20:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2083999999999995e-05 2024-09-18T00:20:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2854 3.f 3:f02b0bca:::rbd_data.112aa7ba4578.0000000000000135:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:20:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:20:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 100.60313342800001, 2024-09-18T00:20:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.082582802999999996, 2024-09-18T00:20:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2854 2024-09-18T00:20:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:20:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:20:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870071+0000", 2024-09-18T00:20:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.227e-06 2024-09-18T00:20:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870083+0000", 2024-09-18T00:20:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1909999999999999e-05 2024-09-18T00:20:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870114+0000", 2024-09-18T00:20:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0576e-05 2024-09-18T00:20:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870129+0000", 2024-09-18T00:20:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.485e-05 2024-09-18T00:20:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870256+0000", 2024-09-18T00:20:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012691300000000001 2024-09-18T00:20:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870493+0000", 2024-09-18T00:20:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023737399999999999 2024-09-18T00:20:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.874933+0000", 2024-09-18T00:20:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044398299999999996 2024-09-18T00:20:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952452+0000", 2024-09-18T00:20:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.077518918000000006 2024-09-18T00:20:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952486+0000", 2024-09-18T00:20:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4035999999999997e-05 2024-09-18T00:20:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952575+0000", 2024-09-18T00:20:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.954e-05 2024-09-18T00:20:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952652+0000", 2024-09-18T00:20:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6629000000000001e-05 2024-09-18T00:20:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2854 3.f e38/32 ondisk, result = 0)", 2024-09-18T00:20:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:20:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 100.58560261300001, 2024-09-18T00:20:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064868625999999999, 2024-09-18T00:20:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:20:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:20:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887602+0000", 2024-09-18T00:20:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.677e-06 2024-09-18T00:20:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887609+0000", 2024-09-18T00:20:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6729999999999992e-06 2024-09-18T00:20:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887637+0000", 2024-09-18T00:20:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8206000000000001e-05 2024-09-18T00:20:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887651+0000", 2024-09-18T00:20:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3387000000000001e-05 2024-09-18T00:20:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952422+0000", 2024-09-18T00:20:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064771366999999996 2024-09-18T00:20:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952441+0000", 2024-09-18T00:20:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8737e-05 2024-09-18T00:20:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952469+0000", 2024-09-18T00:20:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7579000000000001e-05 2024-09-18T00:20:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15879 3.16 3:692b47b6:::rbd_object_map.112a13ec2d6c.000000000000000b:head [call] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:20:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:20:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 77.818438705999995, 2024-09-18T00:20:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.011184018, 2024-09-18T00:20:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15879 2024-09-18T00:20:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:20:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:20:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654768+0000", 2024-09-18T00:20:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9129999999999996e-06 2024-09-18T00:20:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654786+0000", 2024-09-18T00:20:02.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8695999999999999e-05 2024-09-18T00:20:02.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654832+0000", 2024-09-18T00:20:02.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5735000000000002e-05 2024-09-18T00:20:02.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654860+0000", 2024-09-18T00:20:02.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7789000000000001e-05 2024-09-18T00:20:02.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655048+0000", 2024-09-18T00:20:02.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018781900000000001 2024-09-18T00:20:02.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:20:02.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655070+0000", 2024-09-18T00:20:02.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1916000000000002e-05 2024-09-18T00:20:02.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663335+0000", 2024-09-18T00:20:02.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0082651210000000003 2024-09-18T00:20:02.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:02.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663358+0000", 2024-09-18T00:20:02.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2986e-05 2024-09-18T00:20:02.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663739+0000", 2024-09-18T00:20:02.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0003814820000001 2024-09-18T00:20:02.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663920+0000", 2024-09-18T00:20:02.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018116900000000001 2024-09-18T00:20:02.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.665948+0000", 2024-09-18T00:20:02.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020273919999999998 2024-09-18T00:20:02.501 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.501 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.501 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.501 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15881 3.16 3:6cc14920:::rbd_header.112a3285400b:head [watch reconnect cookie 140685950184608 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:20:02.501 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:20:02.501 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 77.818277710999993, 2024-09-18T00:20:02.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0138723599999999, 2024-09-18T00:20:02.502 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.502 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.502 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:02.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:02.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:02.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:02.502 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15881 2024-09-18T00:20:02.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.502 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:20:02.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:20:02.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654926+0000", 2024-09-18T00:20:02.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6470000000000001e-06 2024-09-18T00:20:02.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654935+0000", 2024-09-18T00:20:02.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8859999999999993e-06 2024-09-18T00:20:02.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654960+0000", 2024-09-18T00:20:02.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4689999999999999e-05 2024-09-18T00:20:02.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654976+0000", 2024-09-18T00:20:02.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6319000000000001e-05 2024-09-18T00:20:02.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655238+0000", 2024-09-18T00:20:02.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026145999999999998 2024-09-18T00:20:02.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:02.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655255+0000", 2024-09-18T00:20:02.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7249999999999999e-05 2024-09-18T00:20:02.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663416+0000", 2024-09-18T00:20:02.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081607959999999997 2024-09-18T00:20:02.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:02.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663436+0000", 2024-09-18T00:20:02.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0384e-05 2024-09-18T00:20:02.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666184+0000", 2024-09-18T00:20:02.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002748121 2024-09-18T00:20:02.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666452+0000", 2024-09-18T00:20:02.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026801700000000001 2024-09-18T00:20:02.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.668797+0000", 2024-09-18T00:20:02.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023447899999999998 2024-09-18T00:20:02.507 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.507 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.507 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.507 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17089 3.f 3:f7681386:::rbd_data.112a6efb7564.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:20:02.508 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:20:02.508 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.593350615000006, 2024-09-18T00:20:02.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.12573889399999999, 2024-09-18T00:20:02.508 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.508 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.508 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:02.508 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:02.508 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:02.508 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:02.508 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17089 2024-09-18T00:20:02.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.508 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:20:02.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:20:02.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879854+0000", 2024-09-18T00:20:02.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:20:02.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879867+0000", 2024-09-18T00:20:02.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3103e-05 2024-09-18T00:20:02.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879898+0000", 2024-09-18T00:20:02.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0103e-05 2024-09-18T00:20:02.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879912+0000", 2024-09-18T00:20:02.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4636e-05 2024-09-18T00:20:02.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879992+0000", 2024-09-18T00:20:02.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9814000000000005e-05 2024-09-18T00:20:02.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.880101+0000", 2024-09-18T00:20:02.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109172 2024-09-18T00:20:02.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:02.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.887269+0000", 2024-09-18T00:20:02.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071679300000000003 2024-09-18T00:20:02.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:02.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005269+0000", 2024-09-18T00:20:02.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.117999621 2024-09-18T00:20:02.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:02.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005317+0000", 2024-09-18T00:20:02.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8279000000000003e-05 2024-09-18T00:20:02.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:02.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005464+0000", 2024-09-18T00:20:02.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014661200000000001 2024-09-18T00:20:02.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005591+0000", 2024-09-18T00:20:02.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000127107 2024-09-18T00:20:02.513 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.513 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.513 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.513 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17089 3.f e59/32 ondisk, result = 0)", 2024-09-18T00:20:02.514 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:20:02.514 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.569243749999998, 2024-09-18T00:20:02.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101334403, 2024-09-18T00:20:02.514 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.514 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.514 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:02.514 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:20:02.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:20:02.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903960+0000", 2024-09-18T00:20:02.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.33e-06 2024-09-18T00:20:02.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903966+0000", 2024-09-18T00:20:02.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5929999999999996e-06 2024-09-18T00:20:02.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903986+0000", 2024-09-18T00:20:02.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.018e-05 2024-09-18T00:20:02.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903997+0000", 2024-09-18T00:20:02.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0988999999999999e-05 2024-09-18T00:20:02.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005171+0000", 2024-09-18T00:20:02.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101174267 2024-09-18T00:20:02.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005200+0000", 2024-09-18T00:20:02.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8540000000000001e-05 2024-09-18T00:20:02.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005293+0000", 2024-09-18T00:20:02.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504000000000005e-05 2024-09-18T00:20:02.517 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.517 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.517 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.518 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24529 3.16 3:6857ce1a:::rbd_data.112a990280ef.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:20:02.518 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:20:02.518 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.836432338999998, 2024-09-18T00:20:02.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10693805200000001, 2024-09-18T00:20:02.518 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.518 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.518 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:02.518 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:02.518 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:02.518 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:02.518 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24529 2024-09-18T00:20:02.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.518 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:20:02.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:20:02.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636772+0000", 2024-09-18T00:20:02.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1100000000000001e-06 2024-09-18T00:20:02.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636783+0000", 2024-09-18T00:20:02.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1005999999999999e-05 2024-09-18T00:20:02.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636814+0000", 2024-09-18T00:20:02.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1047000000000003e-05 2024-09-18T00:20:02.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636829+0000", 2024-09-18T00:20:02.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4363e-05 2024-09-18T00:20:02.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636980+0000", 2024-09-18T00:20:02.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015181099999999999 2024-09-18T00:20:02.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.637237+0000", 2024-09-18T00:20:02.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000256352 2024-09-18T00:20:02.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:02.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743536+0000", 2024-09-18T00:20:02.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.106298825 2024-09-18T00:20:02.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:02.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743599+0000", 2024-09-18T00:20:02.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3449999999999997e-05 2024-09-18T00:20:02.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743708+0000", 2024-09-18T00:20:02.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109088 2024-09-18T00:20:02.522 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.522 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.522 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.523 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24805 3.16 3:6de5e15d:::rbd_data.112a990280ef.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:20:02.523 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:20:02.523 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.250629107000002, 2024-09-18T00:20:02.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14775107700000001, 2024-09-18T00:20:02.523 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.523 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.523 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:02.523 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:02.523 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:02.523 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:02.523 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24805 2024-09-18T00:20:02.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.524 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:20:02.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:20:02.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222574+0000", 2024-09-18T00:20:02.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0210000000000001e-06 2024-09-18T00:20:02.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222581+0000", 2024-09-18T00:20:02.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.144e-06 2024-09-18T00:20:02.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222595+0000", 2024-09-18T00:20:02.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4567e-05 2024-09-18T00:20:02.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222603+0000", 2024-09-18T00:20:02.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6529999999999994e-06 2024-09-18T00:20:02.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222687+0000", 2024-09-18T00:20:02.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4587999999999997e-05 2024-09-18T00:20:02.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222828+0000", 2024-09-18T00:20:02.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140178 2024-09-18T00:20:02.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:02.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370231+0000", 2024-09-18T00:20:02.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14740378700000001 2024-09-18T00:20:02.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:02.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370269+0000", 2024-09-18T00:20:02.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7886999999999998e-05 2024-09-18T00:20:02.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370324+0000", 2024-09-18T00:20:02.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5251999999999999e-05 2024-09-18T00:20:02.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.528 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.528 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24852 3.13 3:cc64f0fd:::rbd_data.112a990280ef.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:20:02.528 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:20:02.528 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.017889637000003, 2024-09-18T00:20:02.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.066392943999999995, 2024-09-18T00:20:02.529 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.529 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.529 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:02.529 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:02.529 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:02.529 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:02.529 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24852 2024-09-18T00:20:02.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.529 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:20:02.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:20:02.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455315+0000", 2024-09-18T00:20:02.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.705e-06 2024-09-18T00:20:02.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455324+0000", 2024-09-18T00:20:02.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8120000000000002e-06 2024-09-18T00:20:02.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455349+0000", 2024-09-18T00:20:02.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4250999999999999e-05 2024-09-18T00:20:02.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455361+0000", 2024-09-18T00:20:02.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2805e-05 2024-09-18T00:20:02.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455425+0000", 2024-09-18T00:20:02.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3981999999999998e-05 2024-09-18T00:20:02.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455480+0000", 2024-09-18T00:20:02.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4792e-05 2024-09-18T00:20:02.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:02.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.460671+0000", 2024-09-18T00:20:02.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051907300000000002 2024-09-18T00:20:02.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:02.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521355+0000", 2024-09-18T00:20:02.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060684208000000003 2024-09-18T00:20:02.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:02.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521493+0000", 2024-09-18T00:20:02.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013803200000000001 2024-09-18T00:20:02.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:02.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521590+0000", 2024-09-18T00:20:02.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6339e-05 2024-09-18T00:20:02.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521706+0000", 2024-09-18T00:20:02.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116288 2024-09-18T00:20:02.534 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.534 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.534 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.534 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:25269 3.f 3:f6adfd48:::rbd_data.112aa34d8ec3.0000000000000145:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T00:20:02.534 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:20:02.534 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 53.868299675999999, 2024-09-18T00:20:02.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.075209085999999994, 2024-09-18T00:20:02.535 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.535 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.535 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:02.535 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:02.535 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:02.535 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:02.535 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 25269 2024-09-18T00:20:02.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.535 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:20:02.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:20:02.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604904+0000", 2024-09-18T00:20:02.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:20:02.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604910+0000", 2024-09-18T00:20:02.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.964e-06 2024-09-18T00:20:02.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604925+0000", 2024-09-18T00:20:02.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4894e-05 2024-09-18T00:20:02.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604932+0000", 2024-09-18T00:20:02.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-18T00:20:02.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604990+0000", 2024-09-18T00:20:02.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8143000000000002e-05 2024-09-18T00:20:02.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.605104+0000", 2024-09-18T00:20:02.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011404399999999999 2024-09-18T00:20:02.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:02.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.608755+0000", 2024-09-18T00:20:02.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003651081 2024-09-18T00:20:02.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:02.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679920+0000", 2024-09-18T00:20:02.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.071164543999999996 2024-09-18T00:20:02.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:02.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679978+0000", 2024-09-18T00:20:02.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8585000000000002e-05 2024-09-18T00:20:02.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:02.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680036+0000", 2024-09-18T00:20:02.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7978e-05 2024-09-18T00:20:02.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680112+0000", 2024-09-18T00:20:02.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5527999999999999e-05 2024-09-18T00:20:02.540 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.540 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.540 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.540 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:25269 3.f e72/32 ondisk, result = 0)", 2024-09-18T00:20:02.540 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:20:02.540 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 53.855089290999999, 2024-09-18T00:20:02.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062014761000000002, 2024-09-18T00:20:02.541 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.541 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.541 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:02.541 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:20:02.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:20:02.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618114+0000", 2024-09-18T00:20:02.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1650000000000001e-06 2024-09-18T00:20:02.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618119+0000", 2024-09-18T00:20:02.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-18T00:20:02.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618134+0000", 2024-09-18T00:20:02.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5770999999999999e-05 2024-09-18T00:20:02.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618143+0000", 2024-09-18T00:20:02.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.551e-06 2024-09-18T00:20:02.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679967+0000", 2024-09-18T00:20:02.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061823937000000002 2024-09-18T00:20:02.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679975+0000", 2024-09-18T00:20:02.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7340000000000008e-06 2024-09-18T00:20:02.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680128+0000", 2024-09-18T00:20:02.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015338299999999999 2024-09-18T00:20:02.544 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.544 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.544 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.545 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48484 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 15704064~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:02.545 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:20:02.545 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.105887181999996, 2024-09-18T00:20:02.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058450876999999998, 2024-09-18T00:20:02.545 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.545 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.545 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:02.545 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:02.545 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:02.545 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:02.545 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48484 2024-09-18T00:20:02.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.546 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:20:02.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:20:02.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370429+0000", 2024-09-18T00:20:02.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031138530000000002 2024-09-18T00:20:02.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370435+0000", 2024-09-18T00:20:02.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6269999999999997e-06 2024-09-18T00:20:02.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370455+0000", 2024-09-18T00:20:02.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0426999999999999e-05 2024-09-18T00:20:02.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370465+0000", 2024-09-18T00:20:02.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0118999999999999e-05 2024-09-18T00:20:02.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425525+0000", 2024-09-18T00:20:02.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055059943 2024-09-18T00:20:02.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425645+0000", 2024-09-18T00:20:02.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119392 2024-09-18T00:20:02.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425766+0000", 2024-09-18T00:20:02.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012151599999999999 2024-09-18T00:20:02.549 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.549 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.549 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.549 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48503 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16326656~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:02.549 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:20:02.550 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.102006781, 2024-09-18T00:20:02.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058310395000000001, 2024-09-18T00:20:02.550 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.550 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.550 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:02.550 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:02.550 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:02.550 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:02.550 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48503 2024-09-18T00:20:02.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.550 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:20:02.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:20:02.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:20:02.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0100000000000004e-07 2024-09-18T00:20:02.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371199+0000", 2024-09-18T00:20:02.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8899999999999999e-06 2024-09-18T00:20:02.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371211+0000", 2024-09-18T00:20:02.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1707e-05 2024-09-18T00:20:02.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371218+0000", 2024-09-18T00:20:02.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.131e-06 2024-09-18T00:20:02.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429426+0000", 2024-09-18T00:20:02.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058207771999999998 2024-09-18T00:20:02.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429467+0000", 2024-09-18T00:20:02.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1532999999999997e-05 2024-09-18T00:20:02.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429506+0000", 2024-09-18T00:20:02.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8661000000000001e-05 2024-09-18T00:20:02.555 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.555 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.555 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.555 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48504 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16359424~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:02.555 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:20:02.555 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.101974962, 2024-09-18T00:20:02.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058412775, 2024-09-18T00:20:02.555 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.556 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.556 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:02.556 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:02.556 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:02.556 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:02.556 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48504 2024-09-18T00:20:02.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.556 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:20:02.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:20:02.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:20:02.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6899999999999996e-07 2024-09-18T00:20:02.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371231+0000", 2024-09-18T00:20:02.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6709999999999999e-06 2024-09-18T00:20:02.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371240+0000", 2024-09-18T00:20:02.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7309999999999999e-06 2024-09-18T00:20:02.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371249+0000", 2024-09-18T00:20:02.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4280000000000002e-06 2024-09-18T00:20:02.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429533+0000", 2024-09-18T00:20:02.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058283852999999997 2024-09-18T00:20:02.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429590+0000", 2024-09-18T00:20:02.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6824999999999999e-05 2024-09-18T00:20:02.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429640+0000", 2024-09-18T00:20:02.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0498000000000003e-05 2024-09-18T00:20:02.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.560 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.560 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48505 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16392192~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:02.560 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:20:02.560 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.101941988999997, 2024-09-18T00:20:02.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058482777999999999, 2024-09-18T00:20:02.560 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.560 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.560 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:02.561 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:02.561 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:02.561 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:02.561 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48505 2024-09-18T00:20:02.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.561 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:20:02.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:20:02.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:20:02.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7800000000000001e-07 2024-09-18T00:20:02.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371264+0000", 2024-09-18T00:20:02.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-18T00:20:02.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371274+0000", 2024-09-18T00:20:02.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4690000000000003e-06 2024-09-18T00:20:02.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371281+0000", 2024-09-18T00:20:02.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4120000000000004e-06 2024-09-18T00:20:02.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429665+0000", 2024-09-18T00:20:02.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058384108999999997 2024-09-18T00:20:02.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429705+0000", 2024-09-18T00:20:02.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0299000000000002e-05 2024-09-18T00:20:02.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429743+0000", 2024-09-18T00:20:02.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7951000000000003e-05 2024-09-18T00:20:02.564 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.564 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.564 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.564 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48506 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16424960~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:02.564 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:20:02.565 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.101912777000003, 2024-09-18T00:20:02.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058604485999999997, 2024-09-18T00:20:02.565 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.565 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.565 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:02.565 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:02.565 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:02.565 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:02.565 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48506 2024-09-18T00:20:02.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.565 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:20:02.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:20:02.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371355+0000", 2024-09-18T00:20:02.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5549000000000005e-05 2024-09-18T00:20:02.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371359+0000", 2024-09-18T00:20:02.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1049999999999997e-06 2024-09-18T00:20:02.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371380+0000", 2024-09-18T00:20:02.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0205000000000001e-05 2024-09-18T00:20:02.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371388+0000", 2024-09-18T00:20:02.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3229999999999998e-06 2024-09-18T00:20:02.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429771+0000", 2024-09-18T00:20:02.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058382878999999999 2024-09-18T00:20:02.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429842+0000", 2024-09-18T00:20:02.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1110000000000002e-05 2024-09-18T00:20:02.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429894+0000", 2024-09-18T00:20:02.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2315000000000003e-05 2024-09-18T00:20:02.569 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.569 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.569 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.569 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48507 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16457728~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:02.569 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:20:02.569 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.101801416999997, 2024-09-18T00:20:02.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058597275999999997, 2024-09-18T00:20:02.569 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.569 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.569 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:02.569 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:02.569 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:02.570 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:02.570 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48507 2024-09-18T00:20:02.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.570 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:20:02.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:20:02.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374394+0000", 2024-09-18T00:20:02.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029924449999999998 2024-09-18T00:20:02.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374400+0000", 2024-09-18T00:20:02.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0229999999999998e-06 2024-09-18T00:20:02.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374419+0000", 2024-09-18T00:20:02.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9488e-05 2024-09-18T00:20:02.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374432+0000", 2024-09-18T00:20:02.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2492e-05 2024-09-18T00:20:02.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429917+0000", 2024-09-18T00:20:02.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055485087000000002 2024-09-18T00:20:02.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429958+0000", 2024-09-18T00:20:02.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1258999999999999e-05 2024-09-18T00:20:02.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429998+0000", 2024-09-18T00:20:02.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0482000000000003e-05 2024-09-18T00:20:02.573 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.573 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.573 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.573 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:58332 3.15 3:ac910418:::rbd_data.112a17f95ba0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T00:20:02.573 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:20:02.573 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.415297385999999, 2024-09-18T00:20:02.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.071238735999999997, 2024-09-18T00:20:02.573 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.573 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.574 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:02.574 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:02.574 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:02.574 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:02.574 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 58332 2024-09-18T00:20:02.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.574 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:20:02.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:20:02.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057907+0000", 2024-09-18T00:20:02.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1859999999999999e-06 2024-09-18T00:20:02.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057920+0000", 2024-09-18T00:20:02.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3013000000000001e-05 2024-09-18T00:20:02.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057949+0000", 2024-09-18T00:20:02.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8745999999999999e-05 2024-09-18T00:20:02.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057966+0000", 2024-09-18T00:20:02.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6477000000000002e-05 2024-09-18T00:20:02.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.058035+0000", 2024-09-18T00:20:02.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9560000000000005e-05 2024-09-18T00:20:02.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.058133+0000", 2024-09-18T00:20:02.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8051e-05 2024-09-18T00:20:02.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:20:02.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.061809+0000", 2024-09-18T00:20:02.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003675453 2024-09-18T00:20:02.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:02.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128667+0000", 2024-09-18T00:20:02.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.066857960999999994 2024-09-18T00:20:02.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:02.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128952+0000", 2024-09-18T00:20:02.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028583100000000001 2024-09-18T00:20:02.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:02.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129060+0000", 2024-09-18T00:20:02.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010806000000000001 2024-09-18T00:20:02.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129144+0000", 2024-09-18T00:20:02.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3398000000000003e-05 2024-09-18T00:20:02.579 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.579 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.579 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.580 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:58332 3.15 e96/32 ondisk, result = 0)", 2024-09-18T00:20:02.580 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:20:02.580 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.403104556999999, 2024-09-18T00:20:02.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059068112999999998, 2024-09-18T00:20:02.580 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:02.580 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:02.580 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:02.580 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:02.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:02.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:20:02.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:02.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:20:02.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:02.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:02.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070099+0000", 2024-09-18T00:20:02.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9899999999999999e-07 2024-09-18T00:20:02.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:02.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070103+0000", 2024-09-18T00:20:02.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6880000000000001e-06 2024-09-18T00:20:02.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:02.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070115+0000", 2024-09-18T00:20:02.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2088e-05 2024-09-18T00:20:02.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:02.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070122+0000", 2024-09-18T00:20:02.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3900000000000004e-06 2024-09-18T00:20:02.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:02.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128758+0000", 2024-09-18T00:20:02.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058635541999999999 2024-09-18T00:20:02.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:02.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128774+0000", 2024-09-18T00:20:02.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6858999999999999e-05 2024-09-18T00:20:02.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:02.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:02.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:02.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129166+0000", 2024-09-18T00:20:02.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039164700000000001 2024-09-18T00:20:02.583 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.583 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.583 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.583 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:02.583 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:02.583 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:02.585 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:20:02.585 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:02.581+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:02.686 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:02.681+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:02.743 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 7] out_osds: [6, 1] dead_osds: [] live_osds: [0, 2, 3, 4, 6, 7, 5, 1] 2024-09-18T00:20:02.743 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:20:02.743 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-18T00:20:02.743 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-18T00:20:02.757 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:02.758 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:02.758 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:20:02.758 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:02.772 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:20:02.789 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:02.788+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:02.887 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:20:02.885+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:20:02.936 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:20:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:20:02.937 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:02.950 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:20:02.988 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:02.984+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:03.060 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-18T00:20:03.056+0000 7f63a6107640 -1 mon.a@0(leader).osd e120 definitely_dead 0 2024-09-18T00:20:03.089 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:03.085+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:03.122 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:03.123 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:20:03.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:20:03.123 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:20:03.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.123 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34543 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:03.123 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:20:03.123 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.828639397000003, 2024-09-18T00:20:03.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023207593, 2024-09-18T00:20:03.123 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.123 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.124 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:03.124 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.124 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.124 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.124 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34543 2024-09-18T00:20:03.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.124 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:20:03.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:20:03.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290783+0000", 2024-09-18T00:20:03.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5240000000000001e-06 2024-09-18T00:20:03.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290799+0000", 2024-09-18T00:20:03.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:20:03.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290840+0000", 2024-09-18T00:20:03.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0926000000000001e-05 2024-09-18T00:20:03.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290868+0000", 2024-09-18T00:20:03.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8163000000000001e-05 2024-09-18T00:20:03.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291294+0000", 2024-09-18T00:20:03.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00042588700000000001 2024-09-18T00:20:03.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:20:03.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:20:03.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.986e-05 2024-09-18T00:20:03.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316617+0000", 2024-09-18T00:20:03.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025303583000000001 2024-09-18T00:20:03.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:03.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316637+0000", 2024-09-18T00:20:03.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9388000000000001e-05 2024-09-18T00:20:03.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313933+0000", 2024-09-18T00:20:03.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99729681000000003 2024-09-18T00:20:03.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313987+0000", 2024-09-18T00:20:03.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4069e-05 2024-09-18T00:20:03.130 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.130 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.130 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.130 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34544 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:03.130 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:20:03.130 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.828491042000003, 2024-09-18T00:20:03.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0232412120000001, 2024-09-18T00:20:03.131 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.131 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.131 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:03.131 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.131 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.131 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.131 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34544 2024-09-18T00:20:03.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.131 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:20:03.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:20:03.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290930+0000", 2024-09-18T00:20:03.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5030000000000001e-06 2024-09-18T00:20:03.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290940+0000", 2024-09-18T00:20:03.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9000000000000001e-06 2024-09-18T00:20:03.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290964+0000", 2024-09-18T00:20:03.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4273e-05 2024-09-18T00:20:03.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290982+0000", 2024-09-18T00:20:03.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8216e-05 2024-09-18T00:20:03.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291433+0000", 2024-09-18T00:20:03.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000451278 2024-09-18T00:20:03.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:03.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291446+0000", 2024-09-18T00:20:03.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3039e-05 2024-09-18T00:20:03.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316687+0000", 2024-09-18T00:20:03.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025240069 2024-09-18T00:20:03.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:03.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316701+0000", 2024-09-18T00:20:03.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4100000000000001e-05 2024-09-18T00:20:03.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314142+0000", 2024-09-18T00:20:03.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99744119799999997 2024-09-18T00:20:03.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314169+0000", 2024-09-18T00:20:03.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7636000000000001e-05 2024-09-18T00:20:03.137 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.137 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.137 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.138 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34545 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:03.138 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:20:03.138 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.828412520000001, 2024-09-18T00:20:03.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023257802, 2024-09-18T00:20:03.138 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.138 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.138 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:03.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.138 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34545 2024-09-18T00:20:03.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.138 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:20:03.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:20:03.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291008+0000", 2024-09-18T00:20:03.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.237e-06 2024-09-18T00:20:03.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291015+0000", 2024-09-18T00:20:03.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2030000000000003e-06 2024-09-18T00:20:03.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291035+0000", 2024-09-18T00:20:03.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.978e-05 2024-09-18T00:20:03.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291049+0000", 2024-09-18T00:20:03.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4429e-05 2024-09-18T00:20:03.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291630+0000", 2024-09-18T00:20:03.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00058103199999999999 2024-09-18T00:20:03.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:03.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291644+0000", 2024-09-18T00:20:03.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.342e-05 2024-09-18T00:20:03.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316753+0000", 2024-09-18T00:20:03.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025108703999999999 2024-09-18T00:20:03.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:03.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316766+0000", 2024-09-18T00:20:03.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3237e-05 2024-09-18T00:20:03.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314238+0000", 2024-09-18T00:20:03.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99747232699999999 2024-09-18T00:20:03.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314265+0000", 2024-09-18T00:20:03.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6432999999999999e-05 2024-09-18T00:20:03.143 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.143 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.143 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.143 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34546 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:03.143 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:20:03.143 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.828354181000002, 2024-09-18T00:20:03.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023324535, 2024-09-18T00:20:03.144 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.144 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.144 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:03.144 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.144 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.144 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.144 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34546 2024-09-18T00:20:03.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.144 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:20:03.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:20:03.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291066+0000", 2024-09-18T00:20:03.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.569999999999999e-07 2024-09-18T00:20:03.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291073+0000", 2024-09-18T00:20:03.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.063e-06 2024-09-18T00:20:03.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291090+0000", 2024-09-18T00:20:03.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6977e-05 2024-09-18T00:20:03.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291105+0000", 2024-09-18T00:20:03.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5166e-05 2024-09-18T00:20:03.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291771+0000", 2024-09-18T00:20:03.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00066599699999999999 2024-09-18T00:20:03.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:03.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291785+0000", 2024-09-18T00:20:03.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.327e-05 2024-09-18T00:20:03.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316826+0000", 2024-09-18T00:20:03.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025041955000000001 2024-09-18T00:20:03.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:03.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316840+0000", 2024-09-18T00:20:03.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3256e-05 2024-09-18T00:20:03.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314335+0000", 2024-09-18T00:20:03.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99749496100000001 2024-09-18T00:20:03.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314390+0000", 2024-09-18T00:20:03.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4932999999999997e-05 2024-09-18T00:20:03.148 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.148 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.149 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.149 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34547 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:03.149 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:20:03.149 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.828294882000002, 2024-09-18T00:20:03.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.028510467, 2024-09-18T00:20:03.149 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.149 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.149 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:03.149 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.149 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.149 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.149 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34547 2024-09-18T00:20:03.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.150 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:20:03.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:20:03.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291126+0000", 2024-09-18T00:20:03.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2699999999999999e-06 2024-09-18T00:20:03.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291150+0000", 2024-09-18T00:20:03.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4669999999999999e-05 2024-09-18T00:20:03.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291171+0000", 2024-09-18T00:20:03.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0706000000000001e-05 2024-09-18T00:20:03.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291186+0000", 2024-09-18T00:20:03.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5126999999999999e-05 2024-09-18T00:20:03.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291885+0000", 2024-09-18T00:20:03.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00069850299999999995 2024-09-18T00:20:03.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:03.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291897+0000", 2024-09-18T00:20:03.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-05 2024-09-18T00:20:03.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316887+0000", 2024-09-18T00:20:03.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024990155999999999 2024-09-18T00:20:03.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:03.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316900+0000", 2024-09-18T00:20:03.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.295e-05 2024-09-18T00:20:03.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319594+0000", 2024-09-18T00:20:03.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0026940719999999 2024-09-18T00:20:03.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319635+0000", 2024-09-18T00:20:03.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0583000000000001e-05 2024-09-18T00:20:03.154 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.154 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.154 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.154 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34548 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:03.154 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:20:03.154 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.828147424000001, 2024-09-18T00:20:03.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0285287400000001, 2024-09-18T00:20:03.154 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.155 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.155 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:03.155 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.155 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.155 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.155 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34548 2024-09-18T00:20:03.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.155 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:20:03.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:20:03.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291274+0000", 2024-09-18T00:20:03.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.12e-06 2024-09-18T00:20:03.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291288+0000", 2024-09-18T00:20:03.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4367e-05 2024-09-18T00:20:03.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:20:03.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5698999999999999e-05 2024-09-18T00:20:03.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291333+0000", 2024-09-18T00:20:03.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8783e-05 2024-09-18T00:20:03.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291987+0000", 2024-09-18T00:20:03.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00065417800000000005 2024-09-18T00:20:03.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:03.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.292000+0000", 2024-09-18T00:20:03.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2652e-05 2024-09-18T00:20:03.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316946+0000", 2024-09-18T00:20:03.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024946696000000001 2024-09-18T00:20:03.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:03.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316956+0000", 2024-09-18T00:20:03.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2760000000000006e-06 2024-09-18T00:20:03.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319752+0000", 2024-09-18T00:20:03.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002796083 2024-09-18T00:20:03.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319801+0000", 2024-09-18T00:20:03.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8885999999999999e-05 2024-09-18T00:20:03.159 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.159 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.160 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.160 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41950 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:03.160 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:20:03.160 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.425494714999999, 2024-09-18T00:20:03.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053368486, 2024-09-18T00:20:03.160 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.160 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.160 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:03.160 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.160 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.160 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.160 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41950 2024-09-18T00:20:03.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.161 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:20:03.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:20:03.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693926+0000", 2024-09-18T00:20:03.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4900000000000004e-07 2024-09-18T00:20:03.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693930+0000", 2024-09-18T00:20:03.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0130000000000004e-06 2024-09-18T00:20:03.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693940+0000", 2024-09-18T00:20:03.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0618000000000001e-05 2024-09-18T00:20:03.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693948+0000", 2024-09-18T00:20:03.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3140000000000004e-06 2024-09-18T00:20:03.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747133+0000", 2024-09-18T00:20:03.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053184689 2024-09-18T00:20:03.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:03.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747203+0000", 2024-09-18T00:20:03.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0339000000000005e-05 2024-09-18T00:20:03.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747293+0000", 2024-09-18T00:20:03.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9563999999999998e-05 2024-09-18T00:20:03.164 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.165 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.165 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.165 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41951 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:03.165 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:20:03.165 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.425459947999997, 2024-09-18T00:20:03.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053527695, 2024-09-18T00:20:03.165 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.165 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.165 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:03.165 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.165 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.166 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41951 2024-09-18T00:20:03.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.166 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:20:03.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:20:03.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693960+0000", 2024-09-18T00:20:03.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.16e-07 2024-09-18T00:20:03.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693971+0000", 2024-09-18T00:20:03.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1204e-05 2024-09-18T00:20:03.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693981+0000", 2024-09-18T00:20:03.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-18T00:20:03.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693989+0000", 2024-09-18T00:20:03.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0180000000000001e-06 2024-09-18T00:20:03.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747328+0000", 2024-09-18T00:20:03.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053338508 2024-09-18T00:20:03.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:03.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747400+0000", 2024-09-18T00:20:03.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2750999999999995e-05 2024-09-18T00:20:03.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747487+0000", 2024-09-18T00:20:03.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6527999999999996e-05 2024-09-18T00:20:03.169 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.169 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.169 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.169 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41952 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:03.169 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:20:03.169 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.425419540999997, 2024-09-18T00:20:03.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053728013999999998, 2024-09-18T00:20:03.170 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.170 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.170 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:03.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.170 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41952 2024-09-18T00:20:03.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.170 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:20:03.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:20:03.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694001+0000", 2024-09-18T00:20:03.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3799999999999996e-07 2024-09-18T00:20:03.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694004+0000", 2024-09-18T00:20:03.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.965e-06 2024-09-18T00:20:03.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694013+0000", 2024-09-18T00:20:03.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1689999999999998e-06 2024-09-18T00:20:03.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694022+0000", 2024-09-18T00:20:03.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7560000000000005e-06 2024-09-18T00:20:03.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747531+0000", 2024-09-18T00:20:03.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053508999000000002 2024-09-18T00:20:03.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:03.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747610+0000", 2024-09-18T00:20:03.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9212e-05 2024-09-18T00:20:03.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747728+0000", 2024-09-18T00:20:03.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011797499999999999 2024-09-18T00:20:03.174 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.174 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.174 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.174 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41953 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:03.174 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:20:03.174 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.425385912000003, 2024-09-18T00:20:03.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053936783000000002, 2024-09-18T00:20:03.174 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.174 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.174 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:03.174 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.174 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.175 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.175 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41953 2024-09-18T00:20:03.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.175 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:20:03.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:20:03.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694034+0000", 2024-09-18T00:20:03.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5199999999999996e-07 2024-09-18T00:20:03.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694037+0000", 2024-09-18T00:20:03.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.745e-06 2024-09-18T00:20:03.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694046+0000", 2024-09-18T00:20:03.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2490000000000007e-06 2024-09-18T00:20:03.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694054+0000", 2024-09-18T00:20:03.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1179999999999992e-06 2024-09-18T00:20:03.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747766+0000", 2024-09-18T00:20:03.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053711362999999998 2024-09-18T00:20:03.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:03.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747856+0000", 2024-09-18T00:20:03.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0779000000000006e-05 2024-09-18T00:20:03.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747970+0000", 2024-09-18T00:20:03.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011377700000000001 2024-09-18T00:20:03.178 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.178 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.178 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.178 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41954 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:03.178 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:20:03.178 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.425355758999999, 2024-09-18T00:20:03.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054104898999999998, 2024-09-18T00:20:03.179 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.179 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.179 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:03.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.179 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41954 2024-09-18T00:20:03.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.179 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:20:03.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:20:03.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:20:03.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4000000000000002e-07 2024-09-18T00:20:03.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694067+0000", 2024-09-18T00:20:03.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7329999999999999e-06 2024-09-18T00:20:03.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694076+0000", 2024-09-18T00:20:03.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2019999999999993e-06 2024-09-18T00:20:03.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694083+0000", 2024-09-18T00:20:03.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0029999999999997e-06 2024-09-18T00:20:03.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748003+0000", 2024-09-18T00:20:03.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053920463000000002 2024-09-18T00:20:03.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:03.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748077+0000", 2024-09-18T00:20:03.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4041000000000005e-05 2024-09-18T00:20:03.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748168+0000", 2024-09-18T00:20:03.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0916999999999997e-05 2024-09-18T00:20:03.182 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.182 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.182 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.183 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41955 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 231424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:03.183 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:20:03.183 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.425326921, 2024-09-18T00:20:03.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054272659000000001, 2024-09-18T00:20:03.183 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.183 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.183 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:03.183 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.183 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.183 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.183 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41955 2024-09-18T00:20:03.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.183 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:20:03.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:20:03.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694093+0000", 2024-09-18T00:20:03.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:20:03.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694109+0000", 2024-09-18T00:20:03.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6061000000000001e-05 2024-09-18T00:20:03.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694121+0000", 2024-09-18T00:20:03.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.216e-05 2024-09-18T00:20:03.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694129+0000", 2024-09-18T00:20:03.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0029999999999994e-06 2024-09-18T00:20:03.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748202+0000", 2024-09-18T00:20:03.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054072571 2024-09-18T00:20:03.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:03.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748274+0000", 2024-09-18T00:20:03.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2117000000000001e-05 2024-09-18T00:20:03.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748365+0000", 2024-09-18T00:20:03.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1139999999999998e-05 2024-09-18T00:20:03.187 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.187 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.187 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.187 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41956 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:03.187 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:20:03.187 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.425278712999997, 2024-09-18T00:20:03.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054420496999999998, 2024-09-18T00:20:03.187 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.187 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.187 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:03.188 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.188 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.188 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.188 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41956 2024-09-18T00:20:03.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.188 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:20:03.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:20:03.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:20:03.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8699999999999995e-07 2024-09-18T00:20:03.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694144+0000", 2024-09-18T00:20:03.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3510000000000002e-06 2024-09-18T00:20:03.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694154+0000", 2024-09-18T00:20:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0769999999999997e-06 2024-09-18T00:20:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694163+0000", 2024-09-18T00:20:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3300000000000005e-06 2024-09-18T00:20:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748397+0000", 2024-09-18T00:20:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054234497999999999 2024-09-18T00:20:03.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:03.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748471+0000", 2024-09-18T00:20:03.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3441000000000004e-05 2024-09-18T00:20:03.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748561+0000", 2024-09-18T00:20:03.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0212999999999998e-05 2024-09-18T00:20:03.191 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.191 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.191 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.192 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41957 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:03.192 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:20:03.192 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.425247460999998, 2024-09-18T00:20:03.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054590310000000003, 2024-09-18T00:20:03.192 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.192 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.192 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:03.192 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.192 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.192 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.192 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41957 2024-09-18T00:20:03.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.193 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:20:03.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:20:03.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694173+0000", 2024-09-18T00:20:03.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2500000000000005e-07 2024-09-18T00:20:03.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694175+0000", 2024-09-18T00:20:03.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.655e-06 2024-09-18T00:20:03.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694184+0000", 2024-09-18T00:20:03.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7630000000000006e-06 2024-09-18T00:20:03.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694191+0000", 2024-09-18T00:20:03.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8870000000000003e-06 2024-09-18T00:20:03.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748593+0000", 2024-09-18T00:20:03.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054402525 2024-09-18T00:20:03.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:03.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748669+0000", 2024-09-18T00:20:03.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5532999999999997e-05 2024-09-18T00:20:03.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748762+0000", 2024-09-18T00:20:03.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3221999999999996e-05 2024-09-18T00:20:03.196 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.196 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.196 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.196 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42392 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:03.196 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:20:03.196 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.412218021999998, 2024-09-18T00:20:03.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053378162, 2024-09-18T00:20:03.197 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.197 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.197 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:03.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.197 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42392 2024-09-18T00:20:03.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.197 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:20:03.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:20:03.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707430+0000", 2024-09-18T00:20:03.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022838599999999999 2024-09-18T00:20:03.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707434+0000", 2024-09-18T00:20:03.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5220000000000001e-06 2024-09-18T00:20:03.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707467+0000", 2024-09-18T00:20:03.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2570000000000002e-05 2024-09-18T00:20:03.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707475+0000", 2024-09-18T00:20:03.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:20:03.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760446+0000", 2024-09-18T00:20:03.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052971635000000003 2024-09-18T00:20:03.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:03.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760512+0000", 2024-09-18T00:20:03.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5679999999999995e-05 2024-09-18T00:20:03.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760579+0000", 2024-09-18T00:20:03.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7296000000000001e-05 2024-09-18T00:20:03.201 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.201 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.201 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.201 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42393 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 358400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:03.201 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:20:03.201 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.411929071000003, 2024-09-18T00:20:03.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053269618999999997, 2024-09-18T00:20:03.201 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.201 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.202 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:03.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.202 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42393 2024-09-18T00:20:03.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.202 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:20:03.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:20:03.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707491+0000", 2024-09-18T00:20:03.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1399999999999997e-07 2024-09-18T00:20:03.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707503+0000", 2024-09-18T00:20:03.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1912e-05 2024-09-18T00:20:03.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707512+0000", 2024-09-18T00:20:03.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1030000000000008e-06 2024-09-18T00:20:03.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707520+0000", 2024-09-18T00:20:03.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1249999999999993e-06 2024-09-18T00:20:03.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760605+0000", 2024-09-18T00:20:03.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053085495000000003 2024-09-18T00:20:03.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:03.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760680+0000", 2024-09-18T00:20:03.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5024000000000005e-05 2024-09-18T00:20:03.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760760+0000", 2024-09-18T00:20:03.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9345999999999996e-05 2024-09-18T00:20:03.206 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.206 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.206 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.206 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42394 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:03.206 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:20:03.206 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.411889082999998, 2024-09-18T00:20:03.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053403144999999999, 2024-09-18T00:20:03.206 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.206 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.206 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:03.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.207 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.207 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.207 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42394 2024-09-18T00:20:03.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.207 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:20:03.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:20:03.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711550+0000", 2024-09-18T00:20:03.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040200189999999997 2024-09-18T00:20:03.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711556+0000", 2024-09-18T00:20:03.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2339999999999999e-06 2024-09-18T00:20:03.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711569+0000", 2024-09-18T00:20:03.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2337999999999999e-05 2024-09-18T00:20:03.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711577+0000", 2024-09-18T00:20:03.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5660000000000007e-06 2024-09-18T00:20:03.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760790+0000", 2024-09-18T00:20:03.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049212196999999999 2024-09-18T00:20:03.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:03.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760857+0000", 2024-09-18T00:20:03.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7756000000000007e-05 2024-09-18T00:20:03.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760933+0000", 2024-09-18T00:20:03.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6035e-05 2024-09-18T00:20:03.210 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.210 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.210 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.211 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:55489 3.5 3:a5f1e468:::rbd_data.112a859a01c9.000000000000000c:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T00:20:03.211 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:20:03.211 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.488034957, 2024-09-18T00:20:03.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.05748731, 2024-09-18T00:20:03.211 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.211 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.211 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:03.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.211 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 55489 2024-09-18T00:20:03.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.211 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:20:03.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:20:03.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631388+0000", 2024-09-18T00:20:03.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6169999999999998e-06 2024-09-18T00:20:03.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631405+0000", 2024-09-18T00:20:03.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6643e-05 2024-09-18T00:20:03.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631444+0000", 2024-09-18T00:20:03.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9226e-05 2024-09-18T00:20:03.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631465+0000", 2024-09-18T00:20:03.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0826999999999999e-05 2024-09-18T00:20:03.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631578+0000", 2024-09-18T00:20:03.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011291899999999999 2024-09-18T00:20:03.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:03.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631651+0000", 2024-09-18T00:20:03.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3264999999999997e-05 2024-09-18T00:20:03.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:20:03.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.634073+0000", 2024-09-18T00:20:03.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0024221350000000002 2024-09-18T00:20:03.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:03.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688732+0000", 2024-09-18T00:20:03.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054658610000000003 2024-09-18T00:20:03.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:03.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688794+0000", 2024-09-18T00:20:03.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2255999999999995e-05 2024-09-18T00:20:03.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:03.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688834+0000", 2024-09-18T00:20:03.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9916999999999997e-05 2024-09-18T00:20:03.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688872+0000", 2024-09-18T00:20:03.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7895000000000002e-05 2024-09-18T00:20:03.216 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.216 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.217 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.217 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:58361 3.1d 3:bac90a3d:::rbd_data.112a17f95ba0.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T00:20:03.217 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:20:03.217 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.915689463, 2024-09-18T00:20:03.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101552963, 2024-09-18T00:20:03.217 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.217 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.217 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:03.217 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:03.217 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:03.217 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:03.217 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 58361 2024-09-18T00:20:03.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.218 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:20:03.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:20:03.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203733+0000", 2024-09-18T00:20:03.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.01e-06 2024-09-18T00:20:03.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203745+0000", 2024-09-18T00:20:03.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2513e-05 2024-09-18T00:20:03.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203777+0000", 2024-09-18T00:20:03.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1216000000000001e-05 2024-09-18T00:20:03.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203792+0000", 2024-09-18T00:20:03.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586e-05 2024-09-18T00:20:03.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203911+0000", 2024-09-18T00:20:03.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119234 2024-09-18T00:20:03.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:03.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.204056+0000", 2024-09-18T00:20:03.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000144943 2024-09-18T00:20:03.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:03.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.210219+0000", 2024-09-18T00:20:03.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061630649999999997 2024-09-18T00:20:03.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:03.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305071+0000", 2024-09-18T00:20:03.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.094852120999999998 2024-09-18T00:20:03.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:03.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305106+0000", 2024-09-18T00:20:03.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4848999999999998e-05 2024-09-18T00:20:03.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:03.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305208+0000", 2024-09-18T00:20:03.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101518 2024-09-18T00:20:03.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305283+0000", 2024-09-18T00:20:03.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4907999999999995e-05 2024-09-18T00:20:03.222 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.222 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.222 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.223 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:58361 3.1d e96/79 ondisk, result = 0)", 2024-09-18T00:20:03.223 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:20:03.223 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.888206422, 2024-09-18T00:20:03.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.073876681, 2024-09-18T00:20:03.223 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:03.223 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:03.223 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:03.223 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:03.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:03.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:20:03.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:03.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:20:03.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:03.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:03.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231215+0000", 2024-09-18T00:20:03.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9570000000000001e-06 2024-09-18T00:20:03.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:03.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231223+0000", 2024-09-18T00:20:03.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0900000000000005e-06 2024-09-18T00:20:03.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:03.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231252+0000", 2024-09-18T00:20:03.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9552000000000001e-05 2024-09-18T00:20:03.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:03.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231267+0000", 2024-09-18T00:20:03.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4263e-05 2024-09-18T00:20:03.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:03.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.304871+0000", 2024-09-18T00:20:03.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.073604257000000006 2024-09-18T00:20:03.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:03.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.304889+0000", 2024-09-18T00:20:03.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.808e-05 2024-09-18T00:20:03.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:03.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:03.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:03.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305090+0000", 2024-09-18T00:20:03.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020048200000000001 2024-09-18T00:20:03.226 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.227 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.227 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.227 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:03.227 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:03.227 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:03.228 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:20:03.229 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:20:03.224+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:20:03.329 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:03.325+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:03.356 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:03.356 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:03.356 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:20:03.357 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:03.370 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:20:03.430 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:03.428+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:03.455 INFO:teuthology.orchestra.run.smithi097.stderr:marked in osd.1. 2024-09-18T00:20:03.466 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:20:03.496 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-18T00:20:03.505 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:20:03.505 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:20:03.505 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:03.520 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:20:03.530 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:03.529+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:03.632 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:03.629+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:03.683 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:03.683 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:20:03.683 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:20:03.683 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [] 2024-09-18T00:20:03.683 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:03.697 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:20:03.733 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:03.729+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:03.813 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:03.813 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:03.813 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:20:03.813 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:03.826 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:20:03.834 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:03.833+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:03.935 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:03.933+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:03.947 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:03.947 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:03.948 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:20:03.948 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:20:03.948 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:03.962 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:20:04.036 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:04.033+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:04.080 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:20:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:20:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:20:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2230 3.17 3:ef492888:::rbd_data.112aa7ba4578.000000000000014f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:20:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:20:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 103.812696952, 2024-09-18T00:20:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075065283999999996, 2024-09-18T00:20:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:04.081 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2230 2024-09-18T00:20:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:20:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:20:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.082 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266014+0000", 2024-09-18T00:20:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:20:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266029+0000", 2024-09-18T00:20:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4616e-05 2024-09-18T00:20:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266053+0000", 2024-09-18T00:20:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4369999999999999e-05 2024-09-18T00:20:04.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266067+0000", 2024-09-18T00:20:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4243999999999999e-05 2024-09-18T00:20:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266157+0000", 2024-09-18T00:20:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9909999999999998e-05 2024-09-18T00:20:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266327+0000", 2024-09-18T00:20:04.084 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016981200000000001 2024-09-18T00:20:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:20:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269180+0000", 2024-09-18T00:20:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028526559999999999 2024-09-18T00:20:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340955+0000", 2024-09-18T00:20:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.071774904 2024-09-18T00:20:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:04.085 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340974+0000", 2024-09-18T00:20:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8824000000000001e-05 2024-09-18T00:20:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341028+0000", 2024-09-18T00:20:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4456999999999997e-05 2024-09-18T00:20:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341076+0000", 2024-09-18T00:20:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8284000000000001e-05 2024-09-18T00:20:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.086 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2230 3.17 e36/23 ondisk, result = 0)", 2024-09-18T00:20:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:20:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 103.795353371, 2024-09-18T00:20:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057609645000000001, 2024-09-18T00:20:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.087 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:20:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:20:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283356+0000", 2024-09-18T00:20:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.44e-06 2024-09-18T00:20:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.088 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283362+0000", 2024-09-18T00:20:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2539999999999998e-06 2024-09-18T00:20:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283382+0000", 2024-09-18T00:20:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9896000000000001e-05 2024-09-18T00:20:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283398+0000", 2024-09-18T00:20:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5489999999999999e-05 2024-09-18T00:20:04.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340936+0000", 2024-09-18T00:20:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057538649999999997 2024-09-18T00:20:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340947+0000", 2024-09-18T00:20:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0981e-05 2024-09-18T00:20:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340964+0000", 2024-09-18T00:20:04.090 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6934000000000001e-05 2024-09-18T00:20:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2419 3.17 3:ed94b133:::rbd_data.112aa7ba4578.00000000000001b5:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:20:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:20:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 103.341231898, 2024-09-18T00:20:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.064761812000000002, 2024-09-18T00:20:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:04.091 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2419 2024-09-18T00:20:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:20:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.092 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:20:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737479+0000", 2024-09-18T00:20:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6089999999999999e-06 2024-09-18T00:20:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737487+0000", 2024-09-18T00:20:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7900000000000005e-06 2024-09-18T00:20:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.093 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737507+0000", 2024-09-18T00:20:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9936e-05 2024-09-18T00:20:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737518+0000", 2024-09-18T00:20:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1046999999999999e-05 2024-09-18T00:20:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737599+0000", 2024-09-18T00:20:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0697000000000003e-05 2024-09-18T00:20:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.094 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737697+0000", 2024-09-18T00:20:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8058000000000002e-05 2024-09-18T00:20:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:20:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.740445+0000", 2024-09-18T00:20:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027481279999999999 2024-09-18T00:20:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.096 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802023+0000", 2024-09-18T00:20:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.061577321999999997 2024-09-18T00:20:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802060+0000", 2024-09-18T00:20:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7586000000000002e-05 2024-09-18T00:20:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802158+0000", 2024-09-18T00:20:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7583000000000004e-05 2024-09-18T00:20:04.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802238+0000", 2024-09-18T00:20:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0055999999999994e-05 2024-09-18T00:20:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2419 3.17 e37/23 ondisk, result = 0)", 2024-09-18T00:20:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:20:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 103.329006952, 2024-09-18T00:20:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052342948, 2024-09-18T00:20:04.098 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:20:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:20:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749702+0000", 2024-09-18T00:20:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:20:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749710+0000", 2024-09-18T00:20:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7279999999999995e-06 2024-09-18T00:20:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749736+0000", 2024-09-18T00:20:04.100 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5797999999999999e-05 2024-09-18T00:20:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749751+0000", 2024-09-18T00:20:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5238999999999999e-05 2024-09-18T00:20:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801745+0000", 2024-09-18T00:20:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051994022000000001 2024-09-18T00:20:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.101 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801770+0000", 2024-09-18T00:20:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5103e-05 2024-09-18T00:20:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802044+0000", 2024-09-18T00:20:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027395799999999998 2024-09-18T00:20:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6503 3.1b 3:d9387a1f:::rbd_data.112a3a39fc45.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:20:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:20:04.102 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 98.514333981999997, 2024-09-18T00:20:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050510547000000003, 2024-09-18T00:20:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6503 2024-09-18T00:20:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:20:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:20:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564376+0000", 2024-09-18T00:20:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-06 2024-09-18T00:20:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564386+0000", 2024-09-18T00:20:04.104 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0312999999999999e-05 2024-09-18T00:20:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564408+0000", 2024-09-18T00:20:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2001000000000002e-05 2024-09-18T00:20:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564421+0000", 2024-09-18T00:20:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2271e-05 2024-09-18T00:20:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.105 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564491+0000", 2024-09-18T00:20:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0630000000000006e-05 2024-09-18T00:20:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564606+0000", 2024-09-18T00:20:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011449 2024-09-18T00:20:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.566743+0000", 2024-09-18T00:20:04.106 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021373830000000001 2024-09-18T00:20:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614738+0000", 2024-09-18T00:20:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047995359000000001 2024-09-18T00:20:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614759+0000", 2024-09-18T00:20:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1010999999999999e-05 2024-09-18T00:20:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:04.107 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614826+0000", 2024-09-18T00:20:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6654000000000003e-05 2024-09-18T00:20:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614884+0000", 2024-09-18T00:20:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8332999999999999e-05 2024-09-18T00:20:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6518 3.1 3:8061bf9f:::rbd_data.112a3a39fc45.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:20:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:20:04.108 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 98.427895707999994, 2024-09-18T00:20:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062303123000000002, 2024-09-18T00:20:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6518 2024-09-18T00:20:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:20:04.109 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:20:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650814+0000", 2024-09-18T00:20:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.629e-06 2024-09-18T00:20:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.110 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650823+0000", 2024-09-18T00:20:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1800000000000002e-06 2024-09-18T00:20:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650843+0000", 2024-09-18T00:20:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9836000000000001e-05 2024-09-18T00:20:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650854+0000", 2024-09-18T00:20:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1564e-05 2024-09-18T00:20:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.111 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650917+0000", 2024-09-18T00:20:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3076000000000004e-05 2024-09-18T00:20:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.651029+0000", 2024-09-18T00:20:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111154 2024-09-18T00:20:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.655531+0000", 2024-09-18T00:20:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045027629999999999 2024-09-18T00:20:04.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712924+0000", 2024-09-18T00:20:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057393006000000003 2024-09-18T00:20:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712988+0000", 2024-09-18T00:20:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000005e-05 2024-09-18T00:20:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713056+0000", 2024-09-18T00:20:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7904999999999994e-05 2024-09-18T00:20:04.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713115+0000", 2024-09-18T00:20:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9339999999999998e-05 2024-09-18T00:20:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6518 3.1 e42/23 ondisk, result = 0)", 2024-09-18T00:20:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:20:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 98.415470624999998, 2024-09-18T00:20:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049887304, 2024-09-18T00:20:04.114 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:20:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:20:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.115 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663239+0000", 2024-09-18T00:20:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:20:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663246+0000", 2024-09-18T00:20:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4189999999999996e-06 2024-09-18T00:20:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663265+0000", 2024-09-18T00:20:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8323000000000001e-05 2024-09-18T00:20:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.116 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663277+0000", 2024-09-18T00:20:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2065999999999999e-05 2024-09-18T00:20:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712965+0000", 2024-09-18T00:20:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049688317000000003 2024-09-18T00:20:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712977+0000", 2024-09-18T00:20:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.206e-05 2024-09-18T00:20:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.117 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713125+0000", 2024-09-18T00:20:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014749899999999999 2024-09-18T00:20:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:8831 3.12 3:49cac8cf:::rbd_data.112a9f4259df.0000000000000130:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T00:20:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:20:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 91.797610375999994, 2024-09-18T00:20:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104125047, 2024-09-18T00:20:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.118 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 8831 2024-09-18T00:20:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:20:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.119 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:20:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281099+0000", 2024-09-18T00:20:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.029e-06 2024-09-18T00:20:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281103+0000", 2024-09-18T00:20:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7759999999999997e-06 2024-09-18T00:20:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.120 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281114+0000", 2024-09-18T00:20:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1090000000000001e-05 2024-09-18T00:20:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281120+0000", 2024-09-18T00:20:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.604e-06 2024-09-18T00:20:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281164+0000", 2024-09-18T00:20:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3696999999999999e-05 2024-09-18T00:20:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.121 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281270+0000", 2024-09-18T00:20:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106041 2024-09-18T00:20:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.285596+0000", 2024-09-18T00:20:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043266659999999998 2024-09-18T00:20:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385028+0000", 2024-09-18T00:20:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.099431808999999996 2024-09-18T00:20:04.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385094+0000", 2024-09-18T00:20:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5740000000000004e-05 2024-09-18T00:20:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385166+0000", 2024-09-18T00:20:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1637000000000005e-05 2024-09-18T00:20:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385223+0000", 2024-09-18T00:20:04.123 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6957999999999999e-05 2024-09-18T00:20:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:8831 3.12 e48/32 ondisk, result = 0)", 2024-09-18T00:20:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:20:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 91.781096266000006, 2024-09-18T00:20:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087621558000000002, 2024-09-18T00:20:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.124 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:20:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:20:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297613+0000", 2024-09-18T00:20:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3400000000000001e-06 2024-09-18T00:20:04.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297620+0000", 2024-09-18T00:20:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5869999999999998e-06 2024-09-18T00:20:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297639+0000", 2024-09-18T00:20:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9888999999999999e-05 2024-09-18T00:20:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297661+0000", 2024-09-18T00:20:04.126 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1772000000000001e-05 2024-09-18T00:20:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385073+0000", 2024-09-18T00:20:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087411303999999995 2024-09-18T00:20:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385086+0000", 2024-09-18T00:20:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3040000000000001e-05 2024-09-18T00:20:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.127 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385233+0000", 2024-09-18T00:20:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014762600000000001 2024-09-18T00:20:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22269 3.17 3:e9a1eb35:::rbd_data.112a948d5956.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:20:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:20:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 66.169564440000002, 2024-09-18T00:20:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.063462693000000001, 2024-09-18T00:20:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:04.128 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22269 2024-09-18T00:20:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:20:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:20:04.129 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909146+0000", 2024-09-18T00:20:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3700000000000002e-06 2024-09-18T00:20:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909160+0000", 2024-09-18T00:20:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3825000000000001e-05 2024-09-18T00:20:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909178+0000", 2024-09-18T00:20:04.130 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8567999999999999e-05 2024-09-18T00:20:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909189+0000", 2024-09-18T00:20:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0881e-05 2024-09-18T00:20:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909270+0000", 2024-09-18T00:20:04.131 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0846999999999994e-05 2024-09-18T00:20:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909383+0000", 2024-09-18T00:20:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112704 2024-09-18T00:20:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972310+0000", 2024-09-18T00:20:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062927081999999995 2024-09-18T00:20:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972433+0000", 2024-09-18T00:20:04.132 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-18T00:20:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972606+0000", 2024-09-18T00:20:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017317799999999999 2024-09-18T00:20:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22379 3.17 3:e9a1eb35:::rbd_data.112a948d5956.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:20:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:20:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 65.675489142999993, 2024-09-18T00:20:04.133 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.12560279099999999, 2024-09-18T00:20:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22379 2024-09-18T00:20:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:20:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.134 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:20:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403221+0000", 2024-09-18T00:20:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3269999999999999e-06 2024-09-18T00:20:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403232+0000", 2024-09-18T00:20:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1392e-05 2024-09-18T00:20:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.135 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403264+0000", 2024-09-18T00:20:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1146999999999998e-05 2024-09-18T00:20:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403279+0000", 2024-09-18T00:20:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5109000000000001e-05 2024-09-18T00:20:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403424+0000", 2024-09-18T00:20:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014516999999999999 2024-09-18T00:20:04.136 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403567+0000", 2024-09-18T00:20:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142824 2024-09-18T00:20:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528723+0000", 2024-09-18T00:20:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.12515647299999999 2024-09-18T00:20:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528764+0000", 2024-09-18T00:20:04.137 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0624000000000003e-05 2024-09-18T00:20:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528822+0000", 2024-09-18T00:20:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7725e-05 2024-09-18T00:20:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39268 3.1f 3:fdf3a436:::rbd_data.112a3496573c.0000000000000001:17 [read 352256~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.576221+0000", 2024-09-18T00:20:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 43.502487100000003, 2024-09-18T00:20:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047293515000000001, 2024-09-18T00:20:04.138 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39268 2024-09-18T00:20:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576221+0000", 2024-09-18T00:20:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576221+0000", 2024-09-18T00:20:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576221+0000", 2024-09-18T00:20:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-07 2024-09-18T00:20:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576229+0000", 2024-09-18T00:20:04.140 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9950000000000005e-06 2024-09-18T00:20:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576237+0000", 2024-09-18T00:20:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0010000000000001e-06 2024-09-18T00:20:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576243+0000", 2024-09-18T00:20:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.592e-06 2024-09-18T00:20:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.141 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623336+0000", 2024-09-18T00:20:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047092698000000002 2024-09-18T00:20:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623418+0000", 2024-09-18T00:20:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1977000000000005e-05 2024-09-18T00:20:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623514+0000", 2024-09-18T00:20:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6676999999999996e-05 2024-09-18T00:20:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39269 3.1f 3:fdf3a436:::rbd_data.112a3496573c.0000000000000001:17 [read 362496~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.576250+0000", 2024-09-18T00:20:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 43.502458189000002, 2024-09-18T00:20:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047454169999999997, 2024-09-18T00:20:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39269 2024-09-18T00:20:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.143 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576250+0000", 2024-09-18T00:20:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576250+0000", 2024-09-18T00:20:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576250+0000", 2024-09-18T00:20:04.144 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:20:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576252+0000", 2024-09-18T00:20:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1739999999999999e-06 2024-09-18T00:20:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576258+0000", 2024-09-18T00:20:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6670000000000001e-06 2024-09-18T00:20:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576264+0000", 2024-09-18T00:20:04.145 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1360000000000002e-06 2024-09-18T00:20:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623543+0000", 2024-09-18T00:20:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047278496000000003 2024-09-18T00:20:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623609+0000", 2024-09-18T00:20:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5901000000000005e-05 2024-09-18T00:20:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623704+0000", 2024-09-18T00:20:04.146 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5291999999999995e-05 2024-09-18T00:20:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39270 3.1f 3:fdf3a436:::rbd_data.112a3496573c.0000000000000001:17 [read 372736~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.576271+0000", 2024-09-18T00:20:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 43.502436860000003, 2024-09-18T00:20:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047626546999999998, 2024-09-18T00:20:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:04.147 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39270 2024-09-18T00:20:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576271+0000", 2024-09-18T00:20:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576271+0000", 2024-09-18T00:20:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.148 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576271+0000", 2024-09-18T00:20:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.01e-07 2024-09-18T00:20:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576274+0000", 2024-09-18T00:20:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1150000000000001e-06 2024-09-18T00:20:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576280+0000", 2024-09-18T00:20:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9399999999999999e-06 2024-09-18T00:20:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.149 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576285+0000", 2024-09-18T00:20:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0719999999999999e-06 2024-09-18T00:20:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623731+0000", 2024-09-18T00:20:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047446629999999997 2024-09-18T00:20:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623809+0000", 2024-09-18T00:20:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8141999999999998e-05 2024-09-18T00:20:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.150 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623898+0000", 2024-09-18T00:20:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8246999999999997e-05 2024-09-18T00:20:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39360 3.1f 3:fec8fa94:::rbd_data.112a3496573c.0000000000000006:17 [read 354304~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.579954+0000", 2024-09-18T00:20:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 43.498754011999999, 2024-09-18T00:20:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047250546999999997, 2024-09-18T00:20:04.151 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39360 2024-09-18T00:20:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579954+0000", 2024-09-18T00:20:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.152 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579954+0000", 2024-09-18T00:20:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579954+0000", 2024-09-18T00:20:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7800000000000002e-07 2024-09-18T00:20:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579956+0000", 2024-09-18T00:20:04.153 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.114e-06 2024-09-18T00:20:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579962+0000", 2024-09-18T00:20:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7400000000000001e-06 2024-09-18T00:20:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579967+0000", 2024-09-18T00:20:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.792e-06 2024-09-18T00:20:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.154 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627056+0000", 2024-09-18T00:20:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047089131999999999 2024-09-18T00:20:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627122+0000", 2024-09-18T00:20:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6229000000000006e-05 2024-09-18T00:20:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627204+0000", 2024-09-18T00:20:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2062000000000001e-05 2024-09-18T00:20:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.155 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39361 3.1f 3:fec8fa94:::rbd_data.112a3496573c.0000000000000006:17 [read 364544~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.579975+0000", 2024-09-18T00:20:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 43.498732855999997, 2024-09-18T00:20:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047405364999999998, 2024-09-18T00:20:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:04.156 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39361 2024-09-18T00:20:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579975+0000", 2024-09-18T00:20:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579975+0000", 2024-09-18T00:20:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579976+0000", 2024-09-18T00:20:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7399999999999998e-07 2024-09-18T00:20:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579978+0000", 2024-09-18T00:20:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5359999999999999e-06 2024-09-18T00:20:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579984+0000", 2024-09-18T00:20:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2469999999999997e-06 2024-09-18T00:20:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579989+0000", 2024-09-18T00:20:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6700000000000002e-06 2024-09-18T00:20:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627232+0000", 2024-09-18T00:20:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047243016999999998 2024-09-18T00:20:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627299+0000", 2024-09-18T00:20:04.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7145999999999997e-05 2024-09-18T00:20:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627380+0000", 2024-09-18T00:20:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1274999999999997e-05 2024-09-18T00:20:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39473 3.1f 3:fd448eba:::rbd_data.112a3496573c.000000000000000c:17 [read 366592~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.583299+0000", 2024-09-18T00:20:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 43.495408535000003, 2024-09-18T00:20:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047411813999999997, 2024-09-18T00:20:04.160 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39473 2024-09-18T00:20:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583299+0000", 2024-09-18T00:20:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583299+0000", 2024-09-18T00:20:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583300+0000", 2024-09-18T00:20:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6699999999999999e-07 2024-09-18T00:20:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583302+0000", 2024-09-18T00:20:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.979e-06 2024-09-18T00:20:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583308+0000", 2024-09-18T00:20:04.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.835e-06 2024-09-18T00:20:04.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583313+0000", 2024-09-18T00:20:04.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9409999999999998e-06 2024-09-18T00:20:04.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.630537+0000", 2024-09-18T00:20:04.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047224592000000003 2024-09-18T00:20:04.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.630623+0000", 2024-09-18T00:20:04.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5473e-05 2024-09-18T00:20:04.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.630711+0000", 2024-09-18T00:20:04.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8527000000000001e-05 2024-09-18T00:20:04.164 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.164 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.164 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.164 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:54153 3.b e90/23 3:d3f7b2f3:::rbd_data.112a859a01c9.0000000000000001:head v 90'238, mlcod=90'236)", 2024-09-18T00:20:04.165 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:30.858396+0000", 2024-09-18T00:20:04.165 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 33.220312167000003, 2024-09-18T00:20:04.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056676272999999999, 2024-09-18T00:20:04.165 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.165 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.165 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:04.165 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.165 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.165 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858396+0000", 2024-09-18T00:20:04.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858396+0000", 2024-09-18T00:20:04.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858398+0000", 2024-09-18T00:20:04.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.838e-06 2024-09-18T00:20:04.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860747+0000", 2024-09-18T00:20:04.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0023496160000000001 2024-09-18T00:20:04.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860777+0000", 2024-09-18T00:20:04.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9754e-05 2024-09-18T00:20:04.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860791+0000", 2024-09-18T00:20:04.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.377e-05 2024-09-18T00:20:04.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860879+0000", 2024-09-18T00:20:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7818999999999994e-05 2024-09-18T00:20:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860917+0000", 2024-09-18T00:20:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8124999999999998e-05 2024-09-18T00:20:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915020+0000", 2024-09-18T00:20:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.054103126000000001 2024-09-18T00:20:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915072+0000", 2024-09-18T00:20:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2225000000000002e-05 2024-09-18T00:20:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:58293 3.1b 3:d96f405d:::rbd_data.112a17f95ba0.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T00:20:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:38.838123+0000", 2024-09-18T00:20:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 25.240584524999999, 2024-09-18T00:20:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11992721000000001, 2024-09-18T00:20:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:04.169 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:04.170 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:04.170 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:04.170 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 58293 2024-09-18T00:20:04.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.170 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838123+0000", 2024-09-18T00:20:04.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838123+0000", 2024-09-18T00:20:04.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838125+0000", 2024-09-18T00:20:04.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.158e-06 2024-09-18T00:20:04.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838130+0000", 2024-09-18T00:20:04.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7019999999999998e-06 2024-09-18T00:20:04.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.171 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.171 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838145+0000", 2024-09-18T00:20:04.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4871999999999999e-05 2024-09-18T00:20:04.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838152+0000", 2024-09-18T00:20:04.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3309999999999998e-06 2024-09-18T00:20:04.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838227+0000", 2024-09-18T00:20:04.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4685999999999996e-05 2024-09-18T00:20:04.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838375+0000", 2024-09-18T00:20:04.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014751 2024-09-18T00:20:04.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:04.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.845187+0000", 2024-09-18T00:20:04.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0068119749999999996 2024-09-18T00:20:04.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:04.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.957945+0000", 2024-09-18T00:20:04.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.112758517 2024-09-18T00:20:04.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:04.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.957963+0000", 2024-09-18T00:20:04.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7580000000000001e-05 2024-09-18T00:20:04.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:04.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.958009+0000", 2024-09-18T00:20:04.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6377e-05 2024-09-18T00:20:04.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.958051+0000", 2024-09-18T00:20:04.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1501999999999997e-05 2024-09-18T00:20:04.174 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.174 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.174 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.175 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:58293 3.1b e96/23 ondisk, result = 0)", 2024-09-18T00:20:04.175 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:38.867986+0000", 2024-09-18T00:20:04.175 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 25.210722410999999, 2024-09-18T00:20:04.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.089969120999999999, 2024-09-18T00:20:04.175 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:04.175 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:04.175 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:04.175 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:04.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:04.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.867986+0000", 2024-09-18T00:20:04.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:04.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.867986+0000", 2024-09-18T00:20:04.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:04.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:04.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.867987+0000", 2024-09-18T00:20:04.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9649999999999998e-06 2024-09-18T00:20:04.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.176 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:04.176 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.867996+0000", 2024-09-18T00:20:04.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3259999999999996e-06 2024-09-18T00:20:04.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:04.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.868030+0000", 2024-09-18T00:20:04.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4428999999999999e-05 2024-09-18T00:20:04.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:04.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.868050+0000", 2024-09-18T00:20:04.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9368000000000002e-05 2024-09-18T00:20:04.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.177 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:04.177 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.957905+0000", 2024-09-18T00:20:04.177 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.089855395000000005 2024-09-18T00:20:04.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:04.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.957915+0000", 2024-09-18T00:20:04.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7280000000000007e-06 2024-09-18T00:20:04.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:04.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:04.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:04.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.957955+0000", 2024-09-18T00:20:04.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9910000000000002e-05 2024-09-18T00:20:04.178 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.178 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.178 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.178 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:04.179 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:04.179 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:04.180 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:20:04.181 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:20:04.176+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:20:04.240 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:20:04.281 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:04.281+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:04.352 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:04.558 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:04.558 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:20:04.558 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:04.560 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:20:04.560 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:20:04.561 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:20:04.561 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:20:04.561 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:20:04.562 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:20:04.562 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:20:04.562 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is set 2024-09-18T00:20:04.563 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:20:04.563 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:20:04.564 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:20:04.564 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:20:04.564 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:20:04.564 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:20:04.565 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:20:04.565 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:20:04.565 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:20:04.566 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:20:04.566 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:04.556+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:04.660 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:04.657+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:04.739 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:04.740 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:04.754 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:20:04.762 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:04.757+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:04.862 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:20:04.861+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:20:04.875 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:04.875 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:04.889 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:20:04.963 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:20:04.960+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:20:05.010 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:05.010 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:05.023 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:20:05.066 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:20:05.064+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:20:05.165 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:05.161+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:05.188 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:05.188 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:05.202 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:20:05.266 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:05.265+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:05.325 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:05.325 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:05.339 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:20:05.386 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:05.385+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:05.467 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:20:05.464+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:20:05.474 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:05.474 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:05.492 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:20:05.568 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:05.564+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:05.664 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:05.664 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:05.669 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:05.664+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:05.680 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:20:05.770 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:20:05.768+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:20:05.809 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:05.810 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:05.823 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:20:05.871 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:05.868+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:05.946 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:05.946 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:05.961 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:20:05.972 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:05.968+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:06.073 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:06.072+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:06.083 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:06.083 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:06.097 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:20:06.174 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:06.172+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:06.217 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:06.217 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:06.231 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:20:06.278 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:06.277+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:06.351 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:06.351 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:06.365 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:20:06.376 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:06.372+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:06.476 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:06.473+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:06.529 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:06.529 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:06.543 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:20:06.564 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T00:20:06.577 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:20:06.610 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:06.608+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:06.677 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:06.673+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:06.708 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:06.708 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:06.722 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:20:06.777 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:20:06.776+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:20:06.778 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:20:06.778 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:20:06.797 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1: {} 2024-09-18T00:20:06.798 INFO:teuthology.orchestra.run.smithi097.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T00:20:06.822 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:20:06.822 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:20:06.841 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:20:06.841 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:20:06.863 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:20:06.864 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:20:06.878 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:06.877+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:06.886 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:20:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:20:06.888 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:06.888 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:06.888 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:20:06.889 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:06.903 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:20:06.907 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:20:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:20:06.931 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:20:06.931 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:20:06.993 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:06.992+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:07.070 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:07.071 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:07.071 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:20:07.071 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:20:07.071 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:07.080 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:07.077+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:07.085 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:20:07.181 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:07.176+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:07.250 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:07.250 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:20:07.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:20:07.250 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:20:07.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.251 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34543 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:07.251 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:20:07.251 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.956315281999998, 2024-09-18T00:20:07.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023207593, 2024-09-18T00:20:07.251 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.251 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.251 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:07.251 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.251 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.251 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.251 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34543 2024-09-18T00:20:07.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.251 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:20:07.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:20:07.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290783+0000", 2024-09-18T00:20:07.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5240000000000001e-06 2024-09-18T00:20:07.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290799+0000", 2024-09-18T00:20:07.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:20:07.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290840+0000", 2024-09-18T00:20:07.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0926000000000001e-05 2024-09-18T00:20:07.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290868+0000", 2024-09-18T00:20:07.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8163000000000001e-05 2024-09-18T00:20:07.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291294+0000", 2024-09-18T00:20:07.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00042588700000000001 2024-09-18T00:20:07.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:20:07.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:20:07.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.986e-05 2024-09-18T00:20:07.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316617+0000", 2024-09-18T00:20:07.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025303583000000001 2024-09-18T00:20:07.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:07.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316637+0000", 2024-09-18T00:20:07.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9388000000000001e-05 2024-09-18T00:20:07.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313933+0000", 2024-09-18T00:20:07.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99729681000000003 2024-09-18T00:20:07.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313987+0000", 2024-09-18T00:20:07.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4069e-05 2024-09-18T00:20:07.256 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.256 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.256 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.256 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34544 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:07.256 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:20:07.256 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.956166926999998, 2024-09-18T00:20:07.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0232412120000001, 2024-09-18T00:20:07.256 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.256 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.256 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:07.256 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.257 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.257 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.257 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34544 2024-09-18T00:20:07.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.257 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:20:07.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:20:07.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290930+0000", 2024-09-18T00:20:07.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5030000000000001e-06 2024-09-18T00:20:07.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290940+0000", 2024-09-18T00:20:07.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9000000000000001e-06 2024-09-18T00:20:07.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290964+0000", 2024-09-18T00:20:07.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4273e-05 2024-09-18T00:20:07.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290982+0000", 2024-09-18T00:20:07.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8216e-05 2024-09-18T00:20:07.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291433+0000", 2024-09-18T00:20:07.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000451278 2024-09-18T00:20:07.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:07.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291446+0000", 2024-09-18T00:20:07.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3039e-05 2024-09-18T00:20:07.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316687+0000", 2024-09-18T00:20:07.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025240069 2024-09-18T00:20:07.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:07.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316701+0000", 2024-09-18T00:20:07.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4100000000000001e-05 2024-09-18T00:20:07.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314142+0000", 2024-09-18T00:20:07.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99744119799999997 2024-09-18T00:20:07.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314169+0000", 2024-09-18T00:20:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7636000000000001e-05 2024-09-18T00:20:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34545 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:20:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.956088405000003, 2024-09-18T00:20:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023257802, 2024-09-18T00:20:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34545 2024-09-18T00:20:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:20:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:20:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291008+0000", 2024-09-18T00:20:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.237e-06 2024-09-18T00:20:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291015+0000", 2024-09-18T00:20:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2030000000000003e-06 2024-09-18T00:20:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291035+0000", 2024-09-18T00:20:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.978e-05 2024-09-18T00:20:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291049+0000", 2024-09-18T00:20:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4429e-05 2024-09-18T00:20:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291630+0000", 2024-09-18T00:20:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00058103199999999999 2024-09-18T00:20:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291644+0000", 2024-09-18T00:20:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.342e-05 2024-09-18T00:20:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316753+0000", 2024-09-18T00:20:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025108703999999999 2024-09-18T00:20:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316766+0000", 2024-09-18T00:20:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3237e-05 2024-09-18T00:20:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314238+0000", 2024-09-18T00:20:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99747232699999999 2024-09-18T00:20:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314265+0000", 2024-09-18T00:20:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6432999999999999e-05 2024-09-18T00:20:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34546 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:20:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.956030065999997, 2024-09-18T00:20:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023324535, 2024-09-18T00:20:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34546 2024-09-18T00:20:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:20:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:20:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291066+0000", 2024-09-18T00:20:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.569999999999999e-07 2024-09-18T00:20:07.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291073+0000", 2024-09-18T00:20:07.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.063e-06 2024-09-18T00:20:07.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291090+0000", 2024-09-18T00:20:07.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6977e-05 2024-09-18T00:20:07.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291105+0000", 2024-09-18T00:20:07.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5166e-05 2024-09-18T00:20:07.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291771+0000", 2024-09-18T00:20:07.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00066599699999999999 2024-09-18T00:20:07.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:07.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291785+0000", 2024-09-18T00:20:07.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.327e-05 2024-09-18T00:20:07.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316826+0000", 2024-09-18T00:20:07.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025041955000000001 2024-09-18T00:20:07.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:07.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316840+0000", 2024-09-18T00:20:07.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3256e-05 2024-09-18T00:20:07.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314335+0000", 2024-09-18T00:20:07.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99749496100000001 2024-09-18T00:20:07.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314390+0000", 2024-09-18T00:20:07.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4932999999999997e-05 2024-09-18T00:20:07.274 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.274 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.274 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.274 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34547 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:07.274 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:20:07.274 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.955970766999997, 2024-09-18T00:20:07.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.028510467, 2024-09-18T00:20:07.274 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.274 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.274 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:07.275 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.275 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.275 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.275 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34547 2024-09-18T00:20:07.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.275 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:20:07.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:20:07.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291126+0000", 2024-09-18T00:20:07.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2699999999999999e-06 2024-09-18T00:20:07.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291150+0000", 2024-09-18T00:20:07.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4669999999999999e-05 2024-09-18T00:20:07.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291171+0000", 2024-09-18T00:20:07.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0706000000000001e-05 2024-09-18T00:20:07.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291186+0000", 2024-09-18T00:20:07.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5126999999999999e-05 2024-09-18T00:20:07.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291885+0000", 2024-09-18T00:20:07.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00069850299999999995 2024-09-18T00:20:07.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:07.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291897+0000", 2024-09-18T00:20:07.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-05 2024-09-18T00:20:07.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316887+0000", 2024-09-18T00:20:07.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024990155999999999 2024-09-18T00:20:07.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:07.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316900+0000", 2024-09-18T00:20:07.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.295e-05 2024-09-18T00:20:07.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319594+0000", 2024-09-18T00:20:07.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0026940719999999 2024-09-18T00:20:07.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319635+0000", 2024-09-18T00:20:07.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0583000000000001e-05 2024-09-18T00:20:07.279 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.280 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.280 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.280 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34548 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:07.280 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:20:07.280 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.955823309000003, 2024-09-18T00:20:07.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0285287400000001, 2024-09-18T00:20:07.280 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.280 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.280 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:07.280 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.280 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.280 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.280 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34548 2024-09-18T00:20:07.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.281 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:20:07.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:20:07.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291274+0000", 2024-09-18T00:20:07.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.12e-06 2024-09-18T00:20:07.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291288+0000", 2024-09-18T00:20:07.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4367e-05 2024-09-18T00:20:07.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:20:07.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5698999999999999e-05 2024-09-18T00:20:07.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291333+0000", 2024-09-18T00:20:07.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8783e-05 2024-09-18T00:20:07.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291987+0000", 2024-09-18T00:20:07.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00065417800000000005 2024-09-18T00:20:07.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:07.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.292000+0000", 2024-09-18T00:20:07.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2652e-05 2024-09-18T00:20:07.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316946+0000", 2024-09-18T00:20:07.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024946696000000001 2024-09-18T00:20:07.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:07.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316956+0000", 2024-09-18T00:20:07.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2760000000000006e-06 2024-09-18T00:20:07.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319752+0000", 2024-09-18T00:20:07.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002796083 2024-09-18T00:20:07.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319801+0000", 2024-09-18T00:20:07.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8885999999999999e-05 2024-09-18T00:20:07.285 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.285 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.285 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.285 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41950 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:07.285 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:20:07.286 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.553170600000001, 2024-09-18T00:20:07.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053368486, 2024-09-18T00:20:07.286 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.286 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.286 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:07.286 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.286 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.286 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.286 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41950 2024-09-18T00:20:07.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.286 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:20:07.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:20:07.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693926+0000", 2024-09-18T00:20:07.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4900000000000004e-07 2024-09-18T00:20:07.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693930+0000", 2024-09-18T00:20:07.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0130000000000004e-06 2024-09-18T00:20:07.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693940+0000", 2024-09-18T00:20:07.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0618000000000001e-05 2024-09-18T00:20:07.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693948+0000", 2024-09-18T00:20:07.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3140000000000004e-06 2024-09-18T00:20:07.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747133+0000", 2024-09-18T00:20:07.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053184689 2024-09-18T00:20:07.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:07.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747203+0000", 2024-09-18T00:20:07.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0339000000000005e-05 2024-09-18T00:20:07.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747293+0000", 2024-09-18T00:20:07.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9563999999999998e-05 2024-09-18T00:20:07.289 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.290 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.290 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.290 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41951 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:07.290 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:20:07.290 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.553135832999999, 2024-09-18T00:20:07.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053527695, 2024-09-18T00:20:07.290 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.290 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.290 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:07.290 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.290 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.291 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.291 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41951 2024-09-18T00:20:07.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.291 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:20:07.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:20:07.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693960+0000", 2024-09-18T00:20:07.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.16e-07 2024-09-18T00:20:07.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693971+0000", 2024-09-18T00:20:07.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1204e-05 2024-09-18T00:20:07.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693981+0000", 2024-09-18T00:20:07.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-18T00:20:07.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693989+0000", 2024-09-18T00:20:07.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0180000000000001e-06 2024-09-18T00:20:07.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747328+0000", 2024-09-18T00:20:07.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053338508 2024-09-18T00:20:07.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:07.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747400+0000", 2024-09-18T00:20:07.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2750999999999995e-05 2024-09-18T00:20:07.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747487+0000", 2024-09-18T00:20:07.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6527999999999996e-05 2024-09-18T00:20:07.294 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.294 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.294 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.295 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41952 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:07.295 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:20:07.295 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.553095425999999, 2024-09-18T00:20:07.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053728013999999998, 2024-09-18T00:20:07.295 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.295 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.295 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:07.295 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.296 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.296 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.296 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41952 2024-09-18T00:20:07.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.296 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:20:07.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:20:07.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694001+0000", 2024-09-18T00:20:07.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3799999999999996e-07 2024-09-18T00:20:07.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694004+0000", 2024-09-18T00:20:07.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.965e-06 2024-09-18T00:20:07.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694013+0000", 2024-09-18T00:20:07.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1689999999999998e-06 2024-09-18T00:20:07.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694022+0000", 2024-09-18T00:20:07.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7560000000000005e-06 2024-09-18T00:20:07.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747531+0000", 2024-09-18T00:20:07.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053508999000000002 2024-09-18T00:20:07.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:07.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747610+0000", 2024-09-18T00:20:07.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9212e-05 2024-09-18T00:20:07.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747728+0000", 2024-09-18T00:20:07.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011797499999999999 2024-09-18T00:20:07.299 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.299 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.299 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.299 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41953 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:07.299 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:20:07.299 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.553061796999998, 2024-09-18T00:20:07.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053936783000000002, 2024-09-18T00:20:07.300 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.300 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.300 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:07.300 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.300 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.300 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.300 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41953 2024-09-18T00:20:07.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.300 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:20:07.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:20:07.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694034+0000", 2024-09-18T00:20:07.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5199999999999996e-07 2024-09-18T00:20:07.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694037+0000", 2024-09-18T00:20:07.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.745e-06 2024-09-18T00:20:07.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694046+0000", 2024-09-18T00:20:07.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2490000000000007e-06 2024-09-18T00:20:07.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694054+0000", 2024-09-18T00:20:07.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1179999999999992e-06 2024-09-18T00:20:07.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747766+0000", 2024-09-18T00:20:07.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053711362999999998 2024-09-18T00:20:07.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:07.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747856+0000", 2024-09-18T00:20:07.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0779000000000006e-05 2024-09-18T00:20:07.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747970+0000", 2024-09-18T00:20:07.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011377700000000001 2024-09-18T00:20:07.303 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.303 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.303 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.303 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41954 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:07.303 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:20:07.304 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.553031644000001, 2024-09-18T00:20:07.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054104898999999998, 2024-09-18T00:20:07.304 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.304 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.304 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:07.304 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.304 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.304 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.304 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41954 2024-09-18T00:20:07.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.304 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:20:07.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:20:07.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:20:07.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4000000000000002e-07 2024-09-18T00:20:07.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694067+0000", 2024-09-18T00:20:07.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7329999999999999e-06 2024-09-18T00:20:07.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694076+0000", 2024-09-18T00:20:07.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2019999999999993e-06 2024-09-18T00:20:07.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694083+0000", 2024-09-18T00:20:07.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0029999999999997e-06 2024-09-18T00:20:07.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748003+0000", 2024-09-18T00:20:07.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053920463000000002 2024-09-18T00:20:07.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:07.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748077+0000", 2024-09-18T00:20:07.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4041000000000005e-05 2024-09-18T00:20:07.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748168+0000", 2024-09-18T00:20:07.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0916999999999997e-05 2024-09-18T00:20:07.308 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.308 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.308 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.308 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41955 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 231424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:07.308 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:20:07.308 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.553002806000002, 2024-09-18T00:20:07.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054272659000000001, 2024-09-18T00:20:07.308 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.308 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.308 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:07.308 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.308 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.309 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.309 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41955 2024-09-18T00:20:07.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.309 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:20:07.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:20:07.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694093+0000", 2024-09-18T00:20:07.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:20:07.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694109+0000", 2024-09-18T00:20:07.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6061000000000001e-05 2024-09-18T00:20:07.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694121+0000", 2024-09-18T00:20:07.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.216e-05 2024-09-18T00:20:07.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694129+0000", 2024-09-18T00:20:07.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0029999999999994e-06 2024-09-18T00:20:07.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748202+0000", 2024-09-18T00:20:07.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054072571 2024-09-18T00:20:07.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:07.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748274+0000", 2024-09-18T00:20:07.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2117000000000001e-05 2024-09-18T00:20:07.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748365+0000", 2024-09-18T00:20:07.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1139999999999998e-05 2024-09-18T00:20:07.312 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.312 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.312 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.312 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41956 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:07.312 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:20:07.312 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.552954597999999, 2024-09-18T00:20:07.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054420496999999998, 2024-09-18T00:20:07.313 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.313 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.313 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:07.313 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.313 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.313 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.313 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41956 2024-09-18T00:20:07.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.313 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:20:07.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:20:07.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:20:07.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8699999999999995e-07 2024-09-18T00:20:07.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694144+0000", 2024-09-18T00:20:07.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3510000000000002e-06 2024-09-18T00:20:07.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694154+0000", 2024-09-18T00:20:07.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0769999999999997e-06 2024-09-18T00:20:07.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694163+0000", 2024-09-18T00:20:07.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3300000000000005e-06 2024-09-18T00:20:07.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748397+0000", 2024-09-18T00:20:07.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054234497999999999 2024-09-18T00:20:07.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:07.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748471+0000", 2024-09-18T00:20:07.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3441000000000004e-05 2024-09-18T00:20:07.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748561+0000", 2024-09-18T00:20:07.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0212999999999998e-05 2024-09-18T00:20:07.317 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.317 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.317 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.317 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41957 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:07.317 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:20:07.317 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.552923346, 2024-09-18T00:20:07.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054590310000000003, 2024-09-18T00:20:07.317 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.317 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.318 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:07.318 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.318 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.318 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.318 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41957 2024-09-18T00:20:07.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.318 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:20:07.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:20:07.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694173+0000", 2024-09-18T00:20:07.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2500000000000005e-07 2024-09-18T00:20:07.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694175+0000", 2024-09-18T00:20:07.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.655e-06 2024-09-18T00:20:07.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694184+0000", 2024-09-18T00:20:07.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7630000000000006e-06 2024-09-18T00:20:07.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694191+0000", 2024-09-18T00:20:07.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8870000000000003e-06 2024-09-18T00:20:07.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748593+0000", 2024-09-18T00:20:07.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054402525 2024-09-18T00:20:07.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:07.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748669+0000", 2024-09-18T00:20:07.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5532999999999997e-05 2024-09-18T00:20:07.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748762+0000", 2024-09-18T00:20:07.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3221999999999996e-05 2024-09-18T00:20:07.321 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.321 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.321 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.321 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42392 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:07.321 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:20:07.322 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.539893907, 2024-09-18T00:20:07.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053378162, 2024-09-18T00:20:07.322 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.322 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.322 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:07.322 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.322 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.322 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.322 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42392 2024-09-18T00:20:07.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.322 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:20:07.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:20:07.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707430+0000", 2024-09-18T00:20:07.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022838599999999999 2024-09-18T00:20:07.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707434+0000", 2024-09-18T00:20:07.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5220000000000001e-06 2024-09-18T00:20:07.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707467+0000", 2024-09-18T00:20:07.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2570000000000002e-05 2024-09-18T00:20:07.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707475+0000", 2024-09-18T00:20:07.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:20:07.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760446+0000", 2024-09-18T00:20:07.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052971635000000003 2024-09-18T00:20:07.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:07.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760512+0000", 2024-09-18T00:20:07.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5679999999999995e-05 2024-09-18T00:20:07.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760579+0000", 2024-09-18T00:20:07.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7296000000000001e-05 2024-09-18T00:20:07.325 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.325 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.325 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.326 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42393 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 358400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:07.326 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:20:07.326 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.539604955999998, 2024-09-18T00:20:07.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053269618999999997, 2024-09-18T00:20:07.326 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.326 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.326 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:07.326 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.326 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.326 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.326 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42393 2024-09-18T00:20:07.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.326 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:20:07.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:20:07.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707491+0000", 2024-09-18T00:20:07.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1399999999999997e-07 2024-09-18T00:20:07.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707503+0000", 2024-09-18T00:20:07.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1912e-05 2024-09-18T00:20:07.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707512+0000", 2024-09-18T00:20:07.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1030000000000008e-06 2024-09-18T00:20:07.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707520+0000", 2024-09-18T00:20:07.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1249999999999993e-06 2024-09-18T00:20:07.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760605+0000", 2024-09-18T00:20:07.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053085495000000003 2024-09-18T00:20:07.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:07.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760680+0000", 2024-09-18T00:20:07.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5024000000000005e-05 2024-09-18T00:20:07.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760760+0000", 2024-09-18T00:20:07.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9345999999999996e-05 2024-09-18T00:20:07.329 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.329 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.329 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42394 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:20:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.539564968000001, 2024-09-18T00:20:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053403144999999999, 2024-09-18T00:20:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42394 2024-09-18T00:20:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:20:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:20:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711550+0000", 2024-09-18T00:20:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040200189999999997 2024-09-18T00:20:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711556+0000", 2024-09-18T00:20:07.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2339999999999999e-06 2024-09-18T00:20:07.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711569+0000", 2024-09-18T00:20:07.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2337999999999999e-05 2024-09-18T00:20:07.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711577+0000", 2024-09-18T00:20:07.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5660000000000007e-06 2024-09-18T00:20:07.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760790+0000", 2024-09-18T00:20:07.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049212196999999999 2024-09-18T00:20:07.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:07.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760857+0000", 2024-09-18T00:20:07.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7756000000000007e-05 2024-09-18T00:20:07.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760933+0000", 2024-09-18T00:20:07.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6035e-05 2024-09-18T00:20:07.333 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.333 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.333 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.334 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:55489 3.5 3:a5f1e468:::rbd_data.112a859a01c9.000000000000000c:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T00:20:07.334 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:20:07.334 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.615710841999999, 2024-09-18T00:20:07.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.05748731, 2024-09-18T00:20:07.334 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.334 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.334 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:07.334 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.334 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.334 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.334 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 55489 2024-09-18T00:20:07.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.334 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:20:07.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:20:07.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631388+0000", 2024-09-18T00:20:07.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6169999999999998e-06 2024-09-18T00:20:07.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631405+0000", 2024-09-18T00:20:07.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6643e-05 2024-09-18T00:20:07.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631444+0000", 2024-09-18T00:20:07.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9226e-05 2024-09-18T00:20:07.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631465+0000", 2024-09-18T00:20:07.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0826999999999999e-05 2024-09-18T00:20:07.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.337 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.337 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631578+0000", 2024-09-18T00:20:07.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011291899999999999 2024-09-18T00:20:07.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.337 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:07.337 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631651+0000", 2024-09-18T00:20:07.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3264999999999997e-05 2024-09-18T00:20:07.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.337 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:20:07.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.634073+0000", 2024-09-18T00:20:07.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0024221350000000002 2024-09-18T00:20:07.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:07.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688732+0000", 2024-09-18T00:20:07.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054658610000000003 2024-09-18T00:20:07.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:07.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688794+0000", 2024-09-18T00:20:07.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2255999999999995e-05 2024-09-18T00:20:07.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:07.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688834+0000", 2024-09-18T00:20:07.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9916999999999997e-05 2024-09-18T00:20:07.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688872+0000", 2024-09-18T00:20:07.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7895000000000002e-05 2024-09-18T00:20:07.339 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.339 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.339 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.339 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:58361 3.1d 3:bac90a3d:::rbd_data.112a17f95ba0.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T00:20:07.339 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:20:07.340 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.043365347999998, 2024-09-18T00:20:07.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101552963, 2024-09-18T00:20:07.340 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.340 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.340 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:07.340 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:07.340 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:07.340 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:07.340 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 58361 2024-09-18T00:20:07.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.340 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:20:07.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:20:07.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203733+0000", 2024-09-18T00:20:07.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.01e-06 2024-09-18T00:20:07.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203745+0000", 2024-09-18T00:20:07.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2513e-05 2024-09-18T00:20:07.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203777+0000", 2024-09-18T00:20:07.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1216000000000001e-05 2024-09-18T00:20:07.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203792+0000", 2024-09-18T00:20:07.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586e-05 2024-09-18T00:20:07.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203911+0000", 2024-09-18T00:20:07.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119234 2024-09-18T00:20:07.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:07.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.204056+0000", 2024-09-18T00:20:07.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000144943 2024-09-18T00:20:07.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:07.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.210219+0000", 2024-09-18T00:20:07.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061630649999999997 2024-09-18T00:20:07.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:07.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305071+0000", 2024-09-18T00:20:07.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.094852120999999998 2024-09-18T00:20:07.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:07.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305106+0000", 2024-09-18T00:20:07.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4848999999999998e-05 2024-09-18T00:20:07.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:07.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305208+0000", 2024-09-18T00:20:07.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101518 2024-09-18T00:20:07.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305283+0000", 2024-09-18T00:20:07.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4907999999999995e-05 2024-09-18T00:20:07.345 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.345 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.345 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.345 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:58361 3.1d e96/79 ondisk, result = 0)", 2024-09-18T00:20:07.345 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:20:07.345 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.015882306999998, 2024-09-18T00:20:07.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.073876681, 2024-09-18T00:20:07.345 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:07.345 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:07.345 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:07.345 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:07.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:07.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:20:07.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:07.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:20:07.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:07.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:07.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231215+0000", 2024-09-18T00:20:07.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9570000000000001e-06 2024-09-18T00:20:07.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:07.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231223+0000", 2024-09-18T00:20:07.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0900000000000005e-06 2024-09-18T00:20:07.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:07.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231252+0000", 2024-09-18T00:20:07.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9552000000000001e-05 2024-09-18T00:20:07.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231267+0000", 2024-09-18T00:20:07.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4263e-05 2024-09-18T00:20:07.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:07.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.304871+0000", 2024-09-18T00:20:07.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.073604257000000006 2024-09-18T00:20:07.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:07.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.304889+0000", 2024-09-18T00:20:07.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.808e-05 2024-09-18T00:20:07.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:07.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:07.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:07.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305090+0000", 2024-09-18T00:20:07.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020048200000000001 2024-09-18T00:20:07.349 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.349 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.349 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.349 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:07.350 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:07.350 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:07.351 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:20:07.351 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:07.349+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:07.452 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:20:07.449+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:20:07.475 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is set 2024-09-18T00:20:07.491 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:07.491 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:20:07.492 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:07.511 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T00:20:07.553 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:07.549+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:07.654 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:07.652+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:07.677 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:20:07.677 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:20:07.677 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:07.691 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T00:20:07.755 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:07.753+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:07.856 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:20:07.852+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:20:07.862 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:20:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:20:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:20:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGRecoveryDelete(3.8 e124,122 3:14210294:::rbd_data.112ae20746f5.0000000000000133:head,29'2)", 2024-09-18T00:20:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.818357+0000", 2024-09-18T00:20:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.040663991, 2024-09-18T00:20:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016441399999999999, 2024-09-18T00:20:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.863 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.818357+0000", 2024-09-18T00:20:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.818357+0000", 2024-09-18T00:20:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.818358+0000", 2024-09-18T00:20:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.44e-07 2024-09-18T00:20:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.818362+0000", 2024-09-18T00:20:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4560000000000002e-06 2024-09-18T00:20:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.818373+0000", 2024-09-18T00:20:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0512e-05 2024-09-18T00:20:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.818379+0000", 2024-09-18T00:20:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0329999999999997e-06 2024-09-18T00:20:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.818431+0000", 2024-09-18T00:20:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2468e-05 2024-09-18T00:20:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.818440+0000", 2024-09-18T00:20:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7290000000000006e-06 2024-09-18T00:20:07.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.818521+0000", 2024-09-18T00:20:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1571999999999998e-05 2024-09-18T00:20:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGPush(3.8 124/122 [PushOp(3:12ac789b:::rbd_data.112aa7ba4578.0000000000000130:head, version: 39'36, data_included: [196608~20221], data_size: 20221, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:12ac789b:::rbd_data.112aa7ba4578.0000000000000130:head@39'36, size: 216829, copy_subset: [(0, 216829)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 216829, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-18T00:20:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.838673+0000", 2024-09-18T00:20:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.02034833, 2024-09-18T00:20:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00100775, 2024-09-18T00:20:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.867 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.838673+0000", 2024-09-18T00:20:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.838673+0000", 2024-09-18T00:20:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.838673+0000", 2024-09-18T00:20:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.89e-07 2024-09-18T00:20:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.838701+0000", 2024-09-18T00:20:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.7475e-05 2024-09-18T00:20:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.838717+0000", 2024-09-18T00:20:07.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6099e-05 2024-09-18T00:20:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.838727+0000", 2024-09-18T00:20:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0778000000000001e-05 2024-09-18T00:20:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.838798+0000", 2024-09-18T00:20:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0502999999999999e-05 2024-09-18T00:20:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.838806+0000", 2024-09-18T00:20:07.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.5970000000000003e-06 2024-09-18T00:20:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.839680+0000", 2024-09-18T00:20:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087480900000000005 2024-09-18T00:20:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGPush(3.8 124/122 [PushOp(3:11b3ac8b:::rbd_data.112aa7ba4578.000000000000033a:head, version: 39'45, data_included: [163840~172], data_size: 172, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:11b3ac8b:::rbd_data.112aa7ba4578.000000000000033a:head@39'45, size: 164012, copy_subset: [(0, 164012)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 164012, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-18T00:20:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.847060+0000", 2024-09-18T00:20:07.871 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.01196105, 2024-09-18T00:20:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000409202, 2024-09-18T00:20:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.847060+0000", 2024-09-18T00:20:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.847060+0000", 2024-09-18T00:20:07.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.847060+0000", 2024-09-18T00:20:07.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5400000000000001e-07 2024-09-18T00:20:07.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.847070+0000", 2024-09-18T00:20:07.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1800000000000002e-06 2024-09-18T00:20:07.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.847086+0000", 2024-09-18T00:20:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6183000000000001e-05 2024-09-18T00:20:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.847096+0000", 2024-09-18T00:20:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0592e-05 2024-09-18T00:20:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.847154+0000", 2024-09-18T00:20:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7713000000000001e-05 2024-09-18T00:20:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.847161+0000", 2024-09-18T00:20:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0400000000000004e-06 2024-09-18T00:20:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.847469+0000", 2024-09-18T00:20:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00030793999999999998 2024-09-18T00:20:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGPush(3.8 124/122 [PushOp(3:17544c57:::rbd_data.112aa7ba4578.0000000000000142:head, version: 39'46, data_included: [189300~7308,229376~6272], data_size: 13580, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:17544c57:::rbd_data.112aa7ba4578.0000000000000142:head@39'46, size: 235648, copy_subset: [(0, 235648)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 235648, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-18T00:20:07.876 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.848588+0000", 2024-09-18T00:20:07.876 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.0104326340000001, 2024-09-18T00:20:07.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00050644700000000004, 2024-09-18T00:20:07.876 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.876 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.876 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.876 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.848588+0000", 2024-09-18T00:20:07.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.848588+0000", 2024-09-18T00:20:07.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.848589+0000", 2024-09-18T00:20:07.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1099999999999996e-07 2024-09-18T00:20:07.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.848611+0000", 2024-09-18T00:20:07.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2371e-05 2024-09-18T00:20:07.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.848627+0000", 2024-09-18T00:20:07.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5389000000000001e-05 2024-09-18T00:20:07.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.848637+0000", 2024-09-18T00:20:07.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0125e-05 2024-09-18T00:20:07.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.848690+0000", 2024-09-18T00:20:07.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3579000000000003e-05 2024-09-18T00:20:07.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.848698+0000", 2024-09-18T00:20:07.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9929999999999995e-06 2024-09-18T00:20:07.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.849095+0000", 2024-09-18T00:20:07.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00039647899999999998 2024-09-18T00:20:07.880 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.880 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.880 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.880 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGPush(3.8 124/122 [PushOp(3:16b24e91:::rbd_data.112aa7ba4578.0000000000000161:head, version: 39'47, data_included: [66244~32060], data_size: 32060, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:16b24e91:::rbd_data.112aa7ba4578.0000000000000161:head@39'47, size: 98304, copy_subset: [(0, 98304)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 98304, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-18T00:20:07.880 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.850222+0000", 2024-09-18T00:20:07.880 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.0087991810000001, 2024-09-18T00:20:07.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0014554730000000001, 2024-09-18T00:20:07.880 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.880 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.880 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.881 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.850222+0000", 2024-09-18T00:20:07.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.850222+0000", 2024-09-18T00:20:07.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.850222+0000", 2024-09-18T00:20:07.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.13e-07 2024-09-18T00:20:07.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.850260+0000", 2024-09-18T00:20:07.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7858999999999998e-05 2024-09-18T00:20:07.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.850276+0000", 2024-09-18T00:20:07.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5693999999999999e-05 2024-09-18T00:20:07.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.850286+0000", 2024-09-18T00:20:07.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7820000000000005e-06 2024-09-18T00:20:07.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.850337+0000", 2024-09-18T00:20:07.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0995999999999998e-05 2024-09-18T00:20:07.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.850343+0000", 2024-09-18T00:20:07.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5200000000000003e-06 2024-09-18T00:20:07.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.851677+0000", 2024-09-18T00:20:07.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001334109 2024-09-18T00:20:07.884 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.884 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.884 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.884 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGPush(3.8 124/122 [PushOp(3:17e81ee6:::rbd_id.image_client.0-clone4:head, version: 41'48, data_included: [0~16], data_size: 16, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:17e81ee6:::rbd_id.image_client.0-clone4:head@41'48, size: 16, copy_subset: [(0, 16)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 16, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-18T00:20:07.884 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.852753+0000", 2024-09-18T00:20:07.884 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 1.006267977, 2024-09-18T00:20:07.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00039174199999999999, 2024-09-18T00:20:07.885 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.885 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.885 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.885 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.852753+0000", 2024-09-18T00:20:07.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.852753+0000", 2024-09-18T00:20:07.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.852754+0000", 2024-09-18T00:20:07.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2900000000000004e-07 2024-09-18T00:20:07.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.852762+0000", 2024-09-18T00:20:07.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.32e-06 2024-09-18T00:20:07.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.886 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.886 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.852777+0000", 2024-09-18T00:20:07.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5017e-05 2024-09-18T00:20:07.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.852787+0000", 2024-09-18T00:20:07.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0227999999999999e-05 2024-09-18T00:20:07.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.852838+0000", 2024-09-18T00:20:07.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0559999999999997e-05 2024-09-18T00:20:07.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.852844+0000", 2024-09-18T00:20:07.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4389999999999997e-06 2024-09-18T00:20:07.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.853145+0000", 2024-09-18T00:20:07.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000300649 2024-09-18T00:20:07.888 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.888 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.888 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.888 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGPush(3.8 124/122 [PushOp(3:10111e51:::rbd_data.112ab1fa102a.000000000000001e:head, version: 50'81, data_included: [345755~6501], data_size: 6501, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:10111e51:::rbd_data.112ab1fa102a.000000000000001e:head@50'81, size: 352256, copy_subset: [(0, 352256)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 352256, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-18T00:20:07.889 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.868313+0000", 2024-09-18T00:20:07.889 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.99070747800000003, 2024-09-18T00:20:07.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048062199999999999, 2024-09-18T00:20:07.889 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.889 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.889 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.889 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.868313+0000", 2024-09-18T00:20:07.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.868313+0000", 2024-09-18T00:20:07.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.868314+0000", 2024-09-18T00:20:07.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2399999999999998e-07 2024-09-18T00:20:07.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.868337+0000", 2024-09-18T00:20:07.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3054999999999999e-05 2024-09-18T00:20:07.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.868354+0000", 2024-09-18T00:20:07.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6922000000000002e-05 2024-09-18T00:20:07.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.868365+0000", 2024-09-18T00:20:07.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0769e-05 2024-09-18T00:20:07.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.868433+0000", 2024-09-18T00:20:07.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8029999999999997e-05 2024-09-18T00:20:07.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.868440+0000", 2024-09-18T00:20:07.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9480000000000002e-06 2024-09-18T00:20:07.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.868794+0000", 2024-09-18T00:20:07.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00035437400000000002 2024-09-18T00:20:07.892 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.892 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.892 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.893 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGPush(3.8 124/122 [PushOp(3:104e5119:::rbd_data.112a7b1589a.000000000000000c:a, version: 52'213, data_included: [1097728~16384], data_size: 16384, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:104e5119:::rbd_data.112a7b1589a.000000000000000c:a@52'213, size: 1114112, copy_subset: [(0, 1114112)], clone_subset: {}, snapset: a=[]:{a: [a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1114112, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-18T00:20:07.893 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.875881+0000", 2024-09-18T00:20:07.893 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.98313967999999996, 2024-09-18T00:20:07.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0010325340000000001, 2024-09-18T00:20:07.893 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.893 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.893 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.893 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.875881+0000", 2024-09-18T00:20:07.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.875881+0000", 2024-09-18T00:20:07.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.875882+0000", 2024-09-18T00:20:07.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9800000000000004e-07 2024-09-18T00:20:07.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.875905+0000", 2024-09-18T00:20:07.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3646999999999999e-05 2024-09-18T00:20:07.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.875922+0000", 2024-09-18T00:20:07.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6245000000000001e-05 2024-09-18T00:20:07.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.875933+0000", 2024-09-18T00:20:07.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0944999999999999e-05 2024-09-18T00:20:07.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.876003+0000", 2024-09-18T00:20:07.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0544999999999996e-05 2024-09-18T00:20:07.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.876010+0000", 2024-09-18T00:20:07.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1160000000000001e-06 2024-09-18T00:20:07.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.876914+0000", 2024-09-18T00:20:07.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00090353799999999995 2024-09-18T00:20:07.896 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.896 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.896 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.897 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGPush(3.8 124/122 [PushOp(3:104e5119:::rbd_data.112a7b1589a.000000000000000c:head, version: 52'214, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:104e5119:::rbd_data.112a7b1589a.000000000000000c:head@52'214, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-18T00:20:07.897 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.877926+0000", 2024-09-18T00:20:07.897 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.98109511199999999, 2024-09-18T00:20:07.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034703400000000003, 2024-09-18T00:20:07.897 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.897 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.897 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.897 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.877926+0000", 2024-09-18T00:20:07.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.877926+0000", 2024-09-18T00:20:07.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.877927+0000", 2024-09-18T00:20:07.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4300000000000003e-07 2024-09-18T00:20:07.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.877936+0000", 2024-09-18T00:20:07.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2050000000000008e-06 2024-09-18T00:20:07.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.877954+0000", 2024-09-18T00:20:07.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7847e-05 2024-09-18T00:20:07.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.877964+0000", 2024-09-18T00:20:07.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0672999999999999e-05 2024-09-18T00:20:07.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.878035+0000", 2024-09-18T00:20:07.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1061000000000003e-05 2024-09-18T00:20:07.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.878042+0000", 2024-09-18T00:20:07.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9519999999999997e-06 2024-09-18T00:20:07.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.878273+0000", 2024-09-18T00:20:07.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023065299999999999 2024-09-18T00:20:07.900 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.901 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.901 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGPush(3.8 124/122 [PushOp(3:10f5bea1:::rbd_object_map.112a7b1589a:head, version: 52'222, data_included: [0~36], data_size: 36, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:10f5bea1:::rbd_object_map.112a7b1589a:head@52'222, size: 36, copy_subset: [(0, 36)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 36, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-18T00:20:07.901 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.879276+0000", 2024-09-18T00:20:07.901 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.97974509499999995, 2024-09-18T00:20:07.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00040222000000000001, 2024-09-18T00:20:07.901 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.901 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.901 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.901 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.879276+0000", 2024-09-18T00:20:07.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.879276+0000", 2024-09-18T00:20:07.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.879276+0000", 2024-09-18T00:20:07.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6300000000000005e-07 2024-09-18T00:20:07.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.879285+0000", 2024-09-18T00:20:07.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0159999999999997e-06 2024-09-18T00:20:07.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.879302+0000", 2024-09-18T00:20:07.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6390000000000001e-05 2024-09-18T00:20:07.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.879312+0000", 2024-09-18T00:20:07.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0438999999999999e-05 2024-09-18T00:20:07.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.879377+0000", 2024-09-18T00:20:07.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4797999999999999e-05 2024-09-18T00:20:07.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.879384+0000", 2024-09-18T00:20:07.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9310000000000002e-06 2024-09-18T00:20:07.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.879678+0000", 2024-09-18T00:20:07.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000294083 2024-09-18T00:20:07.905 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.905 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.905 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.905 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGPush(3.8 124/122 [PushOp(3:1527150c:::rbd_data.112a13ec2d6c.0000000000000003:head, version: 52'225, data_included: [81357~5963,1048576~46835], data_size: 52798, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:1527150c:::rbd_data.112a13ec2d6c.0000000000000003:head@52'225, size: 1095411, copy_subset: [(0, 1095411)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1095411, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-18T00:20:07.905 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.880728+0000", 2024-09-18T00:20:07.905 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.97829333299999999, 2024-09-18T00:20:07.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0023411320000000001, 2024-09-18T00:20:07.905 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.905 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.905 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.905 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.880728+0000", 2024-09-18T00:20:07.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.880728+0000", 2024-09-18T00:20:07.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.880728+0000", 2024-09-18T00:20:07.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9599999999999999e-07 2024-09-18T00:20:07.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.880779+0000", 2024-09-18T00:20:07.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.024e-05 2024-09-18T00:20:07.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.880797+0000", 2024-09-18T00:20:07.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7958e-05 2024-09-18T00:20:07.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.880807+0000", 2024-09-18T00:20:07.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0954999999999999e-05 2024-09-18T00:20:07.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.880878+0000", 2024-09-18T00:20:07.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0450999999999993e-05 2024-09-18T00:20:07.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.880886+0000", 2024-09-18T00:20:07.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4959999999999996e-06 2024-09-18T00:20:07.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.883069+0000", 2024-09-18T00:20:07.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002182336 2024-09-18T00:20:07.909 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.909 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.909 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.909 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGPush(3.8 124/122 [PushOp(3:1253c2d0:::rbd_data.112a5ba20d45.0000000000000036:head, version: 62'234, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:1253c2d0:::rbd_data.112a5ba20d45.0000000000000036:head@62'234, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-18T00:20:07.909 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.887245+0000", 2024-09-18T00:20:07.909 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.97177569100000005, 2024-09-18T00:20:07.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00034490399999999999, 2024-09-18T00:20:07.909 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.909 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.909 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.909 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.887245+0000", 2024-09-18T00:20:07.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.887245+0000", 2024-09-18T00:20:07.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.887246+0000", 2024-09-18T00:20:07.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7800000000000001e-07 2024-09-18T00:20:07.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.887255+0000", 2024-09-18T00:20:07.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9230000000000008e-06 2024-09-18T00:20:07.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.887272+0000", 2024-09-18T00:20:07.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6741000000000001e-05 2024-09-18T00:20:07.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.887283+0000", 2024-09-18T00:20:07.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1608e-05 2024-09-18T00:20:07.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.887350+0000", 2024-09-18T00:20:07.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6648000000000003e-05 2024-09-18T00:20:07.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.887357+0000", 2024-09-18T00:20:07.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7870000000000004e-06 2024-09-18T00:20:07.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.887590+0000", 2024-09-18T00:20:07.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00023361899999999999 2024-09-18T00:20:07.913 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.913 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.913 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.913 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGRecoveryDelete(3.8 e124,122 3:148e32c0:::rbd_data.112aa34d8ec3.00000000000000a4:head,72'296)", 2024-09-18T00:20:07.913 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.891886+0000", 2024-09-18T00:20:07.913 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.96713532499999999, 2024-09-18T00:20:07.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000159553, 2024-09-18T00:20:07.913 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.913 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.913 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.914 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.891886+0000", 2024-09-18T00:20:07.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.891886+0000", 2024-09-18T00:20:07.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.891886+0000", 2024-09-18T00:20:07.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6599999999999996e-07 2024-09-18T00:20:07.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.891891+0000", 2024-09-18T00:20:07.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7759999999999997e-06 2024-09-18T00:20:07.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.891905+0000", 2024-09-18T00:20:07.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4205e-05 2024-09-18T00:20:07.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.891913+0000", 2024-09-18T00:20:07.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8590000000000005e-06 2024-09-18T00:20:07.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.891968+0000", 2024-09-18T00:20:07.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5343000000000002e-05 2024-09-18T00:20:07.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.891978+0000", 2024-09-18T00:20:07.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8230000000000006e-06 2024-09-18T00:20:07.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.892045+0000", 2024-09-18T00:20:07.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6880999999999999e-05 2024-09-18T00:20:07.917 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.917 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.917 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.917 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGRecoveryDelete(3.8 e124,122 3:1445ad14:::rbd_data.112aa34d8ec3.0000000000000139:head,72'331)", 2024-09-18T00:20:07.917 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.893245+0000", 2024-09-18T00:20:07.917 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.96577579800000002, 2024-09-18T00:20:07.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017702199999999999, 2024-09-18T00:20:07.917 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.917 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.918 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.918 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.893245+0000", 2024-09-18T00:20:07.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.893245+0000", 2024-09-18T00:20:07.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.893246+0000", 2024-09-18T00:20:07.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8400000000000004e-07 2024-09-18T00:20:07.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.893250+0000", 2024-09-18T00:20:07.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2660000000000003e-06 2024-09-18T00:20:07.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.893264+0000", 2024-09-18T00:20:07.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4011e-05 2024-09-18T00:20:07.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.893272+0000", 2024-09-18T00:20:07.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0139999999999998e-06 2024-09-18T00:20:07.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.893334+0000", 2024-09-18T00:20:07.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1896000000000005e-05 2024-09-18T00:20:07.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.893344+0000", 2024-09-18T00:20:07.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5810000000000003e-06 2024-09-18T00:20:07.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.893422+0000", 2024-09-18T00:20:07.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8670000000000004e-05 2024-09-18T00:20:07.921 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.921 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.921 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.921 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGRecoveryDelete(3.8 e124,122 3:16d0a705:::rbd_data.112af53b3a45.0000000000000079:head,81'499)", 2024-09-18T00:20:07.921 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.913162+0000", 2024-09-18T00:20:07.921 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.94585925199999998, 2024-09-18T00:20:07.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000162795, 2024-09-18T00:20:07.921 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.922 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.922 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.922 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.913162+0000", 2024-09-18T00:20:07.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.913162+0000", 2024-09-18T00:20:07.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.913163+0000", 2024-09-18T00:20:07.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8599999999999997e-07 2024-09-18T00:20:07.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.913167+0000", 2024-09-18T00:20:07.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6120000000000001e-06 2024-09-18T00:20:07.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.913183+0000", 2024-09-18T00:20:07.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5892e-05 2024-09-18T00:20:07.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.913191+0000", 2024-09-18T00:20:07.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4679999999999992e-06 2024-09-18T00:20:07.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.913249+0000", 2024-09-18T00:20:07.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7423999999999998e-05 2024-09-18T00:20:07.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.913257+0000", 2024-09-18T00:20:07.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7789999999999996e-06 2024-09-18T00:20:07.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.913325+0000", 2024-09-18T00:20:07.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7834000000000001e-05 2024-09-18T00:20:07.925 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.925 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.925 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.925 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGRecoveryDelete(3.8 e124,122 3:149130d0:::rbd_data.112af53b3a45.000000000000007a:head,81'500)", 2024-09-18T00:20:07.925 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.914360+0000", 2024-09-18T00:20:07.925 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.94466053800000005, 2024-09-18T00:20:07.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000165334, 2024-09-18T00:20:07.926 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.926 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.926 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.926 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.914360+0000", 2024-09-18T00:20:07.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.914360+0000", 2024-09-18T00:20:07.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.914361+0000", 2024-09-18T00:20:07.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0699999999999996e-07 2024-09-18T00:20:07.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.914365+0000", 2024-09-18T00:20:07.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.301e-06 2024-09-18T00:20:07.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.914380+0000", 2024-09-18T00:20:07.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4329e-05 2024-09-18T00:20:07.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.914388+0000", 2024-09-18T00:20:07.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5520000000000005e-06 2024-09-18T00:20:07.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.914444+0000", 2024-09-18T00:20:07.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5952000000000002e-05 2024-09-18T00:20:07.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.914455+0000", 2024-09-18T00:20:07.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0456e-05 2024-09-18T00:20:07.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.914526+0000", 2024-09-18T00:20:07.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1037000000000004e-05 2024-09-18T00:20:07.929 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.929 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.929 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.929 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGRecoveryDelete(3.8 e124,122 3:128b0428:::rbd_data.112aa373d2ec.0000000000000004:1a,106'582)", 2024-09-18T00:20:07.929 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.923393+0000", 2024-09-18T00:20:07.929 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.93562831899999999, 2024-09-18T00:20:07.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019674199999999999, 2024-09-18T00:20:07.930 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.930 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.930 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.930 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.923393+0000", 2024-09-18T00:20:07.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.923393+0000", 2024-09-18T00:20:07.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.923393+0000", 2024-09-18T00:20:07.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4499999999999997e-07 2024-09-18T00:20:07.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.923397+0000", 2024-09-18T00:20:07.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1500000000000001e-06 2024-09-18T00:20:07.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.923412+0000", 2024-09-18T00:20:07.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4387e-05 2024-09-18T00:20:07.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.923420+0000", 2024-09-18T00:20:07.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4230000000000005e-06 2024-09-18T00:20:07.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.923476+0000", 2024-09-18T00:20:07.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5652000000000002e-05 2024-09-18T00:20:07.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.923486+0000", 2024-09-18T00:20:07.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8889999999999997e-06 2024-09-18T00:20:07.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.923589+0000", 2024-09-18T00:20:07.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103596 2024-09-18T00:20:07.933 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.933 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.933 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.934 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "MOSDPGRecoveryDelete(3.8 e124,122 3:116e0ba8:::rbd_data.112aa373d2ec.0000000000000019:1a,106'584)", 2024-09-18T00:20:07.934 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:06.925713+0000", 2024-09-18T00:20:07.934 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.93330753600000005, 2024-09-18T00:20:07.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019530400000000001, 2024-09-18T00:20:07.934 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.934 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.934 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:07.934 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.934 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.934 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.925713+0000", 2024-09-18T00:20:07.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.925713+0000", 2024-09-18T00:20:07.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.925714+0000", 2024-09-18T00:20:07.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1200000000000002e-07 2024-09-18T00:20:07.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.925719+0000", 2024-09-18T00:20:07.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3610000000000003e-06 2024-09-18T00:20:07.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.925733+0000", 2024-09-18T00:20:07.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4435e-05 2024-09-18T00:20:07.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.925741+0000", 2024-09-18T00:20:07.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3809999999999999e-06 2024-09-18T00:20:07.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.925797+0000", 2024-09-18T00:20:07.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5853000000000002e-05 2024-09-18T00:20:07.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.925807+0000", 2024-09-18T00:20:07.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6530000000000006e-06 2024-09-18T00:20:07.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:06.925909+0000", 2024-09-18T00:20:07.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101909 2024-09-18T00:20:07.937 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.937 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.937 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.938 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:62599 3.a e125/124 3:54661a3d:::rbd_data.112a13ec2d6c.0000000000000040:head v 125'402, mlcod=0'0)", 2024-09-18T00:20:07.938 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:07.619652+0000", 2024-09-18T00:20:07.938 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.23936871000000001, 2024-09-18T00:20:07.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00098891600000000006, 2024-09-18T00:20:07.938 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.938 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.938 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:07.938 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.619652+0000", 2024-09-18T00:20:07.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.939 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.619652+0000", 2024-09-18T00:20:07.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.939 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.619653+0000", 2024-09-18T00:20:07.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.167e-06 2024-09-18T00:20:07.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.939 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.939 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.619661+0000", 2024-09-18T00:20:07.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0430000000000008e-06 2024-09-18T00:20:07.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.619680+0000", 2024-09-18T00:20:07.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8091e-05 2024-09-18T00:20:07.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.619689+0000", 2024-09-18T00:20:07.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5519999999999993e-06 2024-09-18T00:20:07.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.619762+0000", 2024-09-18T00:20:07.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3115999999999996e-05 2024-09-18T00:20:07.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.619786+0000", 2024-09-18T00:20:07.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4207e-05 2024-09-18T00:20:07.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:07.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.620602+0000", 2024-09-18T00:20:07.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00081572800000000005 2024-09-18T00:20:07.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.620641+0000", 2024-09-18T00:20:07.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9011999999999999e-05 2024-09-18T00:20:07.942 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.942 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.942 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.942 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:62600 3.1a e125/124 3:5de5cd66:::rbd_data.112a13ec2d6c.0000000000000041:head v 125'217, mlcod=0'0)", 2024-09-18T00:20:07.942 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:07.622125+0000", 2024-09-18T00:20:07.942 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.236895825, 2024-09-18T00:20:07.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094333000000000004, 2024-09-18T00:20:07.942 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:07.942 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:07.943 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:07.943 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:07.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:07.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.622125+0000", 2024-09-18T00:20:07.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:07.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.622125+0000", 2024-09-18T00:20:07.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:07.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:07.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.622126+0000", 2024-09-18T00:20:07.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.2399999999999996e-07 2024-09-18T00:20:07.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:07.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.622134+0000", 2024-09-18T00:20:07.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9079999999999995e-06 2024-09-18T00:20:07.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:07.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.622150+0000", 2024-09-18T00:20:07.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6354999999999998e-05 2024-09-18T00:20:07.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.944 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:07.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.622160+0000", 2024-09-18T00:20:07.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8099999999999992e-06 2024-09-18T00:20:07.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:07.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.622210+0000", 2024-09-18T00:20:07.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0172e-05 2024-09-18T00:20:07.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:07.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.622226+0000", 2024-09-18T00:20:07.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6177000000000001e-05 2024-09-18T00:20:07.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:07.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.622964+0000", 2024-09-18T00:20:07.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000737157 2024-09-18T00:20:07.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:07.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:07.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:07.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.623068+0000", 2024-09-18T00:20:07.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104827 2024-09-18T00:20:07.946 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.946 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.946 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.946 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:07.946 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:07.946 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:07.948 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:20:07.957 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:07.953+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:08.058 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:08.056+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:08.117 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:08.117 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:08.117 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:20:08.117 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:08.131 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:20:08.177 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:08.176+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:08.260 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:08.256+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:08.304 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:20:08.305 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:08.318 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:20:08.361 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:20:08.360+0000 7f8bf35a1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 40533) UID: 0 2024-09-18T00:20:08.462 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:08.460+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:08.497 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:08.497 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:20:08.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:20:08.497 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:20:08.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.497 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2230 3.17 3:ef492888:::rbd_data.112aa7ba4578.000000000000014f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:20:08.497 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:20:08.498 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 108.22871320199999, 2024-09-18T00:20:08.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075065283999999996, 2024-09-18T00:20:08.498 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.498 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.498 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:08.498 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:08.498 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:08.498 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:08.498 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2230 2024-09-18T00:20:08.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.498 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:20:08.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266011+0000", 2024-09-18T00:20:08.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266014+0000", 2024-09-18T00:20:08.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.207e-06 2024-09-18T00:20:08.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266029+0000", 2024-09-18T00:20:08.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4616e-05 2024-09-18T00:20:08.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266053+0000", 2024-09-18T00:20:08.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4369999999999999e-05 2024-09-18T00:20:08.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266067+0000", 2024-09-18T00:20:08.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4243999999999999e-05 2024-09-18T00:20:08.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266157+0000", 2024-09-18T00:20:08.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9909999999999998e-05 2024-09-18T00:20:08.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.266327+0000", 2024-09-18T00:20:08.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016981200000000001 2024-09-18T00:20:08.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:20:08.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.269180+0000", 2024-09-18T00:20:08.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028526559999999999 2024-09-18T00:20:08.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:08.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340955+0000", 2024-09-18T00:20:08.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.071774904 2024-09-18T00:20:08.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:08.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340974+0000", 2024-09-18T00:20:08.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8824000000000001e-05 2024-09-18T00:20:08.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:08.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341028+0000", 2024-09-18T00:20:08.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4456999999999997e-05 2024-09-18T00:20:08.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.503 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.341076+0000", 2024-09-18T00:20:08.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8284000000000001e-05 2024-09-18T00:20:08.503 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.503 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.503 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.503 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2230 3.17 e36/23 ondisk, result = 0)", 2024-09-18T00:20:08.504 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:20:08.504 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 108.211369621, 2024-09-18T00:20:08.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057609645000000001, 2024-09-18T00:20:08.504 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.504 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.504 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:08.504 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:20:08.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283355+0000", 2024-09-18T00:20:08.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283356+0000", 2024-09-18T00:20:08.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.44e-06 2024-09-18T00:20:08.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283362+0000", 2024-09-18T00:20:08.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2539999999999998e-06 2024-09-18T00:20:08.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283382+0000", 2024-09-18T00:20:08.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9896000000000001e-05 2024-09-18T00:20:08.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.283398+0000", 2024-09-18T00:20:08.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5489999999999999e-05 2024-09-18T00:20:08.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.507 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340936+0000", 2024-09-18T00:20:08.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057538649999999997 2024-09-18T00:20:08.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340947+0000", 2024-09-18T00:20:08.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0981e-05 2024-09-18T00:20:08.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.340964+0000", 2024-09-18T00:20:08.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6934000000000001e-05 2024-09-18T00:20:08.508 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.508 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.508 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.509 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:2419 3.17 3:ed94b133:::rbd_data.112aa7ba4578.00000000000001b5:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:20:08.509 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:20:08.509 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 107.757248148, 2024-09-18T00:20:08.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.064761812000000002, 2024-09-18T00:20:08.509 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.509 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.509 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:08.509 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:08.509 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:08.509 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:08.509 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 2419 2024-09-18T00:20:08.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.510 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:20:08.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737476+0000", 2024-09-18T00:20:08.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737479+0000", 2024-09-18T00:20:08.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6089999999999999e-06 2024-09-18T00:20:08.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737487+0000", 2024-09-18T00:20:08.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7900000000000005e-06 2024-09-18T00:20:08.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737507+0000", 2024-09-18T00:20:08.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9936e-05 2024-09-18T00:20:08.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737518+0000", 2024-09-18T00:20:08.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1046999999999999e-05 2024-09-18T00:20:08.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737599+0000", 2024-09-18T00:20:08.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0697000000000003e-05 2024-09-18T00:20:08.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.737697+0000", 2024-09-18T00:20:08.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8058000000000002e-05 2024-09-18T00:20:08.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 5", 2024-09-18T00:20:08.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.740445+0000", 2024-09-18T00:20:08.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0027481279999999999 2024-09-18T00:20:08.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:08.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802023+0000", 2024-09-18T00:20:08.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.061577321999999997 2024-09-18T00:20:08.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:08.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802060+0000", 2024-09-18T00:20:08.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7586000000000002e-05 2024-09-18T00:20:08.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:08.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802158+0000", 2024-09-18T00:20:08.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7583000000000004e-05 2024-09-18T00:20:08.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802238+0000", 2024-09-18T00:20:08.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0055999999999994e-05 2024-09-18T00:20:08.514 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.514 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.515 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.515 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:2419 3.17 e37/23 ondisk, result = 0)", 2024-09-18T00:20:08.515 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:20:08.515 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 107.745023202, 2024-09-18T00:20:08.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.052342948, 2024-09-18T00:20:08.515 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.515 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.515 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:08.515 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.515 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:20:08.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749701+0000", 2024-09-18T00:20:08.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.516 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.516 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749702+0000", 2024-09-18T00:20:08.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1000000000000001e-06 2024-09-18T00:20:08.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749710+0000", 2024-09-18T00:20:08.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7279999999999995e-06 2024-09-18T00:20:08.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749736+0000", 2024-09-18T00:20:08.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5797999999999999e-05 2024-09-18T00:20:08.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.517 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.517 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.749751+0000", 2024-09-18T00:20:08.517 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5238999999999999e-05 2024-09-18T00:20:08.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801745+0000", 2024-09-18T00:20:08.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051994022000000001 2024-09-18T00:20:08.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.801770+0000", 2024-09-18T00:20:08.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5103e-05 2024-09-18T00:20:08.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:20.802044+0000", 2024-09-18T00:20:08.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027395799999999998 2024-09-18T00:20:08.519 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.519 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.519 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.519 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6503 3.1b 3:d9387a1f:::rbd_data.112a3a39fc45.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:20:08.519 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:20:08.519 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 102.930350232, 2024-09-18T00:20:08.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050510547000000003, 2024-09-18T00:20:08.520 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.520 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.520 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:08.520 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:08.520 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:08.520 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:08.520 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6503 2024-09-18T00:20:08.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.520 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:20:08.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564374+0000", 2024-09-18T00:20:08.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564376+0000", 2024-09-18T00:20:08.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1019999999999999e-06 2024-09-18T00:20:08.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564386+0000", 2024-09-18T00:20:08.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0312999999999999e-05 2024-09-18T00:20:08.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564408+0000", 2024-09-18T00:20:08.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2001000000000002e-05 2024-09-18T00:20:08.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564421+0000", 2024-09-18T00:20:08.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2271e-05 2024-09-18T00:20:08.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.523 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564491+0000", 2024-09-18T00:20:08.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0630000000000006e-05 2024-09-18T00:20:08.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.523 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.564606+0000", 2024-09-18T00:20:08.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011449 2024-09-18T00:20:08.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:08.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.566743+0000", 2024-09-18T00:20:08.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0021373830000000001 2024-09-18T00:20:08.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:08.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614738+0000", 2024-09-18T00:20:08.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047995359000000001 2024-09-18T00:20:08.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:08.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614759+0000", 2024-09-18T00:20:08.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1010999999999999e-05 2024-09-18T00:20:08.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:08.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614826+0000", 2024-09-18T00:20:08.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6654000000000003e-05 2024-09-18T00:20:08.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.614884+0000", 2024-09-18T00:20:08.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8332999999999999e-05 2024-09-18T00:20:08.526 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.526 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.526 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.526 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6518 3.1 3:8061bf9f:::rbd_data.112a3a39fc45.000000000000001a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:20:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:20:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 102.84391195800001, 2024-09-18T00:20:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062303123000000002, 2024-09-18T00:20:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6518 2024-09-18T00:20:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:20:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650812+0000", 2024-09-18T00:20:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650814+0000", 2024-09-18T00:20:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.629e-06 2024-09-18T00:20:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650823+0000", 2024-09-18T00:20:08.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.1800000000000002e-06 2024-09-18T00:20:08.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650843+0000", 2024-09-18T00:20:08.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9836000000000001e-05 2024-09-18T00:20:08.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650854+0000", 2024-09-18T00:20:08.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1564e-05 2024-09-18T00:20:08.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.650917+0000", 2024-09-18T00:20:08.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3076000000000004e-05 2024-09-18T00:20:08.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.651029+0000", 2024-09-18T00:20:08.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111154 2024-09-18T00:20:08.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:08.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.655531+0000", 2024-09-18T00:20:08.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0045027629999999999 2024-09-18T00:20:08.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:08.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712924+0000", 2024-09-18T00:20:08.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.057393006000000003 2024-09-18T00:20:08.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:08.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712988+0000", 2024-09-18T00:20:08.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3670000000000005e-05 2024-09-18T00:20:08.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:08.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713056+0000", 2024-09-18T00:20:08.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7904999999999994e-05 2024-09-18T00:20:08.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713115+0000", 2024-09-18T00:20:08.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9339999999999998e-05 2024-09-18T00:20:08.534 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.534 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.534 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.534 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6518 3.1 e42/23 ondisk, result = 0)", 2024-09-18T00:20:08.534 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:20:08.534 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 102.831486875, 2024-09-18T00:20:08.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049887304, 2024-09-18T00:20:08.535 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.535 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.535 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:08.535 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:20:08.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.535 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663237+0000", 2024-09-18T00:20:08.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663239+0000", 2024-09-18T00:20:08.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6199999999999999e-06 2024-09-18T00:20:08.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663246+0000", 2024-09-18T00:20:08.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4189999999999996e-06 2024-09-18T00:20:08.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663265+0000", 2024-09-18T00:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8323000000000001e-05 2024-09-18T00:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.663277+0000", 2024-09-18T00:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2065999999999999e-05 2024-09-18T00:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712965+0000", 2024-09-18T00:20:08.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.049688317000000003 2024-09-18T00:20:08.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.712977+0000", 2024-09-18T00:20:08.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.206e-05 2024-09-18T00:20:08.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.713125+0000", 2024-09-18T00:20:08.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014749899999999999 2024-09-18T00:20:08.538 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.538 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.538 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.539 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:8831 3.12 3:49cac8cf:::rbd_data.112a9f4259df.0000000000000130:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-18T00:20:08.539 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:20:08.539 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 96.213626626000007, 2024-09-18T00:20:08.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104125047, 2024-09-18T00:20:08.539 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.539 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.539 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:08.539 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:08.539 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:08.539 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:08.539 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 8831 2024-09-18T00:20:08.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.540 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:20:08.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281098+0000", 2024-09-18T00:20:08.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281099+0000", 2024-09-18T00:20:08.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.029e-06 2024-09-18T00:20:08.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281103+0000", 2024-09-18T00:20:08.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7759999999999997e-06 2024-09-18T00:20:08.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281114+0000", 2024-09-18T00:20:08.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1090000000000001e-05 2024-09-18T00:20:08.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281120+0000", 2024-09-18T00:20:08.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.604e-06 2024-09-18T00:20:08.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281164+0000", 2024-09-18T00:20:08.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3696999999999999e-05 2024-09-18T00:20:08.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.542 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.542 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.281270+0000", 2024-09-18T00:20:08.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000106041 2024-09-18T00:20:08.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:08.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.285596+0000", 2024-09-18T00:20:08.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0043266659999999998 2024-09-18T00:20:08.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:08.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385028+0000", 2024-09-18T00:20:08.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.099431808999999996 2024-09-18T00:20:08.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:08.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385094+0000", 2024-09-18T00:20:08.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5740000000000004e-05 2024-09-18T00:20:08.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:08.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385166+0000", 2024-09-18T00:20:08.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1637000000000005e-05 2024-09-18T00:20:08.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385223+0000", 2024-09-18T00:20:08.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6957999999999999e-05 2024-09-18T00:20:08.544 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.544 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.545 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.545 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:8831 3.12 e48/32 ondisk, result = 0)", 2024-09-18T00:20:08.545 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:20:08.545 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 96.197112516000004, 2024-09-18T00:20:08.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087621558000000002, 2024-09-18T00:20:08.545 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.545 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.545 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:08.545 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:20:08.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297612+0000", 2024-09-18T00:20:08.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297613+0000", 2024-09-18T00:20:08.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3400000000000001e-06 2024-09-18T00:20:08.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297620+0000", 2024-09-18T00:20:08.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5869999999999998e-06 2024-09-18T00:20:08.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297639+0000", 2024-09-18T00:20:08.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9888999999999999e-05 2024-09-18T00:20:08.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.297661+0000", 2024-09-18T00:20:08.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1772000000000001e-05 2024-09-18T00:20:08.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385073+0000", 2024-09-18T00:20:08.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.087411303999999995 2024-09-18T00:20:08.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385086+0000", 2024-09-18T00:20:08.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3040000000000001e-05 2024-09-18T00:20:08.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.549 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.549 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:32.385233+0000", 2024-09-18T00:20:08.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014762600000000001 2024-09-18T00:20:08.549 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.549 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.549 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.549 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.549 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22269 3.17 3:e9a1eb35:::rbd_data.112a948d5956.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:20:08.550 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:20:08.550 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 70.58558069, 2024-09-18T00:20:08.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.063462693000000001, 2024-09-18T00:20:08.550 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.550 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.550 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:08.550 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:08.550 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:08.550 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:08.550 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22269 2024-09-18T00:20:08.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.550 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:20:08.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909143+0000", 2024-09-18T00:20:08.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909146+0000", 2024-09-18T00:20:08.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3700000000000002e-06 2024-09-18T00:20:08.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909160+0000", 2024-09-18T00:20:08.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3825000000000001e-05 2024-09-18T00:20:08.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909178+0000", 2024-09-18T00:20:08.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8567999999999999e-05 2024-09-18T00:20:08.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909189+0000", 2024-09-18T00:20:08.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0881e-05 2024-09-18T00:20:08.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909270+0000", 2024-09-18T00:20:08.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0846999999999994e-05 2024-09-18T00:20:08.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.909383+0000", 2024-09-18T00:20:08.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000112704 2024-09-18T00:20:08.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:08.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972310+0000", 2024-09-18T00:20:08.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062927081999999995 2024-09-18T00:20:08.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:08.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972433+0000", 2024-09-18T00:20:08.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00012323799999999999 2024-09-18T00:20:08.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.972606+0000", 2024-09-18T00:20:08.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017317799999999999 2024-09-18T00:20:08.555 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.555 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.555 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.556 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.556 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22379 3.17 3:e9a1eb35:::rbd_data.112a948d5956.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:20:08.556 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:20:08.556 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 70.091505393000006, 2024-09-18T00:20:08.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.12560279099999999, 2024-09-18T00:20:08.556 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.556 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.556 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:08.556 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:08.556 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:08.556 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:08.557 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22379 2024-09-18T00:20:08.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.557 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:20:08.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.557 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.557 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403219+0000", 2024-09-18T00:20:08.557 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403221+0000", 2024-09-18T00:20:08.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3269999999999999e-06 2024-09-18T00:20:08.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.558 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.558 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403232+0000", 2024-09-18T00:20:08.558 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1392e-05 2024-09-18T00:20:08.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.558 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403264+0000", 2024-09-18T00:20:08.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1146999999999998e-05 2024-09-18T00:20:08.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403279+0000", 2024-09-18T00:20:08.559 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5109000000000001e-05 2024-09-18T00:20:08.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.559 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.559 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.559 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403424+0000", 2024-09-18T00:20:08.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014516999999999999 2024-09-18T00:20:08.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.403567+0000", 2024-09-18T00:20:08.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142824 2024-09-18T00:20:08.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.560 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:08.560 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528723+0000", 2024-09-18T00:20:08.560 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.12515647299999999 2024-09-18T00:20:08.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.560 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:08.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528764+0000", 2024-09-18T00:20:08.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0624000000000003e-05 2024-09-18T00:20:08.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.561 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.561 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.561 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.528822+0000", 2024-09-18T00:20:08.561 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7725e-05 2024-09-18T00:20:08.561 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.561 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.561 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.562 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.562 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39268 3.1f 3:fdf3a436:::rbd_data.112a3496573c.0000000000000001:17 [read 352256~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:08.562 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.576221+0000", 2024-09-18T00:20:08.562 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.918503350000002, 2024-09-18T00:20:08.562 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047293515000000001, 2024-09-18T00:20:08.562 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.562 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.562 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:08.562 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:08.562 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:08.562 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:08.563 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39268 2024-09-18T00:20:08.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.563 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.563 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576221+0000", 2024-09-18T00:20:08.563 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.563 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.563 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576221+0000", 2024-09-18T00:20:08.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576221+0000", 2024-09-18T00:20:08.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.75e-07 2024-09-18T00:20:08.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.564 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.564 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.564 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576229+0000", 2024-09-18T00:20:08.564 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9950000000000005e-06 2024-09-18T00:20:08.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576237+0000", 2024-09-18T00:20:08.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.0010000000000001e-06 2024-09-18T00:20:08.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.565 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576243+0000", 2024-09-18T00:20:08.565 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.592e-06 2024-09-18T00:20:08.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.565 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.565 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623336+0000", 2024-09-18T00:20:08.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047092698000000002 2024-09-18T00:20:08.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623418+0000", 2024-09-18T00:20:08.566 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1977000000000005e-05 2024-09-18T00:20:08.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.566 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.566 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.566 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623514+0000", 2024-09-18T00:20:08.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6676999999999996e-05 2024-09-18T00:20:08.567 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.567 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.567 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.567 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.567 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39269 3.1f 3:fdf3a436:::rbd_data.112a3496573c.0000000000000001:17 [read 362496~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:08.567 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.576250+0000", 2024-09-18T00:20:08.567 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.918474439000001, 2024-09-18T00:20:08.567 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047454169999999997, 2024-09-18T00:20:08.567 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.567 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.568 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:08.568 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:08.568 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:08.568 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:08.568 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39269 2024-09-18T00:20:08.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.568 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.568 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.568 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576250+0000", 2024-09-18T00:20:08.568 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.568 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576250+0000", 2024-09-18T00:20:08.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.569 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.569 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576250+0000", 2024-09-18T00:20:08.569 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:20:08.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.569 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576252+0000", 2024-09-18T00:20:08.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1739999999999999e-06 2024-09-18T00:20:08.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576258+0000", 2024-09-18T00:20:08.570 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6670000000000001e-06 2024-09-18T00:20:08.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.570 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.570 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.570 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576264+0000", 2024-09-18T00:20:08.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1360000000000002e-06 2024-09-18T00:20:08.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623543+0000", 2024-09-18T00:20:08.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047278496000000003 2024-09-18T00:20:08.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.571 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623609+0000", 2024-09-18T00:20:08.571 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5901000000000005e-05 2024-09-18T00:20:08.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.571 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.571 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623704+0000", 2024-09-18T00:20:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5291999999999995e-05 2024-09-18T00:20:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39270 3.1f 3:fdf3a436:::rbd_data.112a3496573c.0000000000000001:17 [read 372736~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.576271+0000", 2024-09-18T00:20:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.918453110000002, 2024-09-18T00:20:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047626546999999998, 2024-09-18T00:20:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.572 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39270 2024-09-18T00:20:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576271+0000", 2024-09-18T00:20:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.573 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576271+0000", 2024-09-18T00:20:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576271+0000", 2024-09-18T00:20:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.01e-07 2024-09-18T00:20:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576274+0000", 2024-09-18T00:20:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1150000000000001e-06 2024-09-18T00:20:08.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576280+0000", 2024-09-18T00:20:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.9399999999999999e-06 2024-09-18T00:20:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.576285+0000", 2024-09-18T00:20:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0719999999999999e-06 2024-09-18T00:20:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623731+0000", 2024-09-18T00:20:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047446629999999997 2024-09-18T00:20:08.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623809+0000", 2024-09-18T00:20:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8141999999999998e-05 2024-09-18T00:20:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.623898+0000", 2024-09-18T00:20:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8246999999999997e-05 2024-09-18T00:20:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39360 3.1f 3:fec8fa94:::rbd_data.112a3496573c.0000000000000006:17 [read 354304~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.579954+0000", 2024-09-18T00:20:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.914770261999998, 2024-09-18T00:20:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047250546999999997, 2024-09-18T00:20:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39360 2024-09-18T00:20:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.577 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579954+0000", 2024-09-18T00:20:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579954+0000", 2024-09-18T00:20:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579954+0000", 2024-09-18T00:20:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7800000000000002e-07 2024-09-18T00:20:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579956+0000", 2024-09-18T00:20:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.114e-06 2024-09-18T00:20:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579962+0000", 2024-09-18T00:20:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7400000000000001e-06 2024-09-18T00:20:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579967+0000", 2024-09-18T00:20:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.792e-06 2024-09-18T00:20:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.580 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627056+0000", 2024-09-18T00:20:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047089131999999999 2024-09-18T00:20:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627122+0000", 2024-09-18T00:20:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6229000000000006e-05 2024-09-18T00:20:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627204+0000", 2024-09-18T00:20:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.2062000000000001e-05 2024-09-18T00:20:08.581 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39361 3.1f 3:fec8fa94:::rbd_data.112a3496573c.0000000000000006:17 [read 364544~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.579975+0000", 2024-09-18T00:20:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.914749106000002, 2024-09-18T00:20:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047405364999999998, 2024-09-18T00:20:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.582 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39361 2024-09-18T00:20:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579975+0000", 2024-09-18T00:20:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579975+0000", 2024-09-18T00:20:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579976+0000", 2024-09-18T00:20:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7399999999999998e-07 2024-09-18T00:20:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579978+0000", 2024-09-18T00:20:08.584 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5359999999999999e-06 2024-09-18T00:20:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579984+0000", 2024-09-18T00:20:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2469999999999997e-06 2024-09-18T00:20:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.585 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.579989+0000", 2024-09-18T00:20:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6700000000000002e-06 2024-09-18T00:20:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627232+0000", 2024-09-18T00:20:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047243016999999998 2024-09-18T00:20:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.586 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627299+0000", 2024-09-18T00:20:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7145999999999997e-05 2024-09-18T00:20:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.627380+0000", 2024-09-18T00:20:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1274999999999997e-05 2024-09-18T00:20:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:39473 3.1f 3:fd448eba:::rbd_data.112a3496573c.000000000000000c:17 [read 366592~2048] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.583299+0000", 2024-09-18T00:20:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 47.911424785000001, 2024-09-18T00:20:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047411813999999997, 2024-09-18T00:20:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:08.588 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 39473 2024-09-18T00:20:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583299+0000", 2024-09-18T00:20:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583299+0000", 2024-09-18T00:20:08.589 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583300+0000", 2024-09-18T00:20:08.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6699999999999999e-07 2024-09-18T00:20:08.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.590 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.590 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583302+0000", 2024-09-18T00:20:08.590 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.979e-06 2024-09-18T00:20:08.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583308+0000", 2024-09-18T00:20:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.835e-06 2024-09-18T00:20:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.583313+0000", 2024-09-18T00:20:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9409999999999998e-06 2024-09-18T00:20:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.591 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.630537+0000", 2024-09-18T00:20:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.047224592000000003 2024-09-18T00:20:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.630623+0000", 2024-09-18T00:20:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5473e-05 2024-09-18T00:20:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.592 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.630711+0000", 2024-09-18T00:20:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8527000000000001e-05 2024-09-18T00:20:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.596 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:54153 3.b e90/23 3:d3f7b2f3:::rbd_data.112a859a01c9.0000000000000001:head v 90'238, mlcod=90'236)", 2024-09-18T00:20:08.596 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:30.858396+0000", 2024-09-18T00:20:08.596 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 37.636328417000001, 2024-09-18T00:20:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056676272999999999, 2024-09-18T00:20:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858396+0000", 2024-09-18T00:20:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858396+0000", 2024-09-18T00:20:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858398+0000", 2024-09-18T00:20:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.838e-06 2024-09-18T00:20:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.598 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860747+0000", 2024-09-18T00:20:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0023496160000000001 2024-09-18T00:20:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860777+0000", 2024-09-18T00:20:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9754e-05 2024-09-18T00:20:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860791+0000", 2024-09-18T00:20:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.377e-05 2024-09-18T00:20:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.599 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860879+0000", 2024-09-18T00:20:08.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7818999999999994e-05 2024-09-18T00:20:08.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.860917+0000", 2024-09-18T00:20:08.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8124999999999998e-05 2024-09-18T00:20:08.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.600 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:08.600 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915020+0000", 2024-09-18T00:20:08.600 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.054103126000000001 2024-09-18T00:20:08.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915072+0000", 2024-09-18T00:20:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2225000000000002e-05 2024-09-18T00:20:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:58293 3.1b 3:d96f405d:::rbd_data.112a17f95ba0.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T00:20:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:38.838123+0000", 2024-09-18T00:20:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.656600775000001, 2024-09-18T00:20:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11992721000000001, 2024-09-18T00:20:08.601 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 58293 2024-09-18T00:20:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838123+0000", 2024-09-18T00:20:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838123+0000", 2024-09-18T00:20:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838125+0000", 2024-09-18T00:20:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.158e-06 2024-09-18T00:20:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838130+0000", 2024-09-18T00:20:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7019999999999998e-06 2024-09-18T00:20:08.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838145+0000", 2024-09-18T00:20:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4871999999999999e-05 2024-09-18T00:20:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838152+0000", 2024-09-18T00:20:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3309999999999998e-06 2024-09-18T00:20:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838227+0000", 2024-09-18T00:20:08.604 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4685999999999996e-05 2024-09-18T00:20:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.838375+0000", 2024-09-18T00:20:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014751 2024-09-18T00:20:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.845187+0000", 2024-09-18T00:20:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0068119749999999996 2024-09-18T00:20:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:08.605 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.957945+0000", 2024-09-18T00:20:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.112758517 2024-09-18T00:20:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.957963+0000", 2024-09-18T00:20:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7580000000000001e-05 2024-09-18T00:20:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.958009+0000", 2024-09-18T00:20:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6377e-05 2024-09-18T00:20:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.606 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.958051+0000", 2024-09-18T00:20:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1501999999999997e-05 2024-09-18T00:20:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:58293 3.1b e96/23 ondisk, result = 0)", 2024-09-18T00:20:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:38.867986+0000", 2024-09-18T00:20:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.626738661000001, 2024-09-18T00:20:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.089969120999999999, 2024-09-18T00:20:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:08.607 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.867986+0000", 2024-09-18T00:20:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.867986+0000", 2024-09-18T00:20:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.867987+0000", 2024-09-18T00:20:08.608 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9649999999999998e-06 2024-09-18T00:20:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.867996+0000", 2024-09-18T00:20:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3259999999999996e-06 2024-09-18T00:20:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.868030+0000", 2024-09-18T00:20:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4428999999999999e-05 2024-09-18T00:20:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.609 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.868050+0000", 2024-09-18T00:20:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9368000000000002e-05 2024-09-18T00:20:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.957905+0000", 2024-09-18T00:20:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.089855395000000005 2024-09-18T00:20:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.957915+0000", 2024-09-18T00:20:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.7280000000000007e-06 2024-09-18T00:20:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:08.610 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:38.957955+0000", 2024-09-18T00:20:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9910000000000002e-05 2024-09-18T00:20:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:08.611 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:08.611 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:08.611 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 7, 1] out_osds: [6] dead_osds: [] live_osds: [0, 2, 3, 4, 6, 7, 5, 1] 2024-09-18T00:20:08.612 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T00:20:08.612 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 2, 3, 4, 6, 7, 5, 1] 2024-09-18T00:20:08.612 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-18T00:20:08.612 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T00:20:08.614 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:20:08.654 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:08.652+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:08.676 INFO:tasks.ceph.osd.1:Stopped 2024-09-18T00:20:08.677 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.1 2024-09-18T00:20:08.677 INFO:tasks.thrashosds.thrasher:remote for osd 1 is ubuntu@smithi097.front.sepia.ceph.com 2024-09-18T00:20:08.677 DEBUG:teuthology.orchestra.run.smithi097:> type ceph-objectstore-tool 2024-09-18T00:20:08.715 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:08.713+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:08.721 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T00:20:08.722 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op list-pgs 2024-09-18T00:20:08.782 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:08.782 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:08.782 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:20:08.782 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:08.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:20:08.816 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:08.813+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:08.917 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:08.913+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:08.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T00:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:20:08.970 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:08.984 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:20:09.018 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:09.017+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:09.094 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:20:09.094 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:20:09.119 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:09.117+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:09.154 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:09.154 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:20:09.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:20:09.154 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:20:09.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.154 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:5722 3.4 e41/23 3:20381d86:::rbd_data.112a3a39fc45.000000000000001d:head v 41'107, mlcod=40'105)", 2024-09-18T00:20:09.154 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:20:09.154 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 103.81495688699999, 2024-09-18T00:20:09.154 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01185172, 2024-09-18T00:20:09.154 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.155 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.155 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.155 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:20:09.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.155 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:20:09.155 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.155 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.155 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.155 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335497+0000", 2024-09-18T00:20:09.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.505e-06 2024-09-18T00:20:09.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335945+0000", 2024-09-18T00:20:09.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000447687 2024-09-18T00:20:09.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.156 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.156 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.156 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335961+0000", 2024-09-18T00:20:09.156 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6430999999999999e-05 2024-09-18T00:20:09.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.157 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.157 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335969+0000", 2024-09-18T00:20:09.157 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:20:09.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.157 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336030+0000", 2024-09-18T00:20:09.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1255999999999998e-05 2024-09-18T00:20:09.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.158 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336061+0000", 2024-09-18T00:20:09.158 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0579e-05 2024-09-18T00:20:09.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.158 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.158 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347296+0000", 2024-09-18T00:20:09.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011235663 2024-09-18T00:20:09.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.159 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.159 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347347+0000", 2024-09-18T00:20:09.159 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0914000000000001e-05 2024-09-18T00:20:09.159 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.159 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.159 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.159 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.159 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11457 3.0 3:0214e7b3:::rbd_data.112ab1fa102a.0000000000000150:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:20:09.159 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:20:09.160 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 93.890590949, 2024-09-18T00:20:09.160 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028499165, 2024-09-18T00:20:09.160 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.160 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.160 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.160 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.160 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11457 2024-09-18T00:20:09.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.160 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.160 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.160 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:20:09.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:20:09.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.161 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.161 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.161 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259865+0000", 2024-09-18T00:20:09.161 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7230000000000002e-06 2024-09-18T00:20:09.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259883+0000", 2024-09-18T00:20:09.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8067000000000002e-05 2024-09-18T00:20:09.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259924+0000", 2024-09-18T00:20:09.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0550999999999999e-05 2024-09-18T00:20:09.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.162 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.162 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.162 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259943+0000", 2024-09-18T00:20:09.162 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:20:09.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260054+0000", 2024-09-18T00:20:09.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011075 2024-09-18T00:20:09.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260138+0000", 2024-09-18T00:20:09.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4549000000000007e-05 2024-09-18T00:20:09.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.163 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.163 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.287926+0000", 2024-09-18T00:20:09.163 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027787642000000001 2024-09-18T00:20:09.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288090+0000", 2024-09-18T00:20:09.164 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016402499999999999 2024-09-18T00:20:09.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.164 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.164 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.164 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288361+0000", 2024-09-18T00:20:09.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027097800000000001 2024-09-18T00:20:09.165 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.165 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.165 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.165 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.165 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18661 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:09.165 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:20:09.165 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 76.215823083999993, 2024-09-18T00:20:09.165 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020008037999999999, 2024-09-18T00:20:09.165 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.165 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.165 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.166 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.166 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.166 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.166 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18661 2024-09-18T00:20:09.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.166 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.166 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.166 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.166 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:20:09.166 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:20:09.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.167 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.167 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934633+0000", 2024-09-18T00:20:09.167 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8e-06 2024-09-18T00:20:09.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.167 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935511+0000", 2024-09-18T00:20:09.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087785700000000001 2024-09-18T00:20:09.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935561+0000", 2024-09-18T00:20:09.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9472000000000003e-05 2024-09-18T00:20:09.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.168 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.168 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935586+0000", 2024-09-18T00:20:09.168 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5616e-05 2024-09-18T00:20:09.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935838+0000", 2024-09-18T00:20:09.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025202699999999999 2024-09-18T00:20:09.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.936023+0000", 2024-09-18T00:20:09.169 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018480300000000001 2024-09-18T00:20:09.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.169 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.169 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.169 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953545+0000", 2024-09-18T00:20:09.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017522157 2024-09-18T00:20:09.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953723+0000", 2024-09-18T00:20:09.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017753800000000001 2024-09-18T00:20:09.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.170 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.170 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.170 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954637+0000", 2024-09-18T00:20:09.170 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091476800000000005 2024-09-18T00:20:09.170 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.170 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.170 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.171 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.171 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18662 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:09.171 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:20:09.171 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 76.214818147000003, 2024-09-18T00:20:09.171 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019388843999999999, 2024-09-18T00:20:09.171 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.171 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.171 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.171 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.171 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18662 2024-09-18T00:20:09.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.172 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:20:09.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:20:09.172 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.172 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.172 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.172 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935636+0000", 2024-09-18T00:20:09.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.663e-06 2024-09-18T00:20:09.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935646+0000", 2024-09-18T00:20:09.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000006e-06 2024-09-18T00:20:09.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.173 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935672+0000", 2024-09-18T00:20:09.173 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5868999999999999e-05 2024-09-18T00:20:09.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.173 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935692+0000", 2024-09-18T00:20:09.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0604000000000001e-05 2024-09-18T00:20:09.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950738+0000", 2024-09-18T00:20:09.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015045868 2024-09-18T00:20:09.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.174 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.174 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950992+0000", 2024-09-18T00:20:09.174 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025368400000000001 2024-09-18T00:20:09.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.174 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954674+0000", 2024-09-18T00:20:09.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003682154 2024-09-18T00:20:09.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954806+0000", 2024-09-18T00:20:09.175 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013252000000000001 2024-09-18T00:20:09.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.175 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.175 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.175 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.955023+0000", 2024-09-18T00:20:09.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021681200000000001 2024-09-18T00:20:09.176 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.176 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.176 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.176 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.176 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19382 3.4 3:22f09d3a:::rbd_data.112a7fe29ab4.000000000000015a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:09.176 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:20:09.176 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 75.528408096999996, 2024-09-18T00:20:09.176 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075119302999999998, 2024-09-18T00:20:09.176 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.176 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.176 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.177 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.177 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.177 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.177 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19382 2024-09-18T00:20:09.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.177 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.177 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:20:09.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:20:09.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.178 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.178 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.178 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622045+0000", 2024-09-18T00:20:09.178 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0020000000000001e-06 2024-09-18T00:20:09.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622053+0000", 2024-09-18T00:20:09.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2679999999999997e-06 2024-09-18T00:20:09.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.179 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622062+0000", 2024-09-18T00:20:09.179 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3749999999999992e-06 2024-09-18T00:20:09.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.179 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.179 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622068+0000", 2024-09-18T00:20:09.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5729999999999998e-06 2024-09-18T00:20:09.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622118+0000", 2024-09-18T00:20:09.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0696999999999999e-05 2024-09-18T00:20:09.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.180 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.180 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622222+0000", 2024-09-18T00:20:09.180 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103286 2024-09-18T00:20:09.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.180 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697061+0000", 2024-09-18T00:20:09.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074839141999999997 2024-09-18T00:20:09.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697102+0000", 2024-09-18T00:20:09.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1513e-05 2024-09-18T00:20:09.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.181 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.181 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.181 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697164+0000", 2024-09-18T00:20:09.181 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1446999999999996e-05 2024-09-18T00:20:09.182 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.182 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.182 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.182 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.182 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20246 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:09.182 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:20:09.182 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.994719904999997, 2024-09-18T00:20:09.182 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01018784, 2024-09-18T00:20:09.182 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.182 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.182 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:09.183 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.183 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.183 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.183 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20246 2024-09-18T00:20:09.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.183 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.183 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.183 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.183 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:20:09.183 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:20:09.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:20:09.184 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2599999999999998e-07 2024-09-18T00:20:09.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.184 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.184 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.184 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155735+0000", 2024-09-18T00:20:09.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2699999999999999e-06 2024-09-18T00:20:09.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155744+0000", 2024-09-18T00:20:09.185 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8629999999999997e-06 2024-09-18T00:20:09.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.185 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.185 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.185 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155749+0000", 2024-09-18T00:20:09.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0359999999999997e-06 2024-09-18T00:20:09.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165748+0000", 2024-09-18T00:20:09.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099991449999999992 2024-09-18T00:20:09.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.186 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.186 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.186 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165819+0000", 2024-09-18T00:20:09.186 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0964999999999996e-05 2024-09-18T00:20:09.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.187 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.187 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165920+0000", 2024-09-18T00:20:09.187 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101135 2024-09-18T00:20:09.187 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.187 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.187 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.187 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.187 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20247 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:09.188 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:20:09.188 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.994696563000005, 2024-09-18T00:20:09.188 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010443987, 2024-09-18T00:20:09.188 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.188 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.188 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:09.188 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.188 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.188 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.188 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20247 2024-09-18T00:20:09.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.188 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.188 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:20:09.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:20:09.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.189 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.189 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:20:09.189 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1899999999999998e-07 2024-09-18T00:20:09.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155760+0000", 2024-09-18T00:20:09.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6459999999999999e-06 2024-09-18T00:20:09.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155767+0000", 2024-09-18T00:20:09.190 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0350000000000004e-06 2024-09-18T00:20:09.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.190 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.190 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.190 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155774+0000", 2024-09-18T00:20:09.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:20:09.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165960+0000", 2024-09-18T00:20:09.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186051999999999 2024-09-18T00:20:09.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.191 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.191 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.191 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166048+0000", 2024-09-18T00:20:09.191 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8848e-05 2024-09-18T00:20:09.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.192 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.192 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166200+0000", 2024-09-18T00:20:09.192 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015147000000000001 2024-09-18T00:20:09.192 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.192 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.192 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.192 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.192 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.192 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20248 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:09.192 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:20:09.193 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.994671398999998, 2024-09-18T00:20:09.193 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010617721, 2024-09-18T00:20:09.193 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.193 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.193 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:09.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.193 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.193 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20248 2024-09-18T00:20:09.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.193 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.193 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.193 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.193 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:20:09.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:20:09.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.194 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155782+0000", 2024-09-18T00:20:09.194 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.84e-07 2024-09-18T00:20:09.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.194 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.194 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155784+0000", 2024-09-18T00:20:09.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2800000000000002e-06 2024-09-18T00:20:09.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155790+0000", 2024-09-18T00:20:09.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5679999999999996e-06 2024-09-18T00:20:09.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.195 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.195 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155795+0000", 2024-09-18T00:20:09.195 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6929999999999998e-06 2024-09-18T00:20:09.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.195 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166245+0000", 2024-09-18T00:20:09.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010449847999999999 2024-09-18T00:20:09.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166307+0000", 2024-09-18T00:20:09.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2287999999999997e-05 2024-09-18T00:20:09.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.196 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.196 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166399+0000", 2024-09-18T00:20:09.196 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.166e-05 2024-09-18T00:20:09.196 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.197 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.197 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.197 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.197 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20249 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:09.197 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:20:09.197 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.994651023000003, 2024-09-18T00:20:09.197 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010811983000000001, 2024-09-18T00:20:09.197 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.197 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.197 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:09.197 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.197 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.197 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.198 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20249 2024-09-18T00:20:09.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.198 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:20:09.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.198 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.198 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:20:09.198 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.198 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155802+0000", 2024-09-18T00:20:09.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:20:09.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155804+0000", 2024-09-18T00:20:09.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:20:09.199 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.199 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.199 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.199 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155810+0000", 2024-09-18T00:20:09.199 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0580000000000003e-06 2024-09-18T00:20:09.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155817+0000", 2024-09-18T00:20:09.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3980000000000004e-06 2024-09-18T00:20:09.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.200 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.200 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.200 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166451+0000", 2024-09-18T00:20:09.200 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010634825000000001 2024-09-18T00:20:09.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166520+0000", 2024-09-18T00:20:09.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8317000000000003e-05 2024-09-18T00:20:09.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.201 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.201 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166613+0000", 2024-09-18T00:20:09.201 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3801000000000005e-05 2024-09-18T00:20:09.201 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.201 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.201 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.202 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.202 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:21913 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:20:09.202 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:20:09.202 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 71.856805876999999, 2024-09-18T00:20:09.202 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090508916999999994, 2024-09-18T00:20:09.202 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.202 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.202 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.202 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.202 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.202 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.202 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 21913 2024-09-18T00:20:09.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.203 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:20:09.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.203 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:20:09.203 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.203 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.203 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293648+0000", 2024-09-18T00:20:09.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6400000000000001e-07 2024-09-18T00:20:09.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293653+0000", 2024-09-18T00:20:09.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3820000000000003e-06 2024-09-18T00:20:09.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.204 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.204 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.204 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293665+0000", 2024-09-18T00:20:09.204 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1778e-05 2024-09-18T00:20:09.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293673+0000", 2024-09-18T00:20:09.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4130000000000006e-06 2024-09-18T00:20:09.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.205 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.205 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293719+0000", 2024-09-18T00:20:09.205 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5534999999999997e-05 2024-09-18T00:20:09.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.205 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.206 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.208 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293773+0000", 2024-09-18T00:20:09.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4389999999999999e-05 2024-09-18T00:20:09.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384053+0000", 2024-09-18T00:20:09.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090279617000000006 2024-09-18T00:20:09.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384088+0000", 2024-09-18T00:20:09.209 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5466000000000002e-05 2024-09-18T00:20:09.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.209 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.209 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.209 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384156+0000", 2024-09-18T00:20:09.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7471999999999994e-05 2024-09-18T00:20:09.210 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.210 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.210 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.210 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.210 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22274 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:20:09.210 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:20:09.210 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 71.045689656999997, 2024-09-18T00:20:09.210 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.069820742000000005, 2024-09-18T00:20:09.210 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.210 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.210 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.211 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.211 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.211 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.211 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22274 2024-09-18T00:20:09.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.211 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.211 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:20:09.211 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.211 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.211 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:20:09.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.212 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.212 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104766+0000", 2024-09-18T00:20:09.212 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3859999999999999e-06 2024-09-18T00:20:09.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104783+0000", 2024-09-18T00:20:09.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6697e-05 2024-09-18T00:20:09.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.213 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.213 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.213 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104816+0000", 2024-09-18T00:20:09.213 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2786000000000001e-05 2024-09-18T00:20:09.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104833+0000", 2024-09-18T00:20:09.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7125999999999998e-05 2024-09-18T00:20:09.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104965+0000", 2024-09-18T00:20:09.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013215899999999999 2024-09-18T00:20:09.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.214 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.214 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.214 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.105103+0000", 2024-09-18T00:20:09.214 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138193 2024-09-18T00:20:09.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174496+0000", 2024-09-18T00:20:09.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.06939244 2024-09-18T00:20:09.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.215 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.215 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.215 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174530+0000", 2024-09-18T00:20:09.215 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4601999999999998e-05 2024-09-18T00:20:09.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.216 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.216 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.216 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174584+0000", 2024-09-18T00:20:09.216 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3353000000000002e-05 2024-09-18T00:20:09.216 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.216 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.216 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.217 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.217 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22485 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write 16521958~30847] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:20:09.217 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:20:09.217 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 70.196900647000007, 2024-09-18T00:20:09.217 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012574648000000001, 2024-09-18T00:20:09.217 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.217 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.217 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.217 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.217 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.218 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.218 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22485 2024-09-18T00:20:09.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.218 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:20:09.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.218 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.218 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.218 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:20:09.218 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953554+0000", 2024-09-18T00:20:09.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1150000000000001e-06 2024-09-18T00:20:09.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962036+0000", 2024-09-18T00:20:09.219 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0084817599999999996 2024-09-18T00:20:09.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.219 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.219 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.219 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962062+0000", 2024-09-18T00:20:09.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6499000000000002e-05 2024-09-18T00:20:09.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962074+0000", 2024-09-18T00:20:09.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2095e-05 2024-09-18T00:20:09.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.220 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.220 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962165+0000", 2024-09-18T00:20:09.220 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0703000000000002e-05 2024-09-18T00:20:09.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.220 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962276+0000", 2024-09-18T00:20:09.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111406 2024-09-18T00:20:09.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965925+0000", 2024-09-18T00:20:09.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036488919999999999 2024-09-18T00:20:09.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.221 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.221 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.221 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965992+0000", 2024-09-18T00:20:09.221 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6806999999999999e-05 2024-09-18T00:20:09.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.222 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.222 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.966127+0000", 2024-09-18T00:20:09.222 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013437099999999999 2024-09-18T00:20:09.222 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.222 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.222 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.222 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.222 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22681 3.4 3:22e76021:::rbd_data.112a948d5956.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:20:09.222 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:20:09.222 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 69.862901342000001, 2024-09-18T00:20:09.223 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076591189000000004, 2024-09-18T00:20:09.223 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.223 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.223 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.223 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.223 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.223 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.223 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22681 2024-09-18T00:20:09.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.223 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.223 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.223 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.223 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:20:09.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:20:09.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.224 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287554+0000", 2024-09-18T00:20:09.224 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.514e-06 2024-09-18T00:20:09.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.224 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.224 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287567+0000", 2024-09-18T00:20:09.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-18T00:20:09.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287597+0000", 2024-09-18T00:20:09.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0406e-05 2024-09-18T00:20:09.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.225 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.225 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.225 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287626+0000", 2024-09-18T00:20:09.225 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9306e-05 2024-09-18T00:20:09.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287801+0000", 2024-09-18T00:20:09.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017411400000000001 2024-09-18T00:20:09.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.288053+0000", 2024-09-18T00:20:09.226 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025223500000000001 2024-09-18T00:20:09.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.226 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.226 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.226 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364016+0000", 2024-09-18T00:20:09.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075963301999999996 2024-09-18T00:20:09.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364064+0000", 2024-09-18T00:20:09.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7886000000000002e-05 2024-09-18T00:20:09.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.227 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.227 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.227 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364142+0000", 2024-09-18T00:20:09.227 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8345999999999998e-05 2024-09-18T00:20:09.227 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.227 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.227 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.228 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.228 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:24700 3.4 3:23a8f5d4:::rbd_data.112a990280ef.0000000000000016:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:20:09.228 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:20:09.228 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 62.212435362999997, 2024-09-18T00:20:09.228 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091975649000000007, 2024-09-18T00:20:09.228 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.228 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.228 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.228 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.228 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.228 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.228 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 24700 2024-09-18T00:20:09.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.229 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:20:09.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.229 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:20:09.229 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.229 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.229 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938019+0000", 2024-09-18T00:20:09.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:20:09.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938030+0000", 2024-09-18T00:20:09.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0278000000000001e-05 2024-09-18T00:20:09.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.230 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.230 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938053+0000", 2024-09-18T00:20:09.230 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2946999999999999e-05 2024-09-18T00:20:09.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.230 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938062+0000", 2024-09-18T00:20:09.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8509999999999994e-06 2024-09-18T00:20:09.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938140+0000", 2024-09-18T00:20:09.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7664999999999996e-05 2024-09-18T00:20:09.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.231 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.231 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938314+0000", 2024-09-18T00:20:09.231 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017398500000000001 2024-09-18T00:20:09.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029896+0000", 2024-09-18T00:20:09.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091582078999999997 2024-09-18T00:20:09.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029930+0000", 2024-09-18T00:20:09.232 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3723999999999998e-05 2024-09-18T00:20:09.232 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.232 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.232 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.232 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029993+0000", 2024-09-18T00:20:09.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2901e-05 2024-09-18T00:20:09.233 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.233 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.233 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.233 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.233 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37799 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:09.233 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:20:09.233 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 48.740207212000001, 2024-09-18T00:20:09.233 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186016000000001, 2024-09-18T00:20:09.233 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.233 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.233 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:09.234 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.234 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.234 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.234 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37799 2024-09-18T00:20:09.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.234 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.234 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:20:09.234 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.234 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.234 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:20:09.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410246+0000", 2024-09-18T00:20:09.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.35e-07 2024-09-18T00:20:09.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.235 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.235 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410247+0000", 2024-09-18T00:20:09.235 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0389999999999999e-06 2024-09-18T00:20:09.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.235 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410250+0000", 2024-09-18T00:20:09.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3380000000000001e-06 2024-09-18T00:20:09.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410252+0000", 2024-09-18T00:20:09.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5150000000000001e-06 2024-09-18T00:20:09.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.236 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.236 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420198+0000", 2024-09-18T00:20:09.236 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099454360000000002 2024-09-18T00:20:09.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420284+0000", 2024-09-18T00:20:09.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5638999999999998e-05 2024-09-18T00:20:09.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.237 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.237 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420431+0000", 2024-09-18T00:20:09.237 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014781399999999999 2024-09-18T00:20:09.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.237 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.237 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.237 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.238 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37800 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:09.238 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:20:09.238 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 48.740196935999997, 2024-09-18T00:20:09.238 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010452559, 2024-09-18T00:20:09.238 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.238 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.238 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:09.238 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.238 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.238 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.238 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37800 2024-09-18T00:20:09.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.238 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.238 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:20:09.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:20:09.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.239 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.239 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.239 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:20:09.239 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2499999999999999e-07 2024-09-18T00:20:09.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410257+0000", 2024-09-18T00:20:09.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0750000000000001e-06 2024-09-18T00:20:09.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410260+0000", 2024-09-18T00:20:09.240 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.428e-06 2024-09-18T00:20:09.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.240 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.240 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.240 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410263+0000", 2024-09-18T00:20:09.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5339999999999998e-06 2024-09-18T00:20:09.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420467+0000", 2024-09-18T00:20:09.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010204615 2024-09-18T00:20:09.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.241 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.241 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420557+0000", 2024-09-18T00:20:09.241 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9183000000000003e-05 2024-09-18T00:20:09.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.241 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.242 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.242 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420708+0000", 2024-09-18T00:20:09.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015149900000000001 2024-09-18T00:20:09.242 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.242 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.242 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.242 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.242 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37801 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:09.242 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:20:09.242 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 48.740186598999998, 2024-09-18T00:20:09.242 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010623357, 2024-09-18T00:20:09.243 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.243 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.243 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:09.243 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.243 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.243 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.243 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37801 2024-09-18T00:20:09.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.243 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.243 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.243 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.243 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:20:09.243 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:20:09.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:20:09.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.23e-07 2024-09-18T00:20:09.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.244 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.244 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.244 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410271+0000", 2024-09-18T00:20:09.244 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8040000000000001e-06 2024-09-18T00:20:09.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410274+0000", 2024-09-18T00:20:09.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5310000000000002e-06 2024-09-18T00:20:09.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410277+0000", 2024-09-18T00:20:09.245 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.605e-06 2024-09-18T00:20:09.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.245 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.245 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420743+0000", 2024-09-18T00:20:09.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010465487000000001 2024-09-18T00:20:09.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420811+0000", 2024-09-18T00:20:09.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8845999999999998e-05 2024-09-18T00:20:09.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.246 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.246 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.246 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420889+0000", 2024-09-18T00:20:09.246 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7861000000000006e-05 2024-09-18T00:20:09.246 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.246 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.246 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.247 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.247 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37802 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:09.247 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:20:09.247 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 48.740171973000002, 2024-09-18T00:20:09.247 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010745463, 2024-09-18T00:20:09.247 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.247 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.247 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:09.247 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.247 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.247 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.247 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37802 2024-09-18T00:20:09.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.248 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:20:09.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.248 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.248 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:20:09.248 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.248 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:20:09.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6199999999999999e-07 2024-09-18T00:20:09.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410282+0000", 2024-09-18T00:20:09.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.085e-06 2024-09-18T00:20:09.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.249 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.249 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.249 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410286+0000", 2024-09-18T00:20:09.249 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6610000000000002e-06 2024-09-18T00:20:09.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410288+0000", 2024-09-18T00:20:09.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6410000000000002e-06 2024-09-18T00:20:09.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.250 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420910+0000", 2024-09-18T00:20:09.250 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010621479 2024-09-18T00:20:09.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.250 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.250 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420952+0000", 2024-09-18T00:20:09.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2497999999999999e-05 2024-09-18T00:20:09.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.251 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.251 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.421026+0000", 2024-09-18T00:20:09.251 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3837000000000005e-05 2024-09-18T00:20:09.251 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.251 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.251 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.251 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.251 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:59273 3.a 3:535385f0:::rbd_children:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-18T00:20:09.251 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:43.695268+0000", 2024-09-18T00:20:09.252 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 25.455184359, 2024-09-18T00:20:09.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.89624285599999998, 2024-09-18T00:20:09.252 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.252 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.252 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.252 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.252 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 59273 2024-09-18T00:20:09.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.252 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.252 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.252 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695268+0000", 2024-09-18T00:20:09.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695268+0000", 2024-09-18T00:20:09.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695270+0000", 2024-09-18T00:20:09.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1440000000000001e-06 2024-09-18T00:20:09.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695278+0000", 2024-09-18T00:20:09.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8839999999999994e-06 2024-09-18T00:20:09.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695296+0000", 2024-09-18T00:20:09.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7957000000000001e-05 2024-09-18T00:20:09.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695304+0000", 2024-09-18T00:20:09.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:20:09.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695372+0000", 2024-09-18T00:20:09.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8659999999999997e-05 2024-09-18T00:20:09.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-18T00:20:09.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695383+0000", 2024-09-18T00:20:09.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0852e-05 2024-09-18T00:20:09.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.586073+0000", 2024-09-18T00:20:09.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.89068997800000005 2024-09-18T00:20:09.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.586481+0000", 2024-09-18T00:20:09.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00040750900000000002 2024-09-18T00:20:09.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-18T00:20:09.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.586873+0000", 2024-09-18T00:20:09.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00039179999999999998 2024-09-18T00:20:09.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.588815+0000", 2024-09-18T00:20:09.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019426249999999999 2024-09-18T00:20:09.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:09.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.591259+0000", 2024-09-18T00:20:09.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0024436829999999999 2024-09-18T00:20:09.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.591399+0000", 2024-09-18T00:20:09.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139927 2024-09-18T00:20:09.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.258 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.258 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.591511+0000", 2024-09-18T00:20:09.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011215200000000001 2024-09-18T00:20:09.258 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.258 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.258 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.258 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:59788 3.9 e106/23 3:94325cfd:::rbd_header.112aa373d2ec:head v 106'325, mlcod=105'323)", 2024-09-18T00:20:09.258 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:48.641886+0000", 2024-09-18T00:20:09.258 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 20.508566514000002, 2024-09-18T00:20:09.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010208398, 2024-09-18T00:20:09.259 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.259 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.259 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.259 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641886+0000", 2024-09-18T00:20:09.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641886+0000", 2024-09-18T00:20:09.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641887+0000", 2024-09-18T00:20:09.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0999999999999997e-07 2024-09-18T00:20:09.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641924+0000", 2024-09-18T00:20:09.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7252000000000002e-05 2024-09-18T00:20:09.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641955+0000", 2024-09-18T00:20:09.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0372e-05 2024-09-18T00:20:09.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641974+0000", 2024-09-18T00:20:09.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9695999999999999e-05 2024-09-18T00:20:09.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.642107+0000", 2024-09-18T00:20:09.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013297999999999999 2024-09-18T00:20:09.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.642142+0000", 2024-09-18T00:20:09.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4993000000000002e-05 2024-09-18T00:20:09.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.652006+0000", 2024-09-18T00:20:09.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0098642109999999995 2024-09-18T00:20:09.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.652094+0000", 2024-09-18T00:20:09.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7984000000000003e-05 2024-09-18T00:20:09.262 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.263 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.263 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.263 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.263 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.263 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:09.264 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:20:09.265 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:09.260+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:09.366 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:09.365+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:09.429 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:09.429 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:09.429 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:20:09.429 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:09.443 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:20:09.467 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:09.465+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:09.509 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:20:09.568 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:09.565+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:09.574 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:09.574 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:09.574 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:20:09.575 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:20:09.575 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:09.592 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:20:09.669 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:09.668+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:09.715 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:09.715 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:20:09.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:20:09.716 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:20:09.716 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.716 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:20:09.716 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:20:09.716 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 116.913245355, 2024-09-18T00:20:09.716 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:20:09.716 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.716 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.716 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.716 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.717 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.717 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.717 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:20:09.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.717 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.717 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:20:09.717 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.717 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.717 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:20:09.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:20:09.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:20:09.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.718 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.718 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:20:09.718 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:20:09.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:20:09.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:20:09.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.719 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.719 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:20:09.719 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:20:09.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.719 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:20:09.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:20:09.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.720 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:20:09.720 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:20:09.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.720 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.720 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:09.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:20:09.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:20:09.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:20:09.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:20:09.721 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.721 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.721 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:09.721 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:20:09.721 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:20:09.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:20:09.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:20:09.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.722 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.722 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.722 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:20:09.722 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:20:09.722 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.722 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.723 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:20:09.723 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:20:09.723 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 116.897037482, 2024-09-18T00:20:09.723 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:20:09.723 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.724 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.724 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:09.724 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:20:09.724 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.724 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.724 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.724 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:20:09.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:20:09.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:20:09.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.725 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.725 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.725 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:20:09.725 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:20:09.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:20:09.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:20:09.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.726 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:20:09.726 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:20:09.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.726 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.726 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:20:09.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:20:09.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:20:09.727 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:20:09.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.727 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.727 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.727 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:20:09.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:20:09.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.728 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.728 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.728 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.728 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:20:09.728 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:20:09.728 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 103.86999616200001, 2024-09-18T00:20:09.728 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:20:09.728 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.728 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.729 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.729 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.729 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:20:09.729 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.729 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.729 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.729 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.729 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:20:09.729 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:20:09.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.730 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.730 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:20:09.730 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:20:09.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:20:09.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:20:09.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.731 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:20:09.731 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:20:09.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.731 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.731 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:20:09.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:20:09.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:20:09.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:20:09.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.732 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.732 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.732 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:20:09.732 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:20:09.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:09.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:20:09.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:20:09.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.733 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.733 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:20:09.733 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:20:09.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.733 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:09.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:20:09.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:20:09.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.734 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:20:09.734 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:20:09.734 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.734 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.734 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.735 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:20:09.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:20:09.735 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.735 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.735 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.735 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:20:09.735 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:20:09.735 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 103.84871961, 2024-09-18T00:20:09.735 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:20:09.735 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.736 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.736 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:09.736 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:20:09.736 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.736 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.736 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.736 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:20:09.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:20:09.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:20:09.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.737 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.737 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.737 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:20:09.737 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:20:09.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:20:09.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:20:09.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.738 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.738 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:20:09.738 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:20:09.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.738 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:20:09.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:20:09.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.739 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.739 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:20:09.739 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:20:09.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.740 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.740 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:20:09.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:20:09.740 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.740 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.740 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.740 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.740 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11460 3.19 3:9e578b6c:::rbd_data.112ab1fa102a.0000000000000153:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:20:09.740 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:20:09.740 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 94.446348158000006, 2024-09-18T00:20:09.740 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041989011999999999, 2024-09-18T00:20:09.741 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.741 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.741 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.741 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.741 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.741 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.741 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11460 2024-09-18T00:20:09.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.741 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.741 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.741 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.741 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:20:09.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:20:09.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.742 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.742 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.742 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267067+0000", 2024-09-18T00:20:09.742 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:20:09.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267088+0000", 2024-09-18T00:20:09.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0908999999999999e-05 2024-09-18T00:20:09.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.743 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267133+0000", 2024-09-18T00:20:09.743 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5232000000000003e-05 2024-09-18T00:20:09.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.743 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.743 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267157+0000", 2024-09-18T00:20:09.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3833e-05 2024-09-18T00:20:09.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267300+0000", 2024-09-18T00:20:09.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142282 2024-09-18T00:20:09.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.744 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.744 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.744 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267447+0000", 2024-09-18T00:20:09.744 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147851 2024-09-18T00:20:09.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308889+0000", 2024-09-18T00:20:09.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041441779999999998 2024-09-18T00:20:09.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.745 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.745 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308956+0000", 2024-09-18T00:20:09.745 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7136000000000002e-05 2024-09-18T00:20:09.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.745 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.746 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.746 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.309053+0000", 2024-09-18T00:20:09.746 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6552999999999995e-05 2024-09-18T00:20:09.746 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.746 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.746 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.746 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:16947 3.18 3:18b58c7b:::rbd_data.112a6efb7564.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:20:09.746 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:20:09.746 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 81.096570456999999, 2024-09-18T00:20:09.747 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.101692484, 2024-09-18T00:20:09.747 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.747 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.747 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.747 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.747 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.747 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.747 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 16947 2024-09-18T00:20:09.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.747 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.747 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.747 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:20:09.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:20:09.748 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.748 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.748 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616843+0000", 2024-09-18T00:20:09.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0780000000000001e-06 2024-09-18T00:20:09.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617785+0000", 2024-09-18T00:20:09.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094204100000000002 2024-09-18T00:20:09.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.749 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.749 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.749 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617801+0000", 2024-09-18T00:20:09.749 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6436999999999999e-05 2024-09-18T00:20:09.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617809+0000", 2024-09-18T00:20:09.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9049999999999997e-06 2024-09-18T00:20:09.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.750 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.750 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617904+0000", 2024-09-18T00:20:09.750 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5204999999999994e-05 2024-09-18T00:20:09.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.750 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.618071+0000", 2024-09-18T00:20:09.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016640000000000001 2024-09-18T00:20:09.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.751 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.751 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:09.751 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.624534+0000", 2024-09-18T00:20:09.751 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064634499999999999 2024-09-18T00:20:09.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718299+0000", 2024-09-18T00:20:09.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093764739 2024-09-18T00:20:09.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.752 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.752 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:09.752 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718375+0000", 2024-09-18T00:20:09.752 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6532999999999994e-05 2024-09-18T00:20:09.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718466+0000", 2024-09-18T00:20:09.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0581000000000006e-05 2024-09-18T00:20:09.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.753 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.753 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.753 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718534+0000", 2024-09-18T00:20:09.753 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8114999999999994e-05 2024-09-18T00:20:09.753 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.753 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.754 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.754 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:16947 3.18 e59/23 ondisk, result = 0)", 2024-09-18T00:20:09.754 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:20:09.754 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 81.071353763000005, 2024-09-18T00:20:09.754 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076488069000000006, 2024-09-18T00:20:09.754 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.754 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.754 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:09.754 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.754 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:20:09.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:20:09.755 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.755 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.755 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.755 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642060+0000", 2024-09-18T00:20:09.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6640000000000001e-06 2024-09-18T00:20:09.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642071+0000", 2024-09-18T00:20:09.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1204e-05 2024-09-18T00:20:09.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.756 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.756 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.756 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642091+0000", 2024-09-18T00:20:09.756 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0313999999999999e-05 2024-09-18T00:20:09.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642103+0000", 2024-09-18T00:20:09.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1214e-05 2024-09-18T00:20:09.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.757 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.757 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718350+0000", 2024-09-18T00:20:09.757 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076247681999999997 2024-09-18T00:20:09.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.757 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718366+0000", 2024-09-18T00:20:09.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5420000000000001e-05 2024-09-18T00:20:09.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.758 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.758 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.758 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718546+0000", 2024-09-18T00:20:09.758 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000180571 2024-09-18T00:20:09.758 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.758 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.758 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.759 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.759 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18686 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:09.759 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:20:09.759 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 76.764676393000002, 2024-09-18T00:20:09.759 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041132573999999998, 2024-09-18T00:20:09.759 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.759 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.759 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.759 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.759 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.759 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.759 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18686 2024-09-18T00:20:09.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.760 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:20:09.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.760 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.760 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.760 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:20:09.760 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948737+0000", 2024-09-18T00:20:09.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3769999999999999e-06 2024-09-18T00:20:09.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949968+0000", 2024-09-18T00:20:09.761 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001230493 2024-09-18T00:20:09.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.761 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.761 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.761 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949993+0000", 2024-09-18T00:20:09.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5282e-05 2024-09-18T00:20:09.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950011+0000", 2024-09-18T00:20:09.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8199999999999999e-05 2024-09-18T00:20:09.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.762 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.762 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.762 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950180+0000", 2024-09-18T00:20:09.762 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016867600000000001 2024-09-18T00:20:09.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950337+0000", 2024-09-18T00:20:09.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156902 2024-09-18T00:20:09.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.763 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:09.763 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950818+0000", 2024-09-18T00:20:09.763 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048192599999999998 2024-09-18T00:20:09.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.763 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:09.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978491+0000", 2024-09-18T00:20:09.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027672568000000002 2024-09-18T00:20:09.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989759+0000", 2024-09-18T00:20:09.764 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011268144000000001 2024-09-18T00:20:09.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.764 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.764 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.764 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989824+0000", 2024-09-18T00:20:09.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4556999999999999e-05 2024-09-18T00:20:09.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.765 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.765 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989868+0000", 2024-09-18T00:20:09.765 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4449000000000001e-05 2024-09-18T00:20:09.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.765 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.765 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.765 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.766 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18687 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:09.766 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:20:09.766 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 76.763348628000003, 2024-09-18T00:20:09.766 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040034156000000001, 2024-09-18T00:20:09.766 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.766 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.766 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.766 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.766 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18687 2024-09-18T00:20:09.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.767 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:20:09.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.767 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.767 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.767 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950063+0000", 2024-09-18T00:20:09.767 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950065+0000", 2024-09-18T00:20:09.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5820000000000001e-06 2024-09-18T00:20:09.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.768 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.768 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950072+0000", 2024-09-18T00:20:09.768 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.1029999999999996e-06 2024-09-18T00:20:09.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950092+0000", 2024-09-18T00:20:09.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0288e-05 2024-09-18T00:20:09.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.769 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950107+0000", 2024-09-18T00:20:09.769 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4290999999999999e-05 2024-09-18T00:20:09.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.769 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.769 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975820+0000", 2024-09-18T00:20:09.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.025713573999999999 2024-09-18T00:20:09.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.975980+0000", 2024-09-18T00:20:09.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016002799999999999 2024-09-18T00:20:09.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.770 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.770 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:09.770 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.976299+0000", 2024-09-18T00:20:09.770 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00031853300000000002 2024-09-18T00:20:09.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:09.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989936+0000", 2024-09-18T00:20:09.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013637046999999999 2024-09-18T00:20:09.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.771 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.771 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.771 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990025+0000", 2024-09-18T00:20:09.771 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9040000000000001e-05 2024-09-18T00:20:09.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990064+0000", 2024-09-18T00:20:09.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9394000000000003e-05 2024-09-18T00:20:09.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.772 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.772 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.772 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990098+0000", 2024-09-18T00:20:09.772 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3275999999999998e-05 2024-09-18T00:20:09.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.772 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.773 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.773 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.773 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18688 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:09.773 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:20:09.773 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 76.763247276000001, 2024-09-18T00:20:09.773 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040041259000000003, 2024-09-18T00:20:09.773 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.773 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.773 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.773 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.774 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.774 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18688 2024-09-18T00:20:09.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.774 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.774 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:20:09.774 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.774 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.774 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:20:09.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950166+0000", 2024-09-18T00:20:09.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:20:09.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.775 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.775 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.775 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951295+0000", 2024-09-18T00:20:09.775 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011293480000000001 2024-09-18T00:20:09.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951328+0000", 2024-09-18T00:20:09.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2524999999999998e-05 2024-09-18T00:20:09.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.776 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.776 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951346+0000", 2024-09-18T00:20:09.776 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8170999999999999e-05 2024-09-18T00:20:09.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978546+0000", 2024-09-18T00:20:09.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027200418000000001 2024-09-18T00:20:09.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978666+0000", 2024-09-18T00:20:09.777 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119913 2024-09-18T00:20:09.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.777 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.777 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:09.777 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.979098+0000", 2024-09-18T00:20:09.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00043169399999999999 2024-09-18T00:20:09.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:09.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990000+0000", 2024-09-18T00:20:09.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010901719000000001 2024-09-18T00:20:09.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.778 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.778 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.778 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990109+0000", 2024-09-18T00:20:09.778 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010964299999999999 2024-09-18T00:20:09.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990152+0000", 2024-09-18T00:20:09.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2877000000000003e-05 2024-09-18T00:20:09.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.779 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.779 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.779 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990206+0000", 2024-09-18T00:20:09.779 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3717000000000001e-05 2024-09-18T00:20:09.779 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.779 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.780 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.780 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.780 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18689 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:09.780 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:20:09.780 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 76.762035577999995, 2024-09-18T00:20:09.780 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051022338, 2024-09-18T00:20:09.780 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.780 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.780 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.780 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.781 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.781 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.781 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18689 2024-09-18T00:20:09.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.781 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.781 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.781 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:20:09.781 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.781 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:20:09.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.782 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951378+0000", 2024-09-18T00:20:09.782 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:20:09.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.782 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.782 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951385+0000", 2024-09-18T00:20:09.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-18T00:20:09.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951402+0000", 2024-09-18T00:20:09.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6538999999999999e-05 2024-09-18T00:20:09.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.783 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.783 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.783 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951565+0000", 2024-09-18T00:20:09.783 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016307999999999999 2024-09-18T00:20:09.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988232+0000", 2024-09-18T00:20:09.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036667656999999999 2024-09-18T00:20:09.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.784 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.784 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988318+0000", 2024-09-18T00:20:09.784 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5348999999999999e-05 2024-09-18T00:20:09.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:09.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988574+0000", 2024-09-18T00:20:09.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025670800000000002 2024-09-18T00:20:09.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:09.785 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002254+0000", 2024-09-18T00:20:09.785 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013679274 2024-09-18T00:20:09.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.785 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.785 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002326+0000", 2024-09-18T00:20:09.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2720999999999997e-05 2024-09-18T00:20:09.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002364+0000", 2024-09-18T00:20:09.786 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7314000000000002e-05 2024-09-18T00:20:09.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.786 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.786 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.786 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002399+0000", 2024-09-18T00:20:09.787 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5162e-05 2024-09-18T00:20:09.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.787 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.787 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.787 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.787 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.787 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18690 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:09.788 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:20:09.788 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 76.761818559999995, 2024-09-18T00:20:09.788 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050872134999999999, 2024-09-18T00:20:09.788 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.788 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.788 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.788 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.788 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.788 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.788 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18690 2024-09-18T00:20:09.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.789 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:20:09.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.789 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.789 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:20:09.789 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951595+0000", 2024-09-18T00:20:09.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4479999999999999e-06 2024-09-18T00:20:09.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.790 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952167+0000", 2024-09-18T00:20:09.790 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00057188400000000004 2024-09-18T00:20:09.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.790 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.790 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952192+0000", 2024-09-18T00:20:09.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4936999999999999e-05 2024-09-18T00:20:09.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952206+0000", 2024-09-18T00:20:09.791 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4565e-05 2024-09-18T00:20:09.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.791 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.791 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.791 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990303+0000", 2024-09-18T00:20:09.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038096954000000002 2024-09-18T00:20:09.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990403+0000", 2024-09-18T00:20:09.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9486000000000003e-05 2024-09-18T00:20:09.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.792 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.792 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:09.792 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990773+0000", 2024-09-18T00:20:09.792 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037037799999999998 2024-09-18T00:20:09.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:09.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002316+0000", 2024-09-18T00:20:09.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011542581 2024-09-18T00:20:09.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.793 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.793 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.793 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002407+0000", 2024-09-18T00:20:09.793 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0968e-05 2024-09-18T00:20:09.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002442+0000", 2024-09-18T00:20:09.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4962000000000002e-05 2024-09-18T00:20:09.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.794 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.794 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002466+0000", 2024-09-18T00:20:09.794 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3972e-05 2024-09-18T00:20:09.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.794 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.795 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.795 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.795 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18691 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:09.795 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:20:09.795 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 76.761185682999994, 2024-09-18T00:20:09.795 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051495530999999997, 2024-09-18T00:20:09.795 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.795 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.795 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.795 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.795 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.795 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.796 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18691 2024-09-18T00:20:09.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.796 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:20:09.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.796 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.796 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.796 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:20:09.796 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952228+0000", 2024-09-18T00:20:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:20:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952234+0000", 2024-09-18T00:20:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0310000000000004e-06 2024-09-18T00:20:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.797 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952248+0000", 2024-09-18T00:20:09.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4577e-05 2024-09-18T00:20:09.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952260+0000", 2024-09-18T00:20:09.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1659e-05 2024-09-18T00:20:09.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.798 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.798 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.798 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.000989+0000", 2024-09-18T00:20:09.798 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048729445000000003 2024-09-18T00:20:09.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001076+0000", 2024-09-18T00:20:09.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6323999999999995e-05 2024-09-18T00:20:09.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.799 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:09.799 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001295+0000", 2024-09-18T00:20:09.799 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021944299999999999 2024-09-18T00:20:09.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.799 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003502+0000", 2024-09-18T00:20:09.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022065209999999999 2024-09-18T00:20:09.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:09.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003581+0000", 2024-09-18T00:20:09.800 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9404000000000001e-05 2024-09-18T00:20:09.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.800 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.800 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.800 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003652+0000", 2024-09-18T00:20:09.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0506999999999994e-05 2024-09-18T00:20:09.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.801 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.801 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003722+0000", 2024-09-18T00:20:09.801 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0300000000000001e-05 2024-09-18T00:20:09.801 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.801 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.801 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.801 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.802 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18692 3.7 3:e69124d0:::rbd_data.112a6efb7564.0000000000000001:f [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:09.802 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:20:09.802 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 76.761137622000007, 2024-09-18T00:20:09.802 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050418114999999999, 2024-09-18T00:20:09.802 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.802 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.802 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:09.802 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.802 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.802 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.802 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18692 2024-09-18T00:20:09.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.803 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:20:09.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.803 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.803 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.803 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:20:09.803 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952275+0000", 2024-09-18T00:20:09.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4e-07 2024-09-18T00:20:09.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.804 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952281+0000", 2024-09-18T00:20:09.804 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2000000000000002e-06 2024-09-18T00:20:09.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.804 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.804 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952295+0000", 2024-09-18T00:20:09.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4731e-05 2024-09-18T00:20:09.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952307+0000", 2024-09-18T00:20:09.805 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1877e-05 2024-09-18T00:20:09.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.805 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.805 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.805 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002506+0000", 2024-09-18T00:20:09.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050199230999999997 2024-09-18T00:20:09.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002574+0000", 2024-09-18T00:20:09.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8076000000000003e-05 2024-09-18T00:20:09.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.806 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.806 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.806 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002693+0000", 2024-09-18T00:20:09.806 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011806 2024-09-18T00:20:09.806 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.807 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.807 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.807 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.807 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19383 3.5 3:a1859bb0:::rbd_data.112a7fe29ab4.000000000000015b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:09.807 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:20:09.807 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 76.091126024999994, 2024-09-18T00:20:09.807 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056760192000000001, 2024-09-18T00:20:09.807 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.807 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.807 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.808 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.808 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.808 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.808 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19383 2024-09-18T00:20:09.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.808 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.808 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.808 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.808 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:20:09.808 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:20:09.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.809 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622287+0000", 2024-09-18T00:20:09.809 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-18T00:20:09.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.809 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622292+0000", 2024-09-18T00:20:09.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6110000000000004e-06 2024-09-18T00:20:09.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622304+0000", 2024-09-18T00:20:09.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2106000000000001e-05 2024-09-18T00:20:09.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.810 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.810 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.810 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622309+0000", 2024-09-18T00:20:09.810 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.818e-06 2024-09-18T00:20:09.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622360+0000", 2024-09-18T00:20:09.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0430999999999999e-05 2024-09-18T00:20:09.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.811 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.811 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.811 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622438+0000", 2024-09-18T00:20:09.811 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8362999999999995e-05 2024-09-18T00:20:09.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678811+0000", 2024-09-18T00:20:09.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056372894 2024-09-18T00:20:09.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.812 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678906+0000", 2024-09-18T00:20:09.812 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5078e-05 2024-09-18T00:20:09.812 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.812 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.812 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.813 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.679046+0000", 2024-09-18T00:20:09.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139987 2024-09-18T00:20:09.813 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.813 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.813 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.813 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.813 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22468 3.5 3:a250f1ef:::rbd_data.112a948d5956.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:20:09.813 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:20:09.813 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 70.942044619000001, 2024-09-18T00:20:09.813 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.055599787999999997, 2024-09-18T00:20:09.813 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.813 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.814 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.814 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.814 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.814 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.814 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22468 2024-09-18T00:20:09.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.814 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.814 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.814 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.814 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:20:09.814 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:20:09.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.815 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771370+0000", 2024-09-18T00:20:09.815 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9629999999999998e-06 2024-09-18T00:20:09.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.815 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.815 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771385+0000", 2024-09-18T00:20:09.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5053e-05 2024-09-18T00:20:09.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771429+0000", 2024-09-18T00:20:09.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3192999999999998e-05 2024-09-18T00:20:09.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.816 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.816 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771453+0000", 2024-09-18T00:20:09.816 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3995999999999999e-05 2024-09-18T00:20:09.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771623+0000", 2024-09-18T00:20:09.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000170846 2024-09-18T00:20:09.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.817 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.817 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.817 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771838+0000", 2024-09-18T00:20:09.817 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021416600000000001 2024-09-18T00:20:09.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826866+0000", 2024-09-18T00:20:09.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.05502866 2024-09-18T00:20:09.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.818 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826906+0000", 2024-09-18T00:20:09.818 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9805999999999998e-05 2024-09-18T00:20:09.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.818 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.818 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.819 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826967+0000", 2024-09-18T00:20:09.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1105000000000005e-05 2024-09-18T00:20:09.819 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.819 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.819 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.819 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.819 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22470 3.2 3:474595b6:::rbd_data.112a948d5956.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:20:09.819 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:20:09.819 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 70.885853635000004, 2024-09-18T00:20:09.819 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088321282000000001, 2024-09-18T00:20:09.819 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.820 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.820 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.820 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.820 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.820 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.820 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22470 2024-09-18T00:20:09.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.820 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.820 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.820 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.820 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:20:09.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:20:09.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.821 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.821 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827560+0000", 2024-09-18T00:20:09.821 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.762e-06 2024-09-18T00:20:09.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827569+0000", 2024-09-18T00:20:09.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8740000000000001e-06 2024-09-18T00:20:09.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.822 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827585+0000", 2024-09-18T00:20:09.822 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5962000000000001e-05 2024-09-18T00:20:09.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.822 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.822 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827595+0000", 2024-09-18T00:20:09.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5910000000000002e-06 2024-09-18T00:20:09.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827811+0000", 2024-09-18T00:20:09.823 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000216858 2024-09-18T00:20:09.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.823 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.823 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:20:09.823 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827818+0000", 2024-09-18T00:20:09.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.99e-06 2024-09-18T00:20:09.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829799+0000", 2024-09-18T00:20:09.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019803849999999999 2024-09-18T00:20:09.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.824 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.824 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.824 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829884+0000", 2024-09-18T00:20:09.824 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4852999999999995e-05 2024-09-18T00:20:09.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915616+0000", 2024-09-18T00:20:09.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085732808999999993 2024-09-18T00:20:09.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.825 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.825 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915711+0000", 2024-09-18T00:20:09.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4578000000000001e-05 2024-09-18T00:20:09.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915880+0000", 2024-09-18T00:20:09.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016861999999999999 2024-09-18T00:20:09.826 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.826 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.826 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.826 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.826 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:54153 3.b 3:d3f7b2f3:::rbd_data.112a859a01c9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,zero 4596947~21151] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:20:09.826 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:20:09.826 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.860640899000003, 2024-09-18T00:20:09.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062612886000000006, 2024-09-18T00:20:09.827 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.827 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.827 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.827 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.827 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.827 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.827 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 54153 2024-09-18T00:20:09.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.827 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:20:09.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:20:09.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852773+0000", 2024-09-18T00:20:09.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.426e-06 2024-09-18T00:20:09.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855337+0000", 2024-09-18T00:20:09.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025647830000000002 2024-09-18T00:20:09.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855369+0000", 2024-09-18T00:20:09.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1300999999999997e-05 2024-09-18T00:20:09.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855387+0000", 2024-09-18T00:20:09.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7949999999999999e-05 2024-09-18T00:20:09.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855487+0000", 2024-09-18T00:20:09.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010062 2024-09-18T00:20:09.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855686+0000", 2024-09-18T00:20:09.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019901399999999999 2024-09-18T00:20:09.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:09.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858037+0000", 2024-09-18T00:20:09.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002351264 2024-09-18T00:20:09.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.896302+0000", 2024-09-18T00:20:09.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038264864000000003 2024-09-18T00:20:09.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:09.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915250+0000", 2024-09-18T00:20:09.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018947538999999999 2024-09-18T00:20:09.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915318+0000", 2024-09-18T00:20:09.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7838999999999999e-05 2024-09-18T00:20:09.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915384+0000", 2024-09-18T00:20:09.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6285999999999995e-05 2024-09-18T00:20:09.833 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.833 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.833 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.833 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:55488 3.18 3:1d5573d8:::rbd_data.112a859a01c9.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T00:20:09.833 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:20:09.833 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 38.127810689999997, 2024-09-18T00:20:09.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045116205999999999, 2024-09-18T00:20:09.833 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.834 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.834 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.834 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.834 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.834 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.834 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 55488 2024-09-18T00:20:09.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.834 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:20:09.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:20:09.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585602+0000", 2024-09-18T00:20:09.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4000000000000001e-07 2024-09-18T00:20:09.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585607+0000", 2024-09-18T00:20:09.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5240000000000002e-06 2024-09-18T00:20:09.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585618+0000", 2024-09-18T00:20:09.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1493e-05 2024-09-18T00:20:09.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.836 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585624+0000", 2024-09-18T00:20:09.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7180000000000001e-06 2024-09-18T00:20:09.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585672+0000", 2024-09-18T00:20:09.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8158000000000002e-05 2024-09-18T00:20:09.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585780+0000", 2024-09-18T00:20:09.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010781399999999999 2024-09-18T00:20:09.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:09.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.588202+0000", 2024-09-18T00:20:09.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002422156 2024-09-18T00:20:09.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:09.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630375+0000", 2024-09-18T00:20:09.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042172820999999999 2024-09-18T00:20:09.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630426+0000", 2024-09-18T00:20:09.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1369000000000002e-05 2024-09-18T00:20:09.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630581+0000", 2024-09-18T00:20:09.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000155041 2024-09-18T00:20:09.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630718+0000", 2024-09-18T00:20:09.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000136372 2024-09-18T00:20:09.840 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.840 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.840 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.840 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:62587 3.8 3:1527150c:::rbd_data.112a13ec2d6c.0000000000000003:head [delete] snapc b=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-18T00:20:09.840 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:07.612383+0000", 2024-09-18T00:20:09.840 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 2.1010293459999998, 2024-09-18T00:20:09.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.88651563099999997, 2024-09-18T00:20:09.840 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:09.840 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:09.840 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:09.841 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:09.841 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:09.841 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:09.841 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 62587 2024-09-18T00:20:09.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.841 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:09.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.841 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:09.841 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612383+0000", 2024-09-18T00:20:09.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:09.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612383+0000", 2024-09-18T00:20:09.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:09.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:09.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612385+0000", 2024-09-18T00:20:09.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9489999999999999e-06 2024-09-18T00:20:09.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:09.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612395+0000", 2024-09-18T00:20:09.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9340000000000001e-06 2024-09-18T00:20:09.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:09.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612417+0000", 2024-09-18T00:20:09.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2633000000000002e-05 2024-09-18T00:20:09.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:09.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612430+0000", 2024-09-18T00:20:09.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2639e-05 2024-09-18T00:20:09.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612529+0000", 2024-09-18T00:20:09.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9133000000000001e-05 2024-09-18T00:20:09.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-18T00:20:09.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612543+0000", 2024-09-18T00:20:09.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.412e-05 2024-09-18T00:20:09.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:09.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.490391+0000", 2024-09-18T00:20:09.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.87784777000000003 2024-09-18T00:20:09.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:09.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.490724+0000", 2024-09-18T00:20:09.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033358900000000002 2024-09-18T00:20:09.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-18T00:20:09.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.491039+0000", 2024-09-18T00:20:09.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000314548 2024-09-18T00:20:09.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:09.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.493889+0000", 2024-09-18T00:20:09.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028495339999999999 2024-09-18T00:20:09.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:09.846 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.498625+0000", 2024-09-18T00:20:09.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047369689999999997 2024-09-18T00:20:09.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.846 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:09.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.498794+0000", 2024-09-18T00:20:09.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000168006 2024-09-18T00:20:09.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:09.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:09.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:09.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.498898+0000", 2024-09-18T00:20:09.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104807 2024-09-18T00:20:09.847 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.847 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.847 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.847 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:09.847 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:09.848 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:09.849 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:20:09.850 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:09.844+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:09.950 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:09.949+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:09.980 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:20:09.980 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:09.994 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:20:10.086 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:20:10.085+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:20:10.152 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:10.149+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:10.160 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:10.160 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:10.160 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:20:10.161 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:20:10.161 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:10.175 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:20:10.253 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:10.248+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:10.344 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:10.344 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:20:10.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:20:10.345 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:20:10.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.345 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2418 3.1a 3:59a8b40d:::rbd_data.112aa7ba4578.00000000000001b0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:20:10.345 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:20:10.345 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.605336313, 2024-09-18T00:20:10.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10488562999999999, 2024-09-18T00:20:10.345 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.345 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.345 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:10.345 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.345 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.345 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.345 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2418 2024-09-18T00:20:10.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.346 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:20:10.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:20:10.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735592+0000", 2024-09-18T00:20:10.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5229999999999998e-06 2024-09-18T00:20:10.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735604+0000", 2024-09-18T00:20:10.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2143e-05 2024-09-18T00:20:10.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735629+0000", 2024-09-18T00:20:10.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.497e-05 2024-09-18T00:20:10.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735648+0000", 2024-09-18T00:20:10.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8896999999999999e-05 2024-09-18T00:20:10.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735775+0000", 2024-09-18T00:20:10.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126935 2024-09-18T00:20:10.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.736118+0000", 2024-09-18T00:20:10.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034330800000000001 2024-09-18T00:20:10.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-18T00:20:10.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.741633+0000", 2024-09-18T00:20:10.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055144139999999996 2024-09-18T00:20:10.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:10.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840134+0000", 2024-09-18T00:20:10.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.098501618999999999 2024-09-18T00:20:10.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:10.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840184+0000", 2024-09-18T00:20:10.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9953000000000001e-05 2024-09-18T00:20:10.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:10.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840336+0000", 2024-09-18T00:20:10.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151476 2024-09-18T00:20:10.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840475+0000", 2024-09-18T00:20:10.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013939200000000001 2024-09-18T00:20:10.350 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.350 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.351 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.351 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:22272 3.3 3:c617f123:::rbd_data.112a948d5956.0000000000000009:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:20:10.351 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:20:10.351 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 72.349828114000005, 2024-09-18T00:20:10.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10752577300000001, 2024-09-18T00:20:10.351 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.351 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.351 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:10.351 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.351 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.351 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.351 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 22272 2024-09-18T00:20:10.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.352 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:20:10.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:20:10.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991102+0000", 2024-09-18T00:20:10.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9369999999999997e-06 2024-09-18T00:20:10.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991118+0000", 2024-09-18T00:20:10.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6396e-05 2024-09-18T00:20:10.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991150+0000", 2024-09-18T00:20:10.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2132999999999999e-05 2024-09-18T00:20:10.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991170+0000", 2024-09-18T00:20:10.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0085000000000002e-05 2024-09-18T00:20:10.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991370+0000", 2024-09-18T00:20:10.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019963700000000001 2024-09-18T00:20:10.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991518+0000", 2024-09-18T00:20:10.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000148424 2024-09-18T00:20:10.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:10.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.998850+0000", 2024-09-18T00:20:10.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073315480000000002 2024-09-18T00:20:10.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:10.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098178+0000", 2024-09-18T00:20:10.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.099328067000000006 2024-09-18T00:20:10.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:10.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098314+0000", 2024-09-18T00:20:10.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000135678 2024-09-18T00:20:10.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:10.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098498+0000", 2024-09-18T00:20:10.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018479600000000001 2024-09-18T00:20:10.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098623+0000", 2024-09-18T00:20:10.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000125072 2024-09-18T00:20:10.357 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.357 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.357 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.357 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:23689 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-18T00:20:10.357 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:20:10.357 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.488503335999994, 2024-09-18T00:20:10.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9562625809999998, 2024-09-18T00:20:10.357 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.357 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.357 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:10.357 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.358 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.358 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.358 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 23689 2024-09-18T00:20:10.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.358 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:20:10.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:20:10.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852423+0000", 2024-09-18T00:20:10.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9999999999999995e-07 2024-09-18T00:20:10.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852429+0000", 2024-09-18T00:20:10.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3279999999999996e-06 2024-09-18T00:20:10.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852442+0000", 2024-09-18T00:20:10.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3052e-05 2024-09-18T00:20:10.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852448+0000", 2024-09-18T00:20:10.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2779999999999999e-06 2024-09-18T00:20:10.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852523+0000", 2024-09-18T00:20:10.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4463999999999997e-05 2024-09-18T00:20:10.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:10.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852538+0000", 2024-09-18T00:20:10.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5466999999999999e-05 2024-09-18T00:20:10.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793478+0000", 2024-09-18T00:20:10.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.94094012699999996 2024-09-18T00:20:10.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:10.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793494+0000", 2024-09-18T00:20:10.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5807999999999998e-05 2024-09-18T00:20:10.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.804875+0000", 2024-09-18T00:20:10.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0113811939999999 2024-09-18T00:20:10.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805359+0000", 2024-09-18T00:20:10.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000484248 2024-09-18T00:20:10.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-18T00:20:10.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805872+0000", 2024-09-18T00:20:10.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00051218100000000001 2024-09-18T00:20:10.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:10.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.807790+0000", 2024-09-18T00:20:10.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001918763 2024-09-18T00:20:10.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:10.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808242+0000", 2024-09-18T00:20:10.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045126200000000002 2024-09-18T00:20:10.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:10.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808428+0000", 2024-09-18T00:20:10.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018633000000000001 2024-09-18T00:20:10.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808685+0000", 2024-09-18T00:20:10.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025707900000000001 2024-09-18T00:20:10.364 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.364 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.365 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.365 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47925 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7864320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:10.365 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:20:10.365 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.308341235, 2024-09-18T00:20:10.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086484302999999998, 2024-09-18T00:20:10.365 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.365 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.365 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.365 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.365 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.365 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.366 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47925 2024-09-18T00:20:10.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.366 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:20:10.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:20:10.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:20:10.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.82e-07 2024-09-18T00:20:10.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032587+0000", 2024-09-18T00:20:10.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.058e-06 2024-09-18T00:20:10.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032593+0000", 2024-09-18T00:20:10.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1380000000000004e-06 2024-09-18T00:20:10.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032598+0000", 2024-09-18T00:20:10.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9819999999999999e-06 2024-09-18T00:20:10.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118873+0000", 2024-09-18T00:20:10.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086274609000000002 2024-09-18T00:20:10.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118950+0000", 2024-09-18T00:20:10.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6687999999999995e-05 2024-09-18T00:20:10.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119069+0000", 2024-09-18T00:20:10.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011934600000000001 2024-09-18T00:20:10.369 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.369 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.369 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.369 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47926 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7897088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:10.369 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032604+0000", 2024-09-18T00:20:10.370 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.308321358999997, 2024-09-18T00:20:10.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086712168000000006, 2024-09-18T00:20:10.370 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.370 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.370 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.370 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.370 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.370 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.370 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47926 2024-09-18T00:20:10.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.370 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032604+0000", 2024-09-18T00:20:10.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032604+0000", 2024-09-18T00:20:10.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032605+0000", 2024-09-18T00:20:10.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:20:10.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032611+0000", 2024-09-18T00:20:10.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4860000000000002e-06 2024-09-18T00:20:10.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032618+0000", 2024-09-18T00:20:10.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6699999999999997e-06 2024-09-18T00:20:10.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032623+0000", 2024-09-18T00:20:10.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0860000000000001e-06 2024-09-18T00:20:10.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119111+0000", 2024-09-18T00:20:10.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086488067000000002 2024-09-18T00:20:10.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119185+0000", 2024-09-18T00:20:10.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3758999999999996e-05 2024-09-18T00:20:10.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119317+0000", 2024-09-18T00:20:10.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013159599999999999 2024-09-18T00:20:10.374 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.374 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.374 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.374 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47927 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7929856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:10.374 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:20:10.374 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.308296018, 2024-09-18T00:20:10.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086929532000000004, 2024-09-18T00:20:10.374 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.374 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.374 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.375 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.375 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.375 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.375 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47927 2024-09-18T00:20:10.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.375 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:20:10.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:20:10.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:20:10.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-18T00:20:10.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032632+0000", 2024-09-18T00:20:10.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1320000000000001e-06 2024-09-18T00:20:10.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032639+0000", 2024-09-18T00:20:10.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1840000000000004e-06 2024-09-18T00:20:10.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032644+0000", 2024-09-18T00:20:10.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2519999999999999e-06 2024-09-18T00:20:10.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119358+0000", 2024-09-18T00:20:10.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086714075000000002 2024-09-18T00:20:10.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119438+0000", 2024-09-18T00:20:10.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9756e-05 2024-09-18T00:20:10.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119559+0000", 2024-09-18T00:20:10.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121579 2024-09-18T00:20:10.378 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.378 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.378 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.379 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47928 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7962624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:10.379 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032650+0000", 2024-09-18T00:20:10.379 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.308275594000001, 2024-09-18T00:20:10.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087152732999999996, 2024-09-18T00:20:10.379 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.379 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.379 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.379 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.379 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.379 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.379 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47928 2024-09-18T00:20:10.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.379 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032650+0000", 2024-09-18T00:20:10.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032650+0000", 2024-09-18T00:20:10.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032651+0000", 2024-09-18T00:20:10.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:20:10.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032653+0000", 2024-09-18T00:20:10.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.182e-06 2024-09-18T00:20:10.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032659+0000", 2024-09-18T00:20:10.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2319999999999998e-06 2024-09-18T00:20:10.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032664+0000", 2024-09-18T00:20:10.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0420000000000002e-06 2024-09-18T00:20:10.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119602+0000", 2024-09-18T00:20:10.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086938213 2024-09-18T00:20:10.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119676+0000", 2024-09-18T00:20:10.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3708000000000006e-05 2024-09-18T00:20:10.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119803+0000", 2024-09-18T00:20:10.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126852 2024-09-18T00:20:10.383 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.383 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.383 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.383 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47929 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7995392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:10.383 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032670+0000", 2024-09-18T00:20:10.383 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.308255336000002, 2024-09-18T00:20:10.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087373082000000005, 2024-09-18T00:20:10.383 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.383 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.384 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.384 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.384 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.384 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.384 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47929 2024-09-18T00:20:10.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.384 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032670+0000", 2024-09-18T00:20:10.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032670+0000", 2024-09-18T00:20:10.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032671+0000", 2024-09-18T00:20:10.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:20:10.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032673+0000", 2024-09-18T00:20:10.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.928e-06 2024-09-18T00:20:10.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032679+0000", 2024-09-18T00:20:10.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.02e-06 2024-09-18T00:20:10.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032684+0000", 2024-09-18T00:20:10.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1660000000000002e-06 2024-09-18T00:20:10.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119844+0000", 2024-09-18T00:20:10.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087160248999999995 2024-09-18T00:20:10.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119920+0000", 2024-09-18T00:20:10.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5181999999999999e-05 2024-09-18T00:20:10.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120044+0000", 2024-09-18T00:20:10.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012403300000000001 2024-09-18T00:20:10.387 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.387 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.387 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.388 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47930 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8028160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:10.388 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032690+0000", 2024-09-18T00:20:10.388 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.308235484000001, 2024-09-18T00:20:10.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087595941999999996, 2024-09-18T00:20:10.388 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.388 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.388 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.388 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.388 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.388 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.389 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47930 2024-09-18T00:20:10.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.389 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032690+0000", 2024-09-18T00:20:10.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032690+0000", 2024-09-18T00:20:10.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032691+0000", 2024-09-18T00:20:10.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9399999999999995e-07 2024-09-18T00:20:10.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032693+0000", 2024-09-18T00:20:10.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1299999999999999e-06 2024-09-18T00:20:10.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032699+0000", 2024-09-18T00:20:10.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1879999999999999e-06 2024-09-18T00:20:10.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032704+0000", 2024-09-18T00:20:10.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0939999999999998e-06 2024-09-18T00:20:10.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120087+0000", 2024-09-18T00:20:10.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087383049000000004 2024-09-18T00:20:10.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120164+0000", 2024-09-18T00:20:10.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6679000000000002e-05 2024-09-18T00:20:10.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.393 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120286+0000", 2024-09-18T00:20:10.393 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012230800000000001 2024-09-18T00:20:10.393 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.393 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.393 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.393 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47931 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8060928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:10.393 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:20:10.393 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.308215240999999, 2024-09-18T00:20:10.393 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087823688999999996, 2024-09-18T00:20:10.393 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.393 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.394 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.394 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.394 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.394 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.394 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47931 2024-09-18T00:20:10.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.394 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:20:10.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:20:10.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:20:10.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9200000000000001e-07 2024-09-18T00:20:10.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032713+0000", 2024-09-18T00:20:10.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1239999999999999e-06 2024-09-18T00:20:10.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032719+0000", 2024-09-18T00:20:10.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1659999999999999e-06 2024-09-18T00:20:10.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.396 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032724+0000", 2024-09-18T00:20:10.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9359999999999998e-06 2024-09-18T00:20:10.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.396 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120330+0000", 2024-09-18T00:20:10.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087605559 2024-09-18T00:20:10.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120403+0000", 2024-09-18T00:20:10.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-05 2024-09-18T00:20:10.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120534+0000", 2024-09-18T00:20:10.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013121199999999999 2024-09-18T00:20:10.397 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.397 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.397 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.398 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47932 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8093696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:10.398 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032730+0000", 2024-09-18T00:20:10.398 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.308195343000001, 2024-09-18T00:20:10.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088052486999999999, 2024-09-18T00:20:10.398 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.398 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.398 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.398 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.398 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.398 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.398 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47932 2024-09-18T00:20:10.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.398 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032730+0000", 2024-09-18T00:20:10.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032730+0000", 2024-09-18T00:20:10.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032731+0000", 2024-09-18T00:20:10.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2200000000000004e-07 2024-09-18T00:20:10.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032743+0000", 2024-09-18T00:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1796e-05 2024-09-18T00:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032749+0000", 2024-09-18T00:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6959999999999999e-06 2024-09-18T00:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032755+0000", 2024-09-18T00:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.22e-06 2024-09-18T00:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120578+0000", 2024-09-18T00:20:10.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087823017000000003 2024-09-18T00:20:10.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120654+0000", 2024-09-18T00:20:10.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6476000000000004e-05 2024-09-18T00:20:10.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120783+0000", 2024-09-18T00:20:10.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012876000000000001 2024-09-18T00:20:10.401 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.401 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.402 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.402 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47933 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8126464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:10.402 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032761+0000", 2024-09-18T00:20:10.402 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.308164325, 2024-09-18T00:20:10.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088263233999999996, 2024-09-18T00:20:10.402 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.402 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.402 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.402 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.402 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.402 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.403 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47933 2024-09-18T00:20:10.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.403 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032761+0000", 2024-09-18T00:20:10.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032761+0000", 2024-09-18T00:20:10.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032762+0000", 2024-09-18T00:20:10.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:20:10.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032764+0000", 2024-09-18T00:20:10.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1519999999999999e-06 2024-09-18T00:20:10.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032770+0000", 2024-09-18T00:20:10.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1700000000000002e-06 2024-09-18T00:20:10.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032778+0000", 2024-09-18T00:20:10.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2400000000000001e-06 2024-09-18T00:20:10.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120825+0000", 2024-09-18T00:20:10.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088047046000000004 2024-09-18T00:20:10.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.406 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120898+0000", 2024-09-18T00:20:10.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3011999999999998e-05 2024-09-18T00:20:10.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.406 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121025+0000", 2024-09-18T00:20:10.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012710999999999999 2024-09-18T00:20:10.406 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.406 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.406 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.406 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47934 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8159232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:10.407 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032784+0000", 2024-09-18T00:20:10.407 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.308141773999999, 2024-09-18T00:20:10.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088479458999999996, 2024-09-18T00:20:10.407 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.407 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.407 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.407 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.407 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.407 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.407 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47934 2024-09-18T00:20:10.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.407 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032784+0000", 2024-09-18T00:20:10.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032784+0000", 2024-09-18T00:20:10.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032785+0000", 2024-09-18T00:20:10.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9599999999999999e-07 2024-09-18T00:20:10.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032787+0000", 2024-09-18T00:20:10.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.052e-06 2024-09-18T00:20:10.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032793+0000", 2024-09-18T00:20:10.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2439999999999998e-06 2024-09-18T00:20:10.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032798+0000", 2024-09-18T00:20:10.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9080000000000003e-06 2024-09-18T00:20:10.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121066+0000", 2024-09-18T00:20:10.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088267795999999996 2024-09-18T00:20:10.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121139+0000", 2024-09-18T00:20:10.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2910999999999994e-05 2024-09-18T00:20:10.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121263+0000", 2024-09-18T00:20:10.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000124852 2024-09-18T00:20:10.411 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.411 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.411 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.411 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47935 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8192000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:10.411 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032841+0000", 2024-09-18T00:20:10.411 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.308084694000001, 2024-09-18T00:20:10.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088666069, 2024-09-18T00:20:10.411 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.411 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.411 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.411 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.411 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.411 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.412 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47935 2024-09-18T00:20:10.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.412 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032841+0000", 2024-09-18T00:20:10.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032841+0000", 2024-09-18T00:20:10.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032842+0000", 2024-09-18T00:20:10.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6400000000000002e-07 2024-09-18T00:20:10.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032845+0000", 2024-09-18T00:20:10.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8959999999999999e-06 2024-09-18T00:20:10.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032852+0000", 2024-09-18T00:20:10.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1640000000000004e-06 2024-09-18T00:20:10.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032857+0000", 2024-09-18T00:20:10.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3340000000000001e-06 2024-09-18T00:20:10.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121304+0000", 2024-09-18T00:20:10.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088447001999999997 2024-09-18T00:20:10.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121376+0000", 2024-09-18T00:20:10.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2121999999999998e-05 2024-09-18T00:20:10.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121507+0000", 2024-09-18T00:20:10.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130987 2024-09-18T00:20:10.415 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.415 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.415 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.415 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47936 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:10.415 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:20:10.415 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.308061973000001, 2024-09-18T00:20:10.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088878349999999995, 2024-09-18T00:20:10.416 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.416 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.416 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.416 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.416 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.416 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.416 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47936 2024-09-18T00:20:10.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.416 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:20:10.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:20:10.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:20:10.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8999999999999997e-07 2024-09-18T00:20:10.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032866+0000", 2024-09-18T00:20:10.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.142e-06 2024-09-18T00:20:10.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032873+0000", 2024-09-18T00:20:10.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:20:10.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032878+0000", 2024-09-18T00:20:10.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1220000000000003e-06 2024-09-18T00:20:10.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121548+0000", 2024-09-18T00:20:10.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088670530999999997 2024-09-18T00:20:10.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121621+0000", 2024-09-18T00:20:10.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2941999999999994e-05 2024-09-18T00:20:10.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121742+0000", 2024-09-18T00:20:10.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120903 2024-09-18T00:20:10.419 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.419 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.420 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.420 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47937 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8257536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:10.420 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:20:10.420 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.308041865, 2024-09-18T00:20:10.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089098572000000001, 2024-09-18T00:20:10.420 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.420 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.420 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.420 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.420 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.420 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.421 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47937 2024-09-18T00:20:10.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.421 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.421 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.421 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:20:10.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.421 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.421 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:20:10.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:20:10.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9200000000000001e-07 2024-09-18T00:20:10.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032886+0000", 2024-09-18T00:20:10.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9999999999999999e-06 2024-09-18T00:20:10.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032893+0000", 2024-09-18T00:20:10.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2020000000000001e-06 2024-09-18T00:20:10.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032898+0000", 2024-09-18T00:20:10.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9860000000000002e-06 2024-09-18T00:20:10.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121785+0000", 2024-09-18T00:20:10.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088887318000000007 2024-09-18T00:20:10.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121858+0000", 2024-09-18T00:20:10.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2900999999999999e-05 2024-09-18T00:20:10.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121983+0000", 2024-09-18T00:20:10.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000124673 2024-09-18T00:20:10.424 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.424 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.424 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.424 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47938 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8290304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:10.424 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:20:10.425 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.308022080999997, 2024-09-18T00:20:10.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089335780000000004, 2024-09-18T00:20:10.425 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.425 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.425 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.425 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.425 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.425 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.425 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47938 2024-09-18T00:20:10.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.425 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:20:10.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:20:10.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:20:10.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2799999999999996e-07 2024-09-18T00:20:10.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032923+0000", 2024-09-18T00:20:10.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8218000000000001e-05 2024-09-18T00:20:10.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032930+0000", 2024-09-18T00:20:10.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5440000000000001e-06 2024-09-18T00:20:10.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032936+0000", 2024-09-18T00:20:10.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4720000000000003e-06 2024-09-18T00:20:10.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122023+0000", 2024-09-18T00:20:10.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089087728000000005 2024-09-18T00:20:10.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122117+0000", 2024-09-18T00:20:10.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3295999999999996e-05 2024-09-18T00:20:10.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122240+0000", 2024-09-18T00:20:10.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012289399999999999 2024-09-18T00:20:10.429 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.429 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.429 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.429 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47939 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8323072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:10.429 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032943+0000", 2024-09-18T00:20:10.429 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.307982527999997, 2024-09-18T00:20:10.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089547929999999998, 2024-09-18T00:20:10.430 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.430 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.430 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.430 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.430 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.430 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.430 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47939 2024-09-18T00:20:10.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.430 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.430 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.430 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032943+0000", 2024-09-18T00:20:10.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032943+0000", 2024-09-18T00:20:10.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032944+0000", 2024-09-18T00:20:10.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6199999999999998e-07 2024-09-18T00:20:10.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032946+0000", 2024-09-18T00:20:10.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2840000000000001e-06 2024-09-18T00:20:10.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032952+0000", 2024-09-18T00:20:10.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2700000000000001e-06 2024-09-18T00:20:10.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032957+0000", 2024-09-18T00:20:10.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0200000000000002e-06 2024-09-18T00:20:10.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122280+0000", 2024-09-18T00:20:10.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089322330000000005 2024-09-18T00:20:10.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122354+0000", 2024-09-18T00:20:10.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3794000000000005e-05 2024-09-18T00:20:10.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122491+0000", 2024-09-18T00:20:10.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013767 2024-09-18T00:20:10.433 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.433 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.434 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.434 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47940 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8355840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:10.434 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:20:10.434 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.307962099999997, 2024-09-18T00:20:10.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089758663000000002, 2024-09-18T00:20:10.434 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.434 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.434 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.434 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.434 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.434 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.435 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47940 2024-09-18T00:20:10.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.435 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:20:10.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:20:10.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:20:10.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8599999999999998e-07 2024-09-18T00:20:10.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032966+0000", 2024-09-18T00:20:10.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1160000000000002e-06 2024-09-18T00:20:10.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032972+0000", 2024-09-18T00:20:10.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1639999999999998e-06 2024-09-18T00:20:10.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032978+0000", 2024-09-18T00:20:10.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1320000000000002e-06 2024-09-18T00:20:10.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122520+0000", 2024-09-18T00:20:10.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089541930000000006 2024-09-18T00:20:10.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122601+0000", 2024-09-18T00:20:10.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1207999999999999e-05 2024-09-18T00:20:10.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122722+0000", 2024-09-18T00:20:10.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012162700000000001 2024-09-18T00:20:10.438 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.438 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.438 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.438 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:61476 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-18T00:20:10.438 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:57.749097+0000", 2024-09-18T00:20:10.438 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.591829050999999, 2024-09-18T00:20:10.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99209296800000002, 2024-09-18T00:20:10.439 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.439 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.439 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.439 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.439 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.439 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.439 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 61476 2024-09-18T00:20:10.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.439 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:57.749097+0000", 2024-09-18T00:20:10.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:57.749097+0000", 2024-09-18T00:20:10.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:57.749100+0000", 2024-09-18T00:20:10.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.903e-06 2024-09-18T00:20:10.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:57.749113+0000", 2024-09-18T00:20:10.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3654e-05 2024-09-18T00:20:10.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:57.749146+0000", 2024-09-18T00:20:10.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2815999999999999e-05 2024-09-18T00:20:10.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:57.749161+0000", 2024-09-18T00:20:10.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4893e-05 2024-09-18T00:20:10.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:57.749283+0000", 2024-09-18T00:20:10.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012157100000000001 2024-09-18T00:20:10.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:10.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:57.749304+0000", 2024-09-18T00:20:10.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0999999999999999e-05 2024-09-18T00:20:10.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:58.740510+0000", 2024-09-18T00:20:10.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99120623299999999 2024-09-18T00:20:10.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:58.740839+0000", 2024-09-18T00:20:10.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032949999999999999 2024-09-18T00:20:10.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:58.741190+0000", 2024-09-18T00:20:10.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00035039799999999997 2024-09-18T00:20:10.443 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.443 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.443 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.443 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.443 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.444 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:10.444 INFO:teuthology.orchestra.run.smithi097.stdout:3.a 2024-09-18T00:20:10.444 INFO:teuthology.orchestra.run.smithi097.stdout:3.6 2024-09-18T00:20:10.445 INFO:teuthology.orchestra.run.smithi097.stdout:3.8 2024-09-18T00:20:10.445 INFO:teuthology.orchestra.run.smithi097.stdout:3.4 2024-09-18T00:20:10.445 INFO:teuthology.orchestra.run.smithi097.stdout:3.1a 2024-09-18T00:20:10.445 INFO:teuthology.orchestra.run.smithi097.stdout:3.19 2024-09-18T00:20:10.445 INFO:teuthology.orchestra.run.smithi097.stdout:3.0 2024-09-18T00:20:10.445 INFO:teuthology.orchestra.run.smithi097.stdout:2.6 2024-09-18T00:20:10.445 INFO:teuthology.orchestra.run.smithi097.stdout:2.4 2024-09-18T00:20:10.445 INFO:teuthology.orchestra.run.smithi097.stdout:2.2 2024-09-18T00:20:10.445 INFO:teuthology.orchestra.run.smithi097.stdout:2.0 2024-09-18T00:20:10.446 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:20:10.446 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:20:10.441+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:20:10.503 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is unset 2024-09-18T00:20:10.545 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:10.541+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:10.622 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:10.622 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:10.622 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:20:10.622 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:10.640 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:20:10.646 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:10.644+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:10.747 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:10.745+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:10.802 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:10.803 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:10.803 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:20:10.803 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:20:10.803 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:10.817 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:20:10.858 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:10.857+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:10.925 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op export --pgid 3.6 --file /var/log/ceph/exp.3.6.1 2024-09-18T00:20:10.950 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:10.949+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:10.982 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2229 3.10 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.716283811, 2024-09-18T00:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048730086999999998, 2024-09-18T00:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2229 2024-09-18T00:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263362+0000", 2024-09-18T00:20:10.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3869999999999999e-06 2024-09-18T00:20:10.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263381+0000", 2024-09-18T00:20:10.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9153000000000001e-05 2024-09-18T00:20:10.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263418+0000", 2024-09-18T00:20:10.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6346000000000001e-05 2024-09-18T00:20:10.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263435+0000", 2024-09-18T00:20:10.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7309999999999999e-05 2024-09-18T00:20:10.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263589+0000", 2024-09-18T00:20:10.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015451199999999999 2024-09-18T00:20:10.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263902+0000", 2024-09-18T00:20:10.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031291799999999997 2024-09-18T00:20:10.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:10.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.265995+0000", 2024-09-18T00:20:10.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020931809999999999 2024-09-18T00:20:10.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:10.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311788+0000", 2024-09-18T00:20:10.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045792078 2024-09-18T00:20:10.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:10.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311903+0000", 2024-09-18T00:20:10.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011510699999999999 2024-09-18T00:20:10.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:10.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312009+0000", 2024-09-18T00:20:10.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010630599999999999 2024-09-18T00:20:10.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312089+0000", 2024-09-18T00:20:10.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9789000000000005e-05 2024-09-18T00:20:10.988 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.988 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.988 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.989 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2229 3.10 e36/23 ondisk, result = 0)", 2024-09-18T00:20:10.989 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:20:10.989 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.69713815199999, 2024-09-18T00:20:10.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029600955000000002, 2024-09-18T00:20:10.989 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.989 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.989 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:10.989 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:20:10.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:20:10.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282507+0000", 2024-09-18T00:20:10.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9160000000000001e-06 2024-09-18T00:20:10.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282522+0000", 2024-09-18T00:20:10.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5e-05 2024-09-18T00:20:10.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282558+0000", 2024-09-18T00:20:10.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5694000000000001e-05 2024-09-18T00:20:10.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282580+0000", 2024-09-18T00:20:10.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2323e-05 2024-09-18T00:20:10.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311873+0000", 2024-09-18T00:20:10.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029293013999999999 2024-09-18T00:20:10.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311892+0000", 2024-09-18T00:20:10.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8556e-05 2024-09-18T00:20:10.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312105+0000", 2024-09-18T00:20:10.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021345199999999999 2024-09-18T00:20:10.993 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.993 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.993 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.993 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5714 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:20:10.993 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:20:10.993 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 105.66943564899999, 2024-09-18T00:20:10.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027448727999999999, 2024-09-18T00:20:10.993 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:10.993 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:10.994 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:10.994 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:10.994 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:10.994 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:10.994 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5714 2024-09-18T00:20:10.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.994 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:10.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:10.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:20:10.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:10.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:20:10.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:10.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:10.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310211+0000", 2024-09-18T00:20:10.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5639999999999998e-06 2024-09-18T00:20:10.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:10.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312678+0000", 2024-09-18T00:20:10.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002466761 2024-09-18T00:20:10.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:10.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312746+0000", 2024-09-18T00:20:10.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8046000000000005e-05 2024-09-18T00:20:10.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:10.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312774+0000", 2024-09-18T00:20:10.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7813e-05 2024-09-18T00:20:10.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:10.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312927+0000", 2024-09-18T00:20:10.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015328499999999999 2024-09-18T00:20:10.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:10.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313214+0000", 2024-09-18T00:20:10.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028683099999999998 2024-09-18T00:20:10.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:10.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313708+0000", 2024-09-18T00:20:10.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049379100000000004 2024-09-18T00:20:10.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:10.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336895+0000", 2024-09-18T00:20:10.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023186801 2024-09-18T00:20:10.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:10.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337454+0000", 2024-09-18T00:20:10.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055979399999999996 2024-09-18T00:20:10.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:10.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337571+0000", 2024-09-18T00:20:10.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116821 2024-09-18T00:20:10.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:10.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337656+0000", 2024-09-18T00:20:10.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4221000000000006e-05 2024-09-18T00:20:10.999 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.999 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:10.999 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:10.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:10.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:10.999 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6801 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:20:10.999 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:20:10.999 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.94259920099999, 2024-09-18T00:20:10.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051953602000000002, 2024-09-18T00:20:10.999 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.000 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.000 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:11.000 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.000 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.000 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.000 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6801 2024-09-18T00:20:11.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.000 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:20:11.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:20:11.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037046+0000", 2024-09-18T00:20:11.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.19e-06 2024-09-18T00:20:11.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037062+0000", 2024-09-18T00:20:11.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586999999999998e-05 2024-09-18T00:20:11.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037096+0000", 2024-09-18T00:20:11.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4112000000000001e-05 2024-09-18T00:20:11.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037120+0000", 2024-09-18T00:20:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.355e-05 2024-09-18T00:20:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037246+0000", 2024-09-18T00:20:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012652200000000001 2024-09-18T00:20:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037385+0000", 2024-09-18T00:20:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013924599999999999 2024-09-18T00:20:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.040881+0000", 2024-09-18T00:20:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034954830000000002 2024-09-18T00:20:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088794+0000", 2024-09-18T00:20:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047912897000000003 2024-09-18T00:20:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088828+0000", 2024-09-18T00:20:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3880000000000001e-05 2024-09-18T00:20:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088916+0000", 2024-09-18T00:20:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.865e-05 2024-09-18T00:20:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088997+0000", 2024-09-18T00:20:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0484999999999999e-05 2024-09-18T00:20:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:6801 3.10 e42/23 ondisk, result = 0)", 2024-09-18T00:20:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:20:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.92879179099999, 2024-09-18T00:20:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037960394000000001, 2024-09-18T00:20:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:20:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:20:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050852+0000", 2024-09-18T00:20:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:20:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050860+0000", 2024-09-18T00:20:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7729999999999999e-06 2024-09-18T00:20:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050891+0000", 2024-09-18T00:20:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0617000000000002e-05 2024-09-18T00:20:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050910+0000", 2024-09-18T00:20:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.944e-05 2024-09-18T00:20:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088766+0000", 2024-09-18T00:20:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037855633999999999 2024-09-18T00:20:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088784+0000", 2024-09-18T00:20:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7767000000000001e-05 2024-09-18T00:20:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088811+0000", 2024-09-18T00:20:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7382999999999999e-05 2024-09-18T00:20:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12594 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:20:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.588015252999995, 2024-09-18T00:20:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026492600000000002, 2024-09-18T00:20:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12594 2024-09-18T00:20:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:20:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:20:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391629+0000", 2024-09-18T00:20:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.86e-06 2024-09-18T00:20:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391637+0000", 2024-09-18T00:20:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3999999999999992e-06 2024-09-18T00:20:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391664+0000", 2024-09-18T00:20:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6536000000000001e-05 2024-09-18T00:20:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391681+0000", 2024-09-18T00:20:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6563e-05 2024-09-18T00:20:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417791+0000", 2024-09-18T00:20:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026110936000000001 2024-09-18T00:20:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417933+0000", 2024-09-18T00:20:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014112099999999999 2024-09-18T00:20:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418120+0000", 2024-09-18T00:20:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018718399999999999 2024-09-18T00:20:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12595 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:20:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.587939055000007, 2024-09-18T00:20:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026819428999999999, 2024-09-18T00:20:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12595 2024-09-18T00:20:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:20:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:20:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391705+0000", 2024-09-18T00:20:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3999999999999999e-06 2024-09-18T00:20:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391711+0000", 2024-09-18T00:20:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7459999999999997e-06 2024-09-18T00:20:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391733+0000", 2024-09-18T00:20:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2923000000000001e-05 2024-09-18T00:20:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391748+0000", 2024-09-18T00:20:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4790000000000001e-05 2024-09-18T00:20:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418180+0000", 2024-09-18T00:20:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026431888000000001 2024-09-18T00:20:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418328+0000", 2024-09-18T00:20:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000147725 2024-09-18T00:20:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418523+0000", 2024-09-18T00:20:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000194957 2024-09-18T00:20:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12596 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:20:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.587876086999998, 2024-09-18T00:20:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027220296000000001, 2024-09-18T00:20:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12596 2024-09-18T00:20:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:20:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:20:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391768+0000", 2024-09-18T00:20:11.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.283e-06 2024-09-18T00:20:11.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391773+0000", 2024-09-18T00:20:11.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3970000000000002e-06 2024-09-18T00:20:11.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391795+0000", 2024-09-18T00:20:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1835e-05 2024-09-18T00:20:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391809+0000", 2024-09-18T00:20:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4107000000000001e-05 2024-09-18T00:20:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418582+0000", 2024-09-18T00:20:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026772918999999999 2024-09-18T00:20:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418780+0000", 2024-09-18T00:20:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000198042 2024-09-18T00:20:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418987+0000", 2024-09-18T00:20:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000206713 2024-09-18T00:20:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12597 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:20:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.587815569, 2024-09-18T00:20:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027612714, 2024-09-18T00:20:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12597 2024-09-18T00:20:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:20:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:20:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391828+0000", 2024-09-18T00:20:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.203e-06 2024-09-18T00:20:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391834+0000", 2024-09-18T00:20:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6529999999999999e-06 2024-09-18T00:20:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391856+0000", 2024-09-18T00:20:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2195999999999999e-05 2024-09-18T00:20:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391870+0000", 2024-09-18T00:20:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4063000000000001e-05 2024-09-18T00:20:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419049+0000", 2024-09-18T00:20:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027178751000000001 2024-09-18T00:20:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419239+0000", 2024-09-18T00:20:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019039400000000001 2024-09-18T00:20:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419440+0000", 2024-09-18T00:20:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000200454 2024-09-18T00:20:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12598 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:20:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.587752518000002, 2024-09-18T00:20:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028269056000000001, 2024-09-18T00:20:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12598 2024-09-18T00:20:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:20:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:20:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391891+0000", 2024-09-18T00:20:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.257e-06 2024-09-18T00:20:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391897+0000", 2024-09-18T00:20:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9560000000000002e-06 2024-09-18T00:20:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391920+0000", 2024-09-18T00:20:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3025999999999999e-05 2024-09-18T00:20:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391935+0000", 2024-09-18T00:20:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4853e-05 2024-09-18T00:20:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419527+0000", 2024-09-18T00:20:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027592378000000001 2024-09-18T00:20:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419681+0000", 2024-09-18T00:20:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015316 2024-09-18T00:20:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420159+0000", 2024-09-18T00:20:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000478426 2024-09-18T00:20:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12599 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:20:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.587688662999994, 2024-09-18T00:20:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028649857000000001, 2024-09-18T00:20:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12599 2024-09-18T00:20:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:20:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:20:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391955+0000", 2024-09-18T00:20:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-06 2024-09-18T00:20:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391961+0000", 2024-09-18T00:20:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7100000000000004e-06 2024-09-18T00:20:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391982+0000", 2024-09-18T00:20:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1492e-05 2024-09-18T00:20:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391996+0000", 2024-09-18T00:20:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4194e-05 2024-09-18T00:20:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420237+0000", 2024-09-18T00:20:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028240818000000001 2024-09-18T00:20:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420430+0000", 2024-09-18T00:20:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019228299999999999 2024-09-18T00:20:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420604+0000", 2024-09-18T00:20:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000174117 2024-09-18T00:20:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12600 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 270336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:20:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.587627669, 2024-09-18T00:20:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028964586, 2024-09-18T00:20:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12600 2024-09-18T00:20:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:20:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:20:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392016+0000", 2024-09-18T00:20:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.31e-06 2024-09-18T00:20:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392022+0000", 2024-09-18T00:20:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5600000000000001e-06 2024-09-18T00:20:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392045+0000", 2024-09-18T00:20:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3595999999999999e-05 2024-09-18T00:20:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392060+0000", 2024-09-18T00:20:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4727e-05 2024-09-18T00:20:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420669+0000", 2024-09-18T00:20:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028608855999999998 2024-09-18T00:20:11.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420813+0000", 2024-09-18T00:20:11.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014429500000000001 2024-09-18T00:20:11.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420979+0000", 2024-09-18T00:20:11.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016624200000000001 2024-09-18T00:20:11.041 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.041 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.041 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12601 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:20:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.587563763999995, 2024-09-18T00:20:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029292832000000001, 2024-09-18T00:20:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12601 2024-09-18T00:20:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:20:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:20:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392080+0000", 2024-09-18T00:20:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:20:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392086+0000", 2024-09-18T00:20:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7570000000000001e-06 2024-09-18T00:20:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392105+0000", 2024-09-18T00:20:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9415999999999998e-05 2024-09-18T00:20:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392132+0000", 2024-09-18T00:20:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7019e-05 2024-09-18T00:20:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421049+0000", 2024-09-18T00:20:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028916659000000001 2024-09-18T00:20:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421204+0000", 2024-09-18T00:20:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155224 2024-09-18T00:20:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421372+0000", 2024-09-18T00:20:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016754699999999999 2024-09-18T00:20:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12602 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:20:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.587491133, 2024-09-18T00:20:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029612833000000002, 2024-09-18T00:20:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12602 2024-09-18T00:20:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:20:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:20:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392152+0000", 2024-09-18T00:20:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0470000000000001e-06 2024-09-18T00:20:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392158+0000", 2024-09-18T00:20:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:20:11.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392181+0000", 2024-09-18T00:20:11.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2629e-05 2024-09-18T00:20:11.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392222+0000", 2024-09-18T00:20:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0772000000000002e-05 2024-09-18T00:20:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421441+0000", 2024-09-18T00:20:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029219327 2024-09-18T00:20:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421608+0000", 2024-09-18T00:20:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000166813 2024-09-18T00:20:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421764+0000", 2024-09-18T00:20:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015628500000000001 2024-09-18T00:20:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12603 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:20:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.587400739000003, 2024-09-18T00:20:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029902524, 2024-09-18T00:20:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12603 2024-09-18T00:20:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:20:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:20:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392243+0000", 2024-09-18T00:20:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:20:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392260+0000", 2024-09-18T00:20:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6860000000000001e-05 2024-09-18T00:20:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392282+0000", 2024-09-18T00:20:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2713000000000001e-05 2024-09-18T00:20:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392297+0000", 2024-09-18T00:20:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4375e-05 2024-09-18T00:20:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421830+0000", 2024-09-18T00:20:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029532668000000002 2024-09-18T00:20:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421985+0000", 2024-09-18T00:20:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155613 2024-09-18T00:20:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422144+0000", 2024-09-18T00:20:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000159102 2024-09-18T00:20:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12604 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:20:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.587325687999993, 2024-09-18T00:20:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030228695, 2024-09-18T00:20:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12604 2024-09-18T00:20:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:20:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:20:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392318+0000", 2024-09-18T00:20:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:20:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392337+0000", 2024-09-18T00:20:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:20:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392364+0000", 2024-09-18T00:20:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7288999999999999e-05 2024-09-18T00:20:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392379+0000", 2024-09-18T00:20:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:20:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422209+0000", 2024-09-18T00:20:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029829050999999999 2024-09-18T00:20:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422388+0000", 2024-09-18T00:20:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017988699999999999 2024-09-18T00:20:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422545+0000", 2024-09-18T00:20:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015701199999999999 2024-09-18T00:20:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12605 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:20:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.587241126999999, 2024-09-18T00:20:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030488674, 2024-09-18T00:20:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12605 2024-09-18T00:20:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:20:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:20:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392403+0000", 2024-09-18T00:20:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.883e-06 2024-09-18T00:20:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392421+0000", 2024-09-18T00:20:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7526000000000001e-05 2024-09-18T00:20:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392450+0000", 2024-09-18T00:20:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8972e-05 2024-09-18T00:20:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392488+0000", 2024-09-18T00:20:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8522000000000001e-05 2024-09-18T00:20:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422614+0000", 2024-09-18T00:20:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030125945000000001 2024-09-18T00:20:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422752+0000", 2024-09-18T00:20:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000138178 2024-09-18T00:20:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422890+0000", 2024-09-18T00:20:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013764800000000001 2024-09-18T00:20:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12606 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:20:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.587132467000004, 2024-09-18T00:20:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033727073000000003, 2024-09-18T00:20:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12606 2024-09-18T00:20:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:20:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:20:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392511+0000", 2024-09-18T00:20:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3090000000000001e-06 2024-09-18T00:20:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392517+0000", 2024-09-18T00:20:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.993e-06 2024-09-18T00:20:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392543+0000", 2024-09-18T00:20:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6152999999999998e-05 2024-09-18T00:20:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392558+0000", 2024-09-18T00:20:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4610000000000001e-05 2024-09-18T00:20:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.425959+0000", 2024-09-18T00:20:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033400484000000001 2024-09-18T00:20:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426106+0000", 2024-09-18T00:20:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014779800000000001 2024-09-18T00:20:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426237+0000", 2024-09-18T00:20:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013072600000000001 2024-09-18T00:20:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17562 3.a 3:5578626c:::rbd_data.112a6efb7564.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T00:20:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:20:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 81.021667305999998, 2024-09-18T00:20:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052733453999999999, 2024-09-18T00:20:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17562 2024-09-18T00:20:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:20:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:20:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957977+0000", 2024-09-18T00:20:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.119e-06 2024-09-18T00:20:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957988+0000", 2024-09-18T00:20:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0557000000000001e-05 2024-09-18T00:20:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958017+0000", 2024-09-18T00:20:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9156e-05 2024-09-18T00:20:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958036+0000", 2024-09-18T00:20:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8919e-05 2024-09-18T00:20:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958144+0000", 2024-09-18T00:20:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107905 2024-09-18T00:20:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958284+0000", 2024-09-18T00:20:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140216 2024-09-18T00:20:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.960572+0000", 2024-09-18T00:20:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022881860000000002 2024-09-18T00:20:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010537+0000", 2024-09-18T00:20:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049964664999999998 2024-09-18T00:20:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010590+0000", 2024-09-18T00:20:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3094999999999999e-05 2024-09-18T00:20:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010642+0000", 2024-09-18T00:20:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2073000000000001e-05 2024-09-18T00:20:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010709+0000", 2024-09-18T00:20:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6563000000000006e-05 2024-09-18T00:20:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17562 3.a e60/23 ondisk, result = 0)", 2024-09-18T00:20:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:20:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 81.011463934999995, 2024-09-18T00:20:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042539307999999998, 2024-09-18T00:20:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:20:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:20:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968180+0000", 2024-09-18T00:20:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-18T00:20:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968188+0000", 2024-09-18T00:20:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7419999999999997e-06 2024-09-18T00:20:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968217+0000", 2024-09-18T00:20:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8926999999999999e-05 2024-09-18T00:20:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968231+0000", 2024-09-18T00:20:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4168999999999999e-05 2024-09-18T00:20:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010575+0000", 2024-09-18T00:20:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042343923999999998 2024-09-18T00:20:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010584+0000", 2024-09-18T00:20:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-18T00:20:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010718+0000", 2024-09-18T00:20:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013344499999999999 2024-09-18T00:20:11.082 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.082 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.082 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.082 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.082 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.082 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:11.083 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:20:11.084 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:11.081+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:11.184 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:11.180+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:11.252 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:11.252 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:11.252 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:20:11.252 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:11.266 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:20:11.314 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:20:11.313+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:20:11.385 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:20:11.381+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:20:11.386 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:20:11.386 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:20:11.386 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:11.400 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:20:11.487 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:11.484+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:11.575 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:11.575 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:20:11.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:20:11.576 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:20:11.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.576 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2228 3.15 3:aa2fc059:::rbd_data.112aa7ba4578.000000000000014a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:20:11.576 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:20:11.576 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 111.30889048100001, 2024-09-18T00:20:11.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.068966377999999995, 2024-09-18T00:20:11.576 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.576 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.576 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:11.576 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.576 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.577 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.577 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2228 2024-09-18T00:20:11.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.577 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:20:11.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:20:11.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263099+0000", 2024-09-18T00:20:11.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2359999999999999e-06 2024-09-18T00:20:11.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263115+0000", 2024-09-18T00:20:11.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5996999999999999e-05 2024-09-18T00:20:11.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263156+0000", 2024-09-18T00:20:11.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:20:11.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263183+0000", 2024-09-18T00:20:11.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7233000000000002e-05 2024-09-18T00:20:11.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263304+0000", 2024-09-18T00:20:11.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121209 2024-09-18T00:20:11.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263528+0000", 2024-09-18T00:20:11.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022401800000000001 2024-09-18T00:20:11.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:20:11.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.266458+0000", 2024-09-18T00:20:11.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029296980000000001 2024-09-18T00:20:11.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:11.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331628+0000", 2024-09-18T00:20:11.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.065170045999999995 2024-09-18T00:20:11.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:11.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331819+0000", 2024-09-18T00:20:11.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000190819 2024-09-18T00:20:11.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:11.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331971+0000", 2024-09-18T00:20:11.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151972 2024-09-18T00:20:11.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332063+0000", 2024-09-18T00:20:11.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2083999999999995e-05 2024-09-18T00:20:11.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.582 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.582 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2854 3.f 3:f02b0bca:::rbd_data.112aa7ba4578.0000000000000135:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:20:11.582 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:20:11.582 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.701917991, 2024-09-18T00:20:11.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.082582802999999996, 2024-09-18T00:20:11.583 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.583 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.583 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:11.584 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.584 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.584 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.584 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2854 2024-09-18T00:20:11.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.584 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:20:11.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:20:11.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870071+0000", 2024-09-18T00:20:11.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.227e-06 2024-09-18T00:20:11.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870083+0000", 2024-09-18T00:20:11.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1909999999999999e-05 2024-09-18T00:20:11.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870114+0000", 2024-09-18T00:20:11.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0576e-05 2024-09-18T00:20:11.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870129+0000", 2024-09-18T00:20:11.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.485e-05 2024-09-18T00:20:11.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870256+0000", 2024-09-18T00:20:11.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012691300000000001 2024-09-18T00:20:11.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870493+0000", 2024-09-18T00:20:11.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023737399999999999 2024-09-18T00:20:11.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:11.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.874933+0000", 2024-09-18T00:20:11.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044398299999999996 2024-09-18T00:20:11.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:11.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952452+0000", 2024-09-18T00:20:11.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.077518918000000006 2024-09-18T00:20:11.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:11.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952486+0000", 2024-09-18T00:20:11.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4035999999999997e-05 2024-09-18T00:20:11.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:11.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952575+0000", 2024-09-18T00:20:11.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.954e-05 2024-09-18T00:20:11.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952652+0000", 2024-09-18T00:20:11.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6629000000000001e-05 2024-09-18T00:20:11.589 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.589 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.589 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.590 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2854 3.f e38/32 ondisk, result = 0)", 2024-09-18T00:20:11.590 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:20:11.590 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.684387176, 2024-09-18T00:20:11.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064868625999999999, 2024-09-18T00:20:11.590 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.590 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.590 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.590 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:20:11.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:20:11.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887602+0000", 2024-09-18T00:20:11.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.677e-06 2024-09-18T00:20:11.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887609+0000", 2024-09-18T00:20:11.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6729999999999992e-06 2024-09-18T00:20:11.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887637+0000", 2024-09-18T00:20:11.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8206000000000001e-05 2024-09-18T00:20:11.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887651+0000", 2024-09-18T00:20:11.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3387000000000001e-05 2024-09-18T00:20:11.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952422+0000", 2024-09-18T00:20:11.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064771366999999996 2024-09-18T00:20:11.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952441+0000", 2024-09-18T00:20:11.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8737e-05 2024-09-18T00:20:11.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952469+0000", 2024-09-18T00:20:11.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7579000000000001e-05 2024-09-18T00:20:11.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.593 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.594 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15879 3.16 3:692b47b6:::rbd_object_map.112a13ec2d6c.000000000000000b:head [call] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:20:11.594 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:20:11.594 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 86.917223269000004, 2024-09-18T00:20:11.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.011184018, 2024-09-18T00:20:11.594 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.594 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.594 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.594 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15879 2024-09-18T00:20:11.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.595 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:20:11.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:20:11.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654768+0000", 2024-09-18T00:20:11.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9129999999999996e-06 2024-09-18T00:20:11.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654786+0000", 2024-09-18T00:20:11.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8695999999999999e-05 2024-09-18T00:20:11.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654832+0000", 2024-09-18T00:20:11.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5735000000000002e-05 2024-09-18T00:20:11.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654860+0000", 2024-09-18T00:20:11.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7789000000000001e-05 2024-09-18T00:20:11.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655048+0000", 2024-09-18T00:20:11.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018781900000000001 2024-09-18T00:20:11.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:20:11.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655070+0000", 2024-09-18T00:20:11.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1916000000000002e-05 2024-09-18T00:20:11.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663335+0000", 2024-09-18T00:20:11.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0082651210000000003 2024-09-18T00:20:11.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:11.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663358+0000", 2024-09-18T00:20:11.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2986e-05 2024-09-18T00:20:11.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663739+0000", 2024-09-18T00:20:11.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0003814820000001 2024-09-18T00:20:11.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663920+0000", 2024-09-18T00:20:11.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018116900000000001 2024-09-18T00:20:11.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.665948+0000", 2024-09-18T00:20:11.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020273919999999998 2024-09-18T00:20:11.599 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.599 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.599 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.599 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15881 3.16 3:6cc14920:::rbd_header.112a3285400b:head [watch reconnect cookie 140685950184608 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:20:11.600 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:20:11.600 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 86.917062274000003, 2024-09-18T00:20:11.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0138723599999999, 2024-09-18T00:20:11.600 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.600 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.600 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.600 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.601 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.601 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.601 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15881 2024-09-18T00:20:11.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.601 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:20:11.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:20:11.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654926+0000", 2024-09-18T00:20:11.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6470000000000001e-06 2024-09-18T00:20:11.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654935+0000", 2024-09-18T00:20:11.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8859999999999993e-06 2024-09-18T00:20:11.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654960+0000", 2024-09-18T00:20:11.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4689999999999999e-05 2024-09-18T00:20:11.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654976+0000", 2024-09-18T00:20:11.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6319000000000001e-05 2024-09-18T00:20:11.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655238+0000", 2024-09-18T00:20:11.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026145999999999998 2024-09-18T00:20:11.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:11.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655255+0000", 2024-09-18T00:20:11.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7249999999999999e-05 2024-09-18T00:20:11.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663416+0000", 2024-09-18T00:20:11.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081607959999999997 2024-09-18T00:20:11.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:11.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663436+0000", 2024-09-18T00:20:11.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0384e-05 2024-09-18T00:20:11.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666184+0000", 2024-09-18T00:20:11.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002748121 2024-09-18T00:20:11.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666452+0000", 2024-09-18T00:20:11.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026801700000000001 2024-09-18T00:20:11.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.668797+0000", 2024-09-18T00:20:11.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023447899999999998 2024-09-18T00:20:11.605 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.606 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.606 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.606 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17089 3.f 3:f7681386:::rbd_data.112a6efb7564.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:20:11.606 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:20:11.606 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.692135178000001, 2024-09-18T00:20:11.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.12573889399999999, 2024-09-18T00:20:11.606 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.606 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.606 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:11.606 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.606 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.606 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.607 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17089 2024-09-18T00:20:11.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.607 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:20:11.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:20:11.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879854+0000", 2024-09-18T00:20:11.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:20:11.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879867+0000", 2024-09-18T00:20:11.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3103e-05 2024-09-18T00:20:11.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879898+0000", 2024-09-18T00:20:11.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0103e-05 2024-09-18T00:20:11.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879912+0000", 2024-09-18T00:20:11.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4636e-05 2024-09-18T00:20:11.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879992+0000", 2024-09-18T00:20:11.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9814000000000005e-05 2024-09-18T00:20:11.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.880101+0000", 2024-09-18T00:20:11.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109172 2024-09-18T00:20:11.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:11.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.887269+0000", 2024-09-18T00:20:11.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071679300000000003 2024-09-18T00:20:11.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:11.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005269+0000", 2024-09-18T00:20:11.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.117999621 2024-09-18T00:20:11.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:11.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005317+0000", 2024-09-18T00:20:11.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8279000000000003e-05 2024-09-18T00:20:11.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:11.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005464+0000", 2024-09-18T00:20:11.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014661200000000001 2024-09-18T00:20:11.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005591+0000", 2024-09-18T00:20:11.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000127107 2024-09-18T00:20:11.611 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.611 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.611 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.612 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17089 3.f e59/32 ondisk, result = 0)", 2024-09-18T00:20:11.612 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:20:11.612 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.668028312999994, 2024-09-18T00:20:11.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101334403, 2024-09-18T00:20:11.612 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.612 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.612 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.612 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:20:11.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:20:11.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903960+0000", 2024-09-18T00:20:11.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.33e-06 2024-09-18T00:20:11.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903966+0000", 2024-09-18T00:20:11.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5929999999999996e-06 2024-09-18T00:20:11.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903986+0000", 2024-09-18T00:20:11.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.018e-05 2024-09-18T00:20:11.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903997+0000", 2024-09-18T00:20:11.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0988999999999999e-05 2024-09-18T00:20:11.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005171+0000", 2024-09-18T00:20:11.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101174267 2024-09-18T00:20:11.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005200+0000", 2024-09-18T00:20:11.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8540000000000001e-05 2024-09-18T00:20:11.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005293+0000", 2024-09-18T00:20:11.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504000000000005e-05 2024-09-18T00:20:11.615 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.615 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.615 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.616 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24529 3.16 3:6857ce1a:::rbd_data.112a990280ef.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:20:11.616 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:20:11.616 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.935216901999993, 2024-09-18T00:20:11.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10693805200000001, 2024-09-18T00:20:11.616 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.616 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.616 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:11.616 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.616 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.616 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.616 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24529 2024-09-18T00:20:11.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.617 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:20:11.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:20:11.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636772+0000", 2024-09-18T00:20:11.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1100000000000001e-06 2024-09-18T00:20:11.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636783+0000", 2024-09-18T00:20:11.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1005999999999999e-05 2024-09-18T00:20:11.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636814+0000", 2024-09-18T00:20:11.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1047000000000003e-05 2024-09-18T00:20:11.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636829+0000", 2024-09-18T00:20:11.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4363e-05 2024-09-18T00:20:11.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636980+0000", 2024-09-18T00:20:11.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015181099999999999 2024-09-18T00:20:11.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.637237+0000", 2024-09-18T00:20:11.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000256352 2024-09-18T00:20:11.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:11.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743536+0000", 2024-09-18T00:20:11.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.106298825 2024-09-18T00:20:11.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:11.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743599+0000", 2024-09-18T00:20:11.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3449999999999997e-05 2024-09-18T00:20:11.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743708+0000", 2024-09-18T00:20:11.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109088 2024-09-18T00:20:11.621 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.621 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.621 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.621 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24805 3.16 3:6de5e15d:::rbd_data.112a990280ef.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:20:11.621 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:20:11.621 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.349413670000004, 2024-09-18T00:20:11.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14775107700000001, 2024-09-18T00:20:11.621 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.621 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.621 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:11.621 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.621 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.622 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.622 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24805 2024-09-18T00:20:11.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.622 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:20:11.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:20:11.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222574+0000", 2024-09-18T00:20:11.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0210000000000001e-06 2024-09-18T00:20:11.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222581+0000", 2024-09-18T00:20:11.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.144e-06 2024-09-18T00:20:11.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222595+0000", 2024-09-18T00:20:11.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4567e-05 2024-09-18T00:20:11.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222603+0000", 2024-09-18T00:20:11.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6529999999999994e-06 2024-09-18T00:20:11.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222687+0000", 2024-09-18T00:20:11.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4587999999999997e-05 2024-09-18T00:20:11.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222828+0000", 2024-09-18T00:20:11.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140178 2024-09-18T00:20:11.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:11.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370231+0000", 2024-09-18T00:20:11.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14740378700000001 2024-09-18T00:20:11.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:11.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370269+0000", 2024-09-18T00:20:11.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7886999999999998e-05 2024-09-18T00:20:11.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370324+0000", 2024-09-18T00:20:11.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5251999999999999e-05 2024-09-18T00:20:11.626 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.626 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.626 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.626 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24852 3.13 3:cc64f0fd:::rbd_data.112a990280ef.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:20:11.626 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:20:11.626 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.116674200000006, 2024-09-18T00:20:11.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.066392943999999995, 2024-09-18T00:20:11.626 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.627 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.627 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:11.627 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.627 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.627 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.627 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24852 2024-09-18T00:20:11.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.627 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:20:11.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:20:11.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455315+0000", 2024-09-18T00:20:11.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.705e-06 2024-09-18T00:20:11.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455324+0000", 2024-09-18T00:20:11.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8120000000000002e-06 2024-09-18T00:20:11.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455349+0000", 2024-09-18T00:20:11.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4250999999999999e-05 2024-09-18T00:20:11.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455361+0000", 2024-09-18T00:20:11.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2805e-05 2024-09-18T00:20:11.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455425+0000", 2024-09-18T00:20:11.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3981999999999998e-05 2024-09-18T00:20:11.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455480+0000", 2024-09-18T00:20:11.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4792e-05 2024-09-18T00:20:11.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:11.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.460671+0000", 2024-09-18T00:20:11.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051907300000000002 2024-09-18T00:20:11.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:11.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521355+0000", 2024-09-18T00:20:11.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060684208000000003 2024-09-18T00:20:11.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:11.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521493+0000", 2024-09-18T00:20:11.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013803200000000001 2024-09-18T00:20:11.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:11.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521590+0000", 2024-09-18T00:20:11.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6339e-05 2024-09-18T00:20:11.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521706+0000", 2024-09-18T00:20:11.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116288 2024-09-18T00:20:11.632 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.632 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.632 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.632 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:25269 3.f 3:f6adfd48:::rbd_data.112aa34d8ec3.0000000000000145:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T00:20:11.632 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:20:11.632 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.967084239000002, 2024-09-18T00:20:11.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.075209085999999994, 2024-09-18T00:20:11.632 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.632 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.632 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:11.632 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.632 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.632 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.633 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 25269 2024-09-18T00:20:11.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.633 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:20:11.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:20:11.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604904+0000", 2024-09-18T00:20:11.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:20:11.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604910+0000", 2024-09-18T00:20:11.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.964e-06 2024-09-18T00:20:11.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604925+0000", 2024-09-18T00:20:11.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4894e-05 2024-09-18T00:20:11.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604932+0000", 2024-09-18T00:20:11.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-18T00:20:11.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604990+0000", 2024-09-18T00:20:11.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8143000000000002e-05 2024-09-18T00:20:11.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.605104+0000", 2024-09-18T00:20:11.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011404399999999999 2024-09-18T00:20:11.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:11.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.608755+0000", 2024-09-18T00:20:11.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003651081 2024-09-18T00:20:11.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:11.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679920+0000", 2024-09-18T00:20:11.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.071164543999999996 2024-09-18T00:20:11.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:11.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679978+0000", 2024-09-18T00:20:11.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8585000000000002e-05 2024-09-18T00:20:11.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:11.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680036+0000", 2024-09-18T00:20:11.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7978e-05 2024-09-18T00:20:11.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680112+0000", 2024-09-18T00:20:11.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5527999999999999e-05 2024-09-18T00:20:11.637 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.638 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.638 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.638 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:25269 3.f e72/32 ondisk, result = 0)", 2024-09-18T00:20:11.638 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:20:11.638 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.953873854000001, 2024-09-18T00:20:11.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062014761000000002, 2024-09-18T00:20:11.638 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.638 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.638 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.638 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:20:11.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:20:11.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618114+0000", 2024-09-18T00:20:11.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1650000000000001e-06 2024-09-18T00:20:11.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618119+0000", 2024-09-18T00:20:11.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-18T00:20:11.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618134+0000", 2024-09-18T00:20:11.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5770999999999999e-05 2024-09-18T00:20:11.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618143+0000", 2024-09-18T00:20:11.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.551e-06 2024-09-18T00:20:11.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679967+0000", 2024-09-18T00:20:11.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061823937000000002 2024-09-18T00:20:11.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679975+0000", 2024-09-18T00:20:11.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7340000000000008e-06 2024-09-18T00:20:11.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680128+0000", 2024-09-18T00:20:11.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015338299999999999 2024-09-18T00:20:11.641 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.641 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.642 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.642 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48484 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 15704064~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:11.642 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:20:11.642 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.204671744999999, 2024-09-18T00:20:11.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058450876999999998, 2024-09-18T00:20:11.642 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.642 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.642 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.642 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.642 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.642 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.642 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48484 2024-09-18T00:20:11.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.643 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:20:11.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:20:11.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370429+0000", 2024-09-18T00:20:11.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031138530000000002 2024-09-18T00:20:11.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370435+0000", 2024-09-18T00:20:11.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6269999999999997e-06 2024-09-18T00:20:11.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370455+0000", 2024-09-18T00:20:11.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0426999999999999e-05 2024-09-18T00:20:11.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370465+0000", 2024-09-18T00:20:11.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0118999999999999e-05 2024-09-18T00:20:11.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425525+0000", 2024-09-18T00:20:11.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055059943 2024-09-18T00:20:11.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425645+0000", 2024-09-18T00:20:11.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119392 2024-09-18T00:20:11.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425766+0000", 2024-09-18T00:20:11.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012151599999999999 2024-09-18T00:20:11.646 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.646 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.646 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.646 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48503 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16326656~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:11.646 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:20:11.646 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.200791344000002, 2024-09-18T00:20:11.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058310395000000001, 2024-09-18T00:20:11.646 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.647 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.647 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.647 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48503 2024-09-18T00:20:11.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.647 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:20:11.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:20:11.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:20:11.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0100000000000004e-07 2024-09-18T00:20:11.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371199+0000", 2024-09-18T00:20:11.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8899999999999999e-06 2024-09-18T00:20:11.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371211+0000", 2024-09-18T00:20:11.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1707e-05 2024-09-18T00:20:11.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371218+0000", 2024-09-18T00:20:11.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.131e-06 2024-09-18T00:20:11.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429426+0000", 2024-09-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058207771999999998 2024-09-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429467+0000", 2024-09-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1532999999999997e-05 2024-09-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429506+0000", 2024-09-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8661000000000001e-05 2024-09-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48504 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16359424~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.200759525000002, 2024-09-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058412775, 2024-09-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48504 2024-09-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:20:11.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6899999999999996e-07 2024-09-18T00:20:11.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371231+0000", 2024-09-18T00:20:11.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6709999999999999e-06 2024-09-18T00:20:11.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371240+0000", 2024-09-18T00:20:11.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7309999999999999e-06 2024-09-18T00:20:11.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371249+0000", 2024-09-18T00:20:11.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4280000000000002e-06 2024-09-18T00:20:11.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429533+0000", 2024-09-18T00:20:11.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058283852999999997 2024-09-18T00:20:11.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429590+0000", 2024-09-18T00:20:11.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6824999999999999e-05 2024-09-18T00:20:11.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429640+0000", 2024-09-18T00:20:11.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0498000000000003e-05 2024-09-18T00:20:11.655 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.655 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.655 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.655 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48505 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16392192~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:11.655 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:20:11.655 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.200726551999999, 2024-09-18T00:20:11.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058482777999999999, 2024-09-18T00:20:11.655 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.655 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.655 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.656 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.656 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.656 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.656 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48505 2024-09-18T00:20:11.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.656 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:20:11.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:20:11.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:20:11.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7800000000000001e-07 2024-09-18T00:20:11.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371264+0000", 2024-09-18T00:20:11.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-18T00:20:11.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371274+0000", 2024-09-18T00:20:11.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4690000000000003e-06 2024-09-18T00:20:11.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371281+0000", 2024-09-18T00:20:11.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4120000000000004e-06 2024-09-18T00:20:11.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429665+0000", 2024-09-18T00:20:11.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058384108999999997 2024-09-18T00:20:11.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429705+0000", 2024-09-18T00:20:11.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0299000000000002e-05 2024-09-18T00:20:11.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429743+0000", 2024-09-18T00:20:11.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7951000000000003e-05 2024-09-18T00:20:11.659 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.659 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.659 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.659 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48506 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16424960~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:11.659 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:20:11.659 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.200697339999998, 2024-09-18T00:20:11.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058604485999999997, 2024-09-18T00:20:11.660 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.660 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.660 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.660 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.660 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.660 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.660 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48506 2024-09-18T00:20:11.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.660 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:20:11.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:20:11.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371355+0000", 2024-09-18T00:20:11.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5549000000000005e-05 2024-09-18T00:20:11.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371359+0000", 2024-09-18T00:20:11.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1049999999999997e-06 2024-09-18T00:20:11.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371380+0000", 2024-09-18T00:20:11.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0205000000000001e-05 2024-09-18T00:20:11.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371388+0000", 2024-09-18T00:20:11.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3229999999999998e-06 2024-09-18T00:20:11.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429771+0000", 2024-09-18T00:20:11.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058382878999999999 2024-09-18T00:20:11.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429842+0000", 2024-09-18T00:20:11.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1110000000000002e-05 2024-09-18T00:20:11.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429894+0000", 2024-09-18T00:20:11.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2315000000000003e-05 2024-09-18T00:20:11.663 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.663 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.664 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.664 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48507 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16457728~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:11.664 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:20:11.664 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.20058598, 2024-09-18T00:20:11.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058597275999999997, 2024-09-18T00:20:11.664 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.664 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.664 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.664 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48507 2024-09-18T00:20:11.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.665 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:20:11.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:20:11.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374394+0000", 2024-09-18T00:20:11.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029924449999999998 2024-09-18T00:20:11.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374400+0000", 2024-09-18T00:20:11.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0229999999999998e-06 2024-09-18T00:20:11.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374419+0000", 2024-09-18T00:20:11.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9488e-05 2024-09-18T00:20:11.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374432+0000", 2024-09-18T00:20:11.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2492e-05 2024-09-18T00:20:11.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429917+0000", 2024-09-18T00:20:11.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055485087000000002 2024-09-18T00:20:11.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429958+0000", 2024-09-18T00:20:11.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1258999999999999e-05 2024-09-18T00:20:11.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429998+0000", 2024-09-18T00:20:11.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0482000000000003e-05 2024-09-18T00:20:11.668 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.668 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.668 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.668 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:58332 3.15 3:ac910418:::rbd_data.112a17f95ba0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T00:20:11.668 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:20:11.668 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.514081949000001, 2024-09-18T00:20:11.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.071238735999999997, 2024-09-18T00:20:11.668 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.668 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.668 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:11.668 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:11.669 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:11.669 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:11.669 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 58332 2024-09-18T00:20:11.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.669 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:20:11.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:20:11.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057907+0000", 2024-09-18T00:20:11.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1859999999999999e-06 2024-09-18T00:20:11.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057920+0000", 2024-09-18T00:20:11.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3013000000000001e-05 2024-09-18T00:20:11.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057949+0000", 2024-09-18T00:20:11.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8745999999999999e-05 2024-09-18T00:20:11.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057966+0000", 2024-09-18T00:20:11.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6477000000000002e-05 2024-09-18T00:20:11.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.058035+0000", 2024-09-18T00:20:11.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9560000000000005e-05 2024-09-18T00:20:11.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.058133+0000", 2024-09-18T00:20:11.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8051e-05 2024-09-18T00:20:11.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:20:11.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.061809+0000", 2024-09-18T00:20:11.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003675453 2024-09-18T00:20:11.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:11.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128667+0000", 2024-09-18T00:20:11.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.066857960999999994 2024-09-18T00:20:11.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:11.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128952+0000", 2024-09-18T00:20:11.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028583100000000001 2024-09-18T00:20:11.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:11.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129060+0000", 2024-09-18T00:20:11.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010806000000000001 2024-09-18T00:20:11.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129144+0000", 2024-09-18T00:20:11.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3398000000000003e-05 2024-09-18T00:20:11.674 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.674 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.674 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.674 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:58332 3.15 e96/32 ondisk, result = 0)", 2024-09-18T00:20:11.674 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:20:11.674 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.501889120000001, 2024-09-18T00:20:11.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059068112999999998, 2024-09-18T00:20:11.674 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:11.674 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:11.675 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:11.675 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:11.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:11.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:20:11.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:11.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:20:11.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:11.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:11.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070099+0000", 2024-09-18T00:20:11.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9899999999999999e-07 2024-09-18T00:20:11.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:11.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070103+0000", 2024-09-18T00:20:11.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6880000000000001e-06 2024-09-18T00:20:11.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:11.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070115+0000", 2024-09-18T00:20:11.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2088e-05 2024-09-18T00:20:11.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:11.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070122+0000", 2024-09-18T00:20:11.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3900000000000004e-06 2024-09-18T00:20:11.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:11.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128758+0000", 2024-09-18T00:20:11.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058635541999999999 2024-09-18T00:20:11.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:11.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128774+0000", 2024-09-18T00:20:11.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6858999999999999e-05 2024-09-18T00:20:11.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:11.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:11.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:11.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129166+0000", 2024-09-18T00:20:11.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039164700000000001 2024-09-18T00:20:11.678 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.678 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.678 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.678 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:11.678 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:11.678 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:11.680 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:20:11.680 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:11.676+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:11.780 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:20:11.776+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:20:11.852 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:11.852 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:11.852 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:20:11.852 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:11.866 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:20:11.910 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:11.908+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:11.982 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:11.980+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:11.988 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:11.988 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:11.988 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:20:11.988 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:20:11.988 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:12.002 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:20:12.083 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:20:12.080+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:20:12.173 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:12.173 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:20:12.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:20:12.173 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:20:12.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.173 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34543 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:12.173 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:20:12.173 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.878964641000003, 2024-09-18T00:20:12.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023207593, 2024-09-18T00:20:12.174 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.174 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.174 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:12.174 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.174 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.174 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.174 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34543 2024-09-18T00:20:12.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.174 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:20:12.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:20:12.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290783+0000", 2024-09-18T00:20:12.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5240000000000001e-06 2024-09-18T00:20:12.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290799+0000", 2024-09-18T00:20:12.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:20:12.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290840+0000", 2024-09-18T00:20:12.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0926000000000001e-05 2024-09-18T00:20:12.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290868+0000", 2024-09-18T00:20:12.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8163000000000001e-05 2024-09-18T00:20:12.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291294+0000", 2024-09-18T00:20:12.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00042588700000000001 2024-09-18T00:20:12.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:20:12.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:20:12.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.986e-05 2024-09-18T00:20:12.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316617+0000", 2024-09-18T00:20:12.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025303583000000001 2024-09-18T00:20:12.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:12.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316637+0000", 2024-09-18T00:20:12.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9388000000000001e-05 2024-09-18T00:20:12.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313933+0000", 2024-09-18T00:20:12.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99729681000000003 2024-09-18T00:20:12.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313987+0000", 2024-09-18T00:20:12.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4069e-05 2024-09-18T00:20:12.179 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.179 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.179 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.179 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34544 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:12.179 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:20:12.179 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.878816286000003, 2024-09-18T00:20:12.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0232412120000001, 2024-09-18T00:20:12.180 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.180 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.180 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:12.180 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.180 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.180 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.180 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34544 2024-09-18T00:20:12.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.180 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:20:12.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:20:12.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290930+0000", 2024-09-18T00:20:12.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5030000000000001e-06 2024-09-18T00:20:12.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290940+0000", 2024-09-18T00:20:12.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9000000000000001e-06 2024-09-18T00:20:12.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290964+0000", 2024-09-18T00:20:12.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4273e-05 2024-09-18T00:20:12.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290982+0000", 2024-09-18T00:20:12.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8216e-05 2024-09-18T00:20:12.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291433+0000", 2024-09-18T00:20:12.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000451278 2024-09-18T00:20:12.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:12.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291446+0000", 2024-09-18T00:20:12.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3039e-05 2024-09-18T00:20:12.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316687+0000", 2024-09-18T00:20:12.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025240069 2024-09-18T00:20:12.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:12.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316701+0000", 2024-09-18T00:20:12.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4100000000000001e-05 2024-09-18T00:20:12.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314142+0000", 2024-09-18T00:20:12.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99744119799999997 2024-09-18T00:20:12.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314169+0000", 2024-09-18T00:20:12.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7636000000000001e-05 2024-09-18T00:20:12.185 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.185 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.185 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.185 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34545 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:12.185 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:20:12.185 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.878737764, 2024-09-18T00:20:12.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023257802, 2024-09-18T00:20:12.185 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.185 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.185 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:12.186 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.186 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.186 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.186 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34545 2024-09-18T00:20:12.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.186 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:20:12.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:20:12.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291008+0000", 2024-09-18T00:20:12.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.237e-06 2024-09-18T00:20:12.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291015+0000", 2024-09-18T00:20:12.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2030000000000003e-06 2024-09-18T00:20:12.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291035+0000", 2024-09-18T00:20:12.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.978e-05 2024-09-18T00:20:12.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291049+0000", 2024-09-18T00:20:12.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4429e-05 2024-09-18T00:20:12.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291630+0000", 2024-09-18T00:20:12.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00058103199999999999 2024-09-18T00:20:12.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:12.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291644+0000", 2024-09-18T00:20:12.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.342e-05 2024-09-18T00:20:12.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316753+0000", 2024-09-18T00:20:12.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025108703999999999 2024-09-18T00:20:12.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:12.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316766+0000", 2024-09-18T00:20:12.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3237e-05 2024-09-18T00:20:12.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314238+0000", 2024-09-18T00:20:12.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99747232699999999 2024-09-18T00:20:12.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314265+0000", 2024-09-18T00:20:12.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6432999999999999e-05 2024-09-18T00:20:12.191 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.191 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.191 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.191 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34546 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:12.191 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:20:12.191 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.878679425000001, 2024-09-18T00:20:12.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023324535, 2024-09-18T00:20:12.192 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.192 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.192 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:12.192 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.192 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.192 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.192 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34546 2024-09-18T00:20:12.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.192 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:20:12.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:20:12.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291066+0000", 2024-09-18T00:20:12.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.569999999999999e-07 2024-09-18T00:20:12.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291073+0000", 2024-09-18T00:20:12.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.063e-06 2024-09-18T00:20:12.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291090+0000", 2024-09-18T00:20:12.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6977e-05 2024-09-18T00:20:12.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291105+0000", 2024-09-18T00:20:12.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5166e-05 2024-09-18T00:20:12.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291771+0000", 2024-09-18T00:20:12.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00066599699999999999 2024-09-18T00:20:12.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:12.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291785+0000", 2024-09-18T00:20:12.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.327e-05 2024-09-18T00:20:12.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316826+0000", 2024-09-18T00:20:12.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025041955000000001 2024-09-18T00:20:12.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:12.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316840+0000", 2024-09-18T00:20:12.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3256e-05 2024-09-18T00:20:12.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314335+0000", 2024-09-18T00:20:12.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99749496100000001 2024-09-18T00:20:12.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314390+0000", 2024-09-18T00:20:12.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4932999999999997e-05 2024-09-18T00:20:12.197 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.197 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.197 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.198 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34547 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:12.198 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:20:12.198 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.878620126000001, 2024-09-18T00:20:12.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.028510467, 2024-09-18T00:20:12.198 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.198 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.198 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:12.198 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.198 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.198 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.198 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34547 2024-09-18T00:20:12.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.199 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:20:12.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:20:12.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291126+0000", 2024-09-18T00:20:12.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2699999999999999e-06 2024-09-18T00:20:12.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291150+0000", 2024-09-18T00:20:12.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4669999999999999e-05 2024-09-18T00:20:12.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291171+0000", 2024-09-18T00:20:12.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0706000000000001e-05 2024-09-18T00:20:12.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291186+0000", 2024-09-18T00:20:12.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5126999999999999e-05 2024-09-18T00:20:12.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291885+0000", 2024-09-18T00:20:12.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00069850299999999995 2024-09-18T00:20:12.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:12.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291897+0000", 2024-09-18T00:20:12.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-05 2024-09-18T00:20:12.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316887+0000", 2024-09-18T00:20:12.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024990155999999999 2024-09-18T00:20:12.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:12.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316900+0000", 2024-09-18T00:20:12.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.295e-05 2024-09-18T00:20:12.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319594+0000", 2024-09-18T00:20:12.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0026940719999999 2024-09-18T00:20:12.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319635+0000", 2024-09-18T00:20:12.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0583000000000001e-05 2024-09-18T00:20:12.203 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.203 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.203 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.203 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34548 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:12.203 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:20:12.204 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.878472668000001, 2024-09-18T00:20:12.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0285287400000001, 2024-09-18T00:20:12.204 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.204 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.204 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:12.204 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.204 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.204 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.204 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34548 2024-09-18T00:20:12.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.204 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:20:12.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:20:12.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291274+0000", 2024-09-18T00:20:12.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.12e-06 2024-09-18T00:20:12.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291288+0000", 2024-09-18T00:20:12.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4367e-05 2024-09-18T00:20:12.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:20:12.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5698999999999999e-05 2024-09-18T00:20:12.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291333+0000", 2024-09-18T00:20:12.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8783e-05 2024-09-18T00:20:12.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291987+0000", 2024-09-18T00:20:12.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00065417800000000005 2024-09-18T00:20:12.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:12.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.292000+0000", 2024-09-18T00:20:12.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2652e-05 2024-09-18T00:20:12.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316946+0000", 2024-09-18T00:20:12.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024946696000000001 2024-09-18T00:20:12.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:12.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316956+0000", 2024-09-18T00:20:12.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2760000000000006e-06 2024-09-18T00:20:12.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319752+0000", 2024-09-18T00:20:12.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002796083 2024-09-18T00:20:12.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319801+0000", 2024-09-18T00:20:12.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8885999999999999e-05 2024-09-18T00:20:12.209 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.209 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.209 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.209 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41950 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:12.209 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:20:12.209 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.475819958999999, 2024-09-18T00:20:12.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053368486, 2024-09-18T00:20:12.209 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.209 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.209 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:12.209 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.210 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.210 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.210 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41950 2024-09-18T00:20:12.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.210 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:20:12.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:20:12.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693926+0000", 2024-09-18T00:20:12.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4900000000000004e-07 2024-09-18T00:20:12.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693930+0000", 2024-09-18T00:20:12.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0130000000000004e-06 2024-09-18T00:20:12.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693940+0000", 2024-09-18T00:20:12.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0618000000000001e-05 2024-09-18T00:20:12.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693948+0000", 2024-09-18T00:20:12.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3140000000000004e-06 2024-09-18T00:20:12.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747133+0000", 2024-09-18T00:20:12.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053184689 2024-09-18T00:20:12.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:12.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747203+0000", 2024-09-18T00:20:12.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0339000000000005e-05 2024-09-18T00:20:12.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747293+0000", 2024-09-18T00:20:12.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9563999999999998e-05 2024-09-18T00:20:12.213 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.213 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.213 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.213 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41951 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:12.214 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:20:12.214 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.475785192000004, 2024-09-18T00:20:12.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053527695, 2024-09-18T00:20:12.214 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.214 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.214 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:12.214 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.214 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.214 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.214 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41951 2024-09-18T00:20:12.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.214 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:20:12.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:20:12.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693960+0000", 2024-09-18T00:20:12.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.16e-07 2024-09-18T00:20:12.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693971+0000", 2024-09-18T00:20:12.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1204e-05 2024-09-18T00:20:12.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693981+0000", 2024-09-18T00:20:12.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-18T00:20:12.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693989+0000", 2024-09-18T00:20:12.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0180000000000001e-06 2024-09-18T00:20:12.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747328+0000", 2024-09-18T00:20:12.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053338508 2024-09-18T00:20:12.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:12.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747400+0000", 2024-09-18T00:20:12.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2750999999999995e-05 2024-09-18T00:20:12.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747487+0000", 2024-09-18T00:20:12.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6527999999999996e-05 2024-09-18T00:20:12.218 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.218 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.218 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.218 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41952 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:12.218 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:20:12.218 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.475744785000003, 2024-09-18T00:20:12.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053728013999999998, 2024-09-18T00:20:12.218 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.218 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.218 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:12.218 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.219 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.219 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.219 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41952 2024-09-18T00:20:12.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.219 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:20:12.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:20:12.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694001+0000", 2024-09-18T00:20:12.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3799999999999996e-07 2024-09-18T00:20:12.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694004+0000", 2024-09-18T00:20:12.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.965e-06 2024-09-18T00:20:12.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694013+0000", 2024-09-18T00:20:12.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1689999999999998e-06 2024-09-18T00:20:12.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694022+0000", 2024-09-18T00:20:12.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7560000000000005e-06 2024-09-18T00:20:12.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747531+0000", 2024-09-18T00:20:12.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053508999000000002 2024-09-18T00:20:12.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:12.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747610+0000", 2024-09-18T00:20:12.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9212e-05 2024-09-18T00:20:12.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747728+0000", 2024-09-18T00:20:12.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011797499999999999 2024-09-18T00:20:12.222 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.222 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.222 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.223 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41953 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:12.223 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:20:12.223 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.475711156000003, 2024-09-18T00:20:12.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053936783000000002, 2024-09-18T00:20:12.223 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.223 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.223 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:12.223 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.223 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.223 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.223 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41953 2024-09-18T00:20:12.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.223 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:20:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:20:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694034+0000", 2024-09-18T00:20:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5199999999999996e-07 2024-09-18T00:20:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694037+0000", 2024-09-18T00:20:12.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.745e-06 2024-09-18T00:20:12.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694046+0000", 2024-09-18T00:20:12.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2490000000000007e-06 2024-09-18T00:20:12.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694054+0000", 2024-09-18T00:20:12.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1179999999999992e-06 2024-09-18T00:20:12.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747766+0000", 2024-09-18T00:20:12.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053711362999999998 2024-09-18T00:20:12.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:12.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747856+0000", 2024-09-18T00:20:12.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0779000000000006e-05 2024-09-18T00:20:12.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747970+0000", 2024-09-18T00:20:12.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011377700000000001 2024-09-18T00:20:12.227 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.227 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.227 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.227 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41954 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:12.227 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:20:12.227 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.475681002999998, 2024-09-18T00:20:12.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054104898999999998, 2024-09-18T00:20:12.228 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.228 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.228 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:12.228 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.228 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.228 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.228 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41954 2024-09-18T00:20:12.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.228 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:20:12.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:20:12.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:20:12.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4000000000000002e-07 2024-09-18T00:20:12.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694067+0000", 2024-09-18T00:20:12.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7329999999999999e-06 2024-09-18T00:20:12.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694076+0000", 2024-09-18T00:20:12.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2019999999999993e-06 2024-09-18T00:20:12.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694083+0000", 2024-09-18T00:20:12.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0029999999999997e-06 2024-09-18T00:20:12.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748003+0000", 2024-09-18T00:20:12.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053920463000000002 2024-09-18T00:20:12.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:12.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748077+0000", 2024-09-18T00:20:12.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4041000000000005e-05 2024-09-18T00:20:12.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748168+0000", 2024-09-18T00:20:12.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0916999999999997e-05 2024-09-18T00:20:12.231 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.231 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.231 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.232 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41955 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 231424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:12.232 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:20:12.232 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.475652165, 2024-09-18T00:20:12.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054272659000000001, 2024-09-18T00:20:12.232 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.232 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.232 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:12.232 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.232 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.232 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.232 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41955 2024-09-18T00:20:12.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.233 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:20:12.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:20:12.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694093+0000", 2024-09-18T00:20:12.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:20:12.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694109+0000", 2024-09-18T00:20:12.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6061000000000001e-05 2024-09-18T00:20:12.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694121+0000", 2024-09-18T00:20:12.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.216e-05 2024-09-18T00:20:12.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694129+0000", 2024-09-18T00:20:12.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0029999999999994e-06 2024-09-18T00:20:12.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748202+0000", 2024-09-18T00:20:12.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054072571 2024-09-18T00:20:12.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:12.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748274+0000", 2024-09-18T00:20:12.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2117000000000001e-05 2024-09-18T00:20:12.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748365+0000", 2024-09-18T00:20:12.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1139999999999998e-05 2024-09-18T00:20:12.236 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.236 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.236 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.237 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41956 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:12.237 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:20:12.237 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.475603956999997, 2024-09-18T00:20:12.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054420496999999998, 2024-09-18T00:20:12.237 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.237 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.237 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:12.237 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.237 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.237 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.237 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41956 2024-09-18T00:20:12.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.238 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:20:12.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:20:12.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:20:12.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8699999999999995e-07 2024-09-18T00:20:12.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694144+0000", 2024-09-18T00:20:12.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3510000000000002e-06 2024-09-18T00:20:12.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694154+0000", 2024-09-18T00:20:12.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0769999999999997e-06 2024-09-18T00:20:12.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694163+0000", 2024-09-18T00:20:12.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3300000000000005e-06 2024-09-18T00:20:12.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748397+0000", 2024-09-18T00:20:12.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054234497999999999 2024-09-18T00:20:12.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:12.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748471+0000", 2024-09-18T00:20:12.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3441000000000004e-05 2024-09-18T00:20:12.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748561+0000", 2024-09-18T00:20:12.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0212999999999998e-05 2024-09-18T00:20:12.241 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.241 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.241 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.241 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41957 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:12.241 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:20:12.241 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.475572704999998, 2024-09-18T00:20:12.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054590310000000003, 2024-09-18T00:20:12.241 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.241 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.242 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:12.242 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.242 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.242 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.242 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41957 2024-09-18T00:20:12.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.242 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:20:12.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:20:12.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694173+0000", 2024-09-18T00:20:12.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2500000000000005e-07 2024-09-18T00:20:12.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694175+0000", 2024-09-18T00:20:12.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.655e-06 2024-09-18T00:20:12.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694184+0000", 2024-09-18T00:20:12.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7630000000000006e-06 2024-09-18T00:20:12.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694191+0000", 2024-09-18T00:20:12.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8870000000000003e-06 2024-09-18T00:20:12.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748593+0000", 2024-09-18T00:20:12.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054402525 2024-09-18T00:20:12.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:12.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748669+0000", 2024-09-18T00:20:12.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5532999999999997e-05 2024-09-18T00:20:12.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748762+0000", 2024-09-18T00:20:12.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3221999999999996e-05 2024-09-18T00:20:12.245 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.245 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.245 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.246 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42392 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:12.246 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:20:12.246 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.462543265999997, 2024-09-18T00:20:12.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053378162, 2024-09-18T00:20:12.246 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.246 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.246 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:12.246 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.246 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.246 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.246 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42392 2024-09-18T00:20:12.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.246 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:20:12.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:20:12.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707430+0000", 2024-09-18T00:20:12.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022838599999999999 2024-09-18T00:20:12.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707434+0000", 2024-09-18T00:20:12.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5220000000000001e-06 2024-09-18T00:20:12.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707467+0000", 2024-09-18T00:20:12.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2570000000000002e-05 2024-09-18T00:20:12.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707475+0000", 2024-09-18T00:20:12.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:20:12.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760446+0000", 2024-09-18T00:20:12.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052971635000000003 2024-09-18T00:20:12.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:12.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760512+0000", 2024-09-18T00:20:12.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5679999999999995e-05 2024-09-18T00:20:12.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760579+0000", 2024-09-18T00:20:12.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7296000000000001e-05 2024-09-18T00:20:12.250 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.250 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.250 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.250 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42393 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 358400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:12.250 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:20:12.250 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.462254315000003, 2024-09-18T00:20:12.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053269618999999997, 2024-09-18T00:20:12.250 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.251 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.251 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:12.251 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.251 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.251 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.251 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42393 2024-09-18T00:20:12.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.251 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:20:12.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:20:12.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707491+0000", 2024-09-18T00:20:12.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1399999999999997e-07 2024-09-18T00:20:12.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707503+0000", 2024-09-18T00:20:12.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1912e-05 2024-09-18T00:20:12.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707512+0000", 2024-09-18T00:20:12.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1030000000000008e-06 2024-09-18T00:20:12.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707520+0000", 2024-09-18T00:20:12.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1249999999999993e-06 2024-09-18T00:20:12.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760605+0000", 2024-09-18T00:20:12.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053085495000000003 2024-09-18T00:20:12.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:12.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760680+0000", 2024-09-18T00:20:12.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5024000000000005e-05 2024-09-18T00:20:12.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760760+0000", 2024-09-18T00:20:12.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9345999999999996e-05 2024-09-18T00:20:12.254 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.254 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.254 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.255 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42394 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:12.255 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:20:12.255 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.462214326999998, 2024-09-18T00:20:12.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053403144999999999, 2024-09-18T00:20:12.255 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.255 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.255 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:12.255 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.255 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.255 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.255 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42394 2024-09-18T00:20:12.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.255 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:20:12.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:20:12.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711550+0000", 2024-09-18T00:20:12.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040200189999999997 2024-09-18T00:20:12.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711556+0000", 2024-09-18T00:20:12.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2339999999999999e-06 2024-09-18T00:20:12.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711569+0000", 2024-09-18T00:20:12.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2337999999999999e-05 2024-09-18T00:20:12.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711577+0000", 2024-09-18T00:20:12.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5660000000000007e-06 2024-09-18T00:20:12.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760790+0000", 2024-09-18T00:20:12.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049212196999999999 2024-09-18T00:20:12.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:12.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760857+0000", 2024-09-18T00:20:12.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7756000000000007e-05 2024-09-18T00:20:12.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760933+0000", 2024-09-18T00:20:12.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6035e-05 2024-09-18T00:20:12.259 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.259 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.259 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.259 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:55489 3.5 3:a5f1e468:::rbd_data.112a859a01c9.000000000000000c:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T00:20:12.259 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:20:12.259 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 40.538360201000003, 2024-09-18T00:20:12.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.05748731, 2024-09-18T00:20:12.259 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.259 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.259 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:12.259 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.259 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.259 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.260 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 55489 2024-09-18T00:20:12.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.260 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:20:12.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:20:12.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631388+0000", 2024-09-18T00:20:12.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6169999999999998e-06 2024-09-18T00:20:12.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631405+0000", 2024-09-18T00:20:12.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6643e-05 2024-09-18T00:20:12.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631444+0000", 2024-09-18T00:20:12.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9226e-05 2024-09-18T00:20:12.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631465+0000", 2024-09-18T00:20:12.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0826999999999999e-05 2024-09-18T00:20:12.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631578+0000", 2024-09-18T00:20:12.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011291899999999999 2024-09-18T00:20:12.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:12.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631651+0000", 2024-09-18T00:20:12.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3264999999999997e-05 2024-09-18T00:20:12.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:20:12.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.634073+0000", 2024-09-18T00:20:12.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0024221350000000002 2024-09-18T00:20:12.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:12.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688732+0000", 2024-09-18T00:20:12.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054658610000000003 2024-09-18T00:20:12.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:12.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688794+0000", 2024-09-18T00:20:12.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2255999999999995e-05 2024-09-18T00:20:12.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:12.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688834+0000", 2024-09-18T00:20:12.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9916999999999997e-05 2024-09-18T00:20:12.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688872+0000", 2024-09-18T00:20:12.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7895000000000002e-05 2024-09-18T00:20:12.265 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.265 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.265 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.265 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:58361 3.1d 3:bac90a3d:::rbd_data.112a17f95ba0.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T00:20:12.265 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:20:12.265 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.966014706999999, 2024-09-18T00:20:12.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101552963, 2024-09-18T00:20:12.265 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.265 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.265 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:12.265 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:12.266 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:12.266 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:12.266 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 58361 2024-09-18T00:20:12.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.266 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:20:12.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:20:12.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203733+0000", 2024-09-18T00:20:12.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.01e-06 2024-09-18T00:20:12.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203745+0000", 2024-09-18T00:20:12.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2513e-05 2024-09-18T00:20:12.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203777+0000", 2024-09-18T00:20:12.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1216000000000001e-05 2024-09-18T00:20:12.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203792+0000", 2024-09-18T00:20:12.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586e-05 2024-09-18T00:20:12.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203911+0000", 2024-09-18T00:20:12.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119234 2024-09-18T00:20:12.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:12.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.204056+0000", 2024-09-18T00:20:12.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000144943 2024-09-18T00:20:12.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:12.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.210219+0000", 2024-09-18T00:20:12.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061630649999999997 2024-09-18T00:20:12.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:12.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305071+0000", 2024-09-18T00:20:12.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.094852120999999998 2024-09-18T00:20:12.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:12.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305106+0000", 2024-09-18T00:20:12.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4848999999999998e-05 2024-09-18T00:20:12.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:12.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305208+0000", 2024-09-18T00:20:12.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101518 2024-09-18T00:20:12.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305283+0000", 2024-09-18T00:20:12.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4907999999999995e-05 2024-09-18T00:20:12.271 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.271 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.271 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.271 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:58361 3.1d e96/79 ondisk, result = 0)", 2024-09-18T00:20:12.271 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:20:12.271 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.938531666000003, 2024-09-18T00:20:12.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.073876681, 2024-09-18T00:20:12.271 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:12.271 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:12.271 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:12.271 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:12.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:12.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:20:12.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:12.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:20:12.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:12.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:12.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231215+0000", 2024-09-18T00:20:12.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9570000000000001e-06 2024-09-18T00:20:12.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:12.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231223+0000", 2024-09-18T00:20:12.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0900000000000005e-06 2024-09-18T00:20:12.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:12.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231252+0000", 2024-09-18T00:20:12.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9552000000000001e-05 2024-09-18T00:20:12.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:12.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231267+0000", 2024-09-18T00:20:12.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4263e-05 2024-09-18T00:20:12.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:12.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.304871+0000", 2024-09-18T00:20:12.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.073604257000000006 2024-09-18T00:20:12.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:12.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.304889+0000", 2024-09-18T00:20:12.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.808e-05 2024-09-18T00:20:12.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:12.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:12.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:12.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305090+0000", 2024-09-18T00:20:12.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020048200000000001 2024-09-18T00:20:12.275 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.275 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.275 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.275 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:12.275 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:12.275 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:12.276 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:20:12.277 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:12.272+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:12.378 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:12.376+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:12.408 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:12.408 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:12.421 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:20:12.479 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:12.476+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:12.508 INFO:teuthology.orchestra.run.smithi097.stderr:Exporting 3.6 info 3.6( v 126'321 (0'0,126'321] local-lis/les=124/125 n=11 ec=23/23 lis/c=124/124 les/c/f=125/125/0 sis=124) 2024-09-18T00:20:12.517 INFO:teuthology.orchestra.run.smithi097.stderr:Read #3:60e10a89:::rbd_data.112a4712e9d2.0000000000000027:head# 2024-09-18T00:20:12.518 INFO:teuthology.orchestra.run.smithi097.stderr:Read #3:6104537f:::rbd_data.112a960351d8.0000000000000004:head# 2024-09-18T00:20:12.531 INFO:teuthology.orchestra.run.smithi097.stderr:Read #3:617842cf:::rbd_data.112a5ba20d45.000000000000003f:head# 2024-09-18T00:20:12.531 INFO:teuthology.orchestra.run.smithi097.stderr:Read #3:62e7f20b:::rbd_data.112a9f4259df.0000000000000001:head# 2024-09-18T00:20:12.532 INFO:teuthology.orchestra.run.smithi097.stderr:Read #3:6404e862:::rbd_data.112ab1fa102a.0000000000000004:head# 2024-09-18T00:20:12.533 INFO:teuthology.orchestra.run.smithi097.stderr:Read #3:6437f226:::rbd_data.112aa7ba4578.0000000000000145:head# 2024-09-18T00:20:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:Read #3:649263c7:::rbd_data.112aa7ba4578.0000000000000140:head# 2024-09-18T00:20:12.535 INFO:teuthology.orchestra.run.smithi097.stderr:Read #3:64d23a70:::rbd_data.112a4712e9d2.0000000000000020:head# 2024-09-18T00:20:12.535 INFO:teuthology.orchestra.run.smithi097.stderr:Read #3:65cb5a01:::rbd_data.112a960351d8.000000000000000f:head# 2024-09-18T00:20:12.535 INFO:teuthology.orchestra.run.smithi097.stderr:Read #3:66496aed:::rbd_object_map.112a960351d8:head# 2024-09-18T00:20:12.535 INFO:teuthology.orchestra.run.smithi097.stderr:Read #3:67857717:::rbd_data.112a9f4259df.0000000000000004:head# 2024-09-18T00:20:12.536 INFO:teuthology.orchestra.run.smithi097.stderr:Export successful 2024-09-18T00:20:12.537 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:20:12.546 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:12.547 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:12.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:20:12.579 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:12.576+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:12.681 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:12.676+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:12.729 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:12.729 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:12.744 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:20:12.789 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:12.788+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:12.882 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:12.880+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:12.903 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:12.903 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:12.903 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:20:12.903 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:12.917 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:20:12.982 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:20:12.980+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:20:13.037 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:13.037 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:13.037 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:20:13.038 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:20:13.038 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:13.051 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:20:13.079 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --force --op remove --pgid 3.6 2024-09-18T00:20:13.123 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:13.120+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:13.184 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:13.180+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:13.252 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:20:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:20:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:20:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:5722 3.4 e41/23 3:20381d86:::rbd_data.112a3a39fc45.000000000000001d:head v 41'107, mlcod=40'105)", 2024-09-18T00:20:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:20:13.252 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 107.91446804, 2024-09-18T00:20:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01185172, 2024-09-18T00:20:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:20:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:20:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335497+0000", 2024-09-18T00:20:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.505e-06 2024-09-18T00:20:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335945+0000", 2024-09-18T00:20:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000447687 2024-09-18T00:20:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.254 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335961+0000", 2024-09-18T00:20:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6430999999999999e-05 2024-09-18T00:20:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335969+0000", 2024-09-18T00:20:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:20:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336030+0000", 2024-09-18T00:20:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1255999999999998e-05 2024-09-18T00:20:13.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336061+0000", 2024-09-18T00:20:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0579e-05 2024-09-18T00:20:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347296+0000", 2024-09-18T00:20:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011235663 2024-09-18T00:20:13.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347347+0000", 2024-09-18T00:20:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0914000000000001e-05 2024-09-18T00:20:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11457 3.0 3:0214e7b3:::rbd_data.112ab1fa102a.0000000000000150:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:20:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:20:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 97.990102101999994, 2024-09-18T00:20:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028499165, 2024-09-18T00:20:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.258 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11457 2024-09-18T00:20:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:20:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:20:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.259 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259865+0000", 2024-09-18T00:20:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7230000000000002e-06 2024-09-18T00:20:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259883+0000", 2024-09-18T00:20:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8067000000000002e-05 2024-09-18T00:20:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259924+0000", 2024-09-18T00:20:13.260 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0550999999999999e-05 2024-09-18T00:20:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259943+0000", 2024-09-18T00:20:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:20:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260054+0000", 2024-09-18T00:20:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011075 2024-09-18T00:20:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.261 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260138+0000", 2024-09-18T00:20:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4549000000000007e-05 2024-09-18T00:20:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.287926+0000", 2024-09-18T00:20:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027787642000000001 2024-09-18T00:20:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288090+0000", 2024-09-18T00:20:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016402499999999999 2024-09-18T00:20:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.262 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288361+0000", 2024-09-18T00:20:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027097800000000001 2024-09-18T00:20:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18661 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:20:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.315334237000002, 2024-09-18T00:20:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020008037999999999, 2024-09-18T00:20:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.263 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18661 2024-09-18T00:20:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:20:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.264 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:20:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934633+0000", 2024-09-18T00:20:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8e-06 2024-09-18T00:20:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935511+0000", 2024-09-18T00:20:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087785700000000001 2024-09-18T00:20:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.265 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935561+0000", 2024-09-18T00:20:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9472000000000003e-05 2024-09-18T00:20:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935586+0000", 2024-09-18T00:20:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5616e-05 2024-09-18T00:20:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935838+0000", 2024-09-18T00:20:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025202699999999999 2024-09-18T00:20:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.266 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.936023+0000", 2024-09-18T00:20:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018480300000000001 2024-09-18T00:20:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953545+0000", 2024-09-18T00:20:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017522157 2024-09-18T00:20:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.267 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953723+0000", 2024-09-18T00:20:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017753800000000001 2024-09-18T00:20:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954637+0000", 2024-09-18T00:20:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091476800000000005 2024-09-18T00:20:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.268 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18662 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:20:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.314329299999997, 2024-09-18T00:20:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019388843999999999, 2024-09-18T00:20:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18662 2024-09-18T00:20:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:20:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:20:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.270 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935636+0000", 2024-09-18T00:20:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.663e-06 2024-09-18T00:20:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935646+0000", 2024-09-18T00:20:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000006e-06 2024-09-18T00:20:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.271 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935672+0000", 2024-09-18T00:20:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5868999999999999e-05 2024-09-18T00:20:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935692+0000", 2024-09-18T00:20:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0604000000000001e-05 2024-09-18T00:20:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.272 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950738+0000", 2024-09-18T00:20:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015045868 2024-09-18T00:20:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950992+0000", 2024-09-18T00:20:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025368400000000001 2024-09-18T00:20:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954674+0000", 2024-09-18T00:20:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003682154 2024-09-18T00:20:13.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954806+0000", 2024-09-18T00:20:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013252000000000001 2024-09-18T00:20:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.955023+0000", 2024-09-18T00:20:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021681200000000001 2024-09-18T00:20:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.274 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19382 3.4 3:22f09d3a:::rbd_data.112a7fe29ab4.000000000000015a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:20:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 79.627919250000005, 2024-09-18T00:20:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075119302999999998, 2024-09-18T00:20:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.275 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19382 2024-09-18T00:20:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:20:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.276 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:20:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622045+0000", 2024-09-18T00:20:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0020000000000001e-06 2024-09-18T00:20:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622053+0000", 2024-09-18T00:20:13.277 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2679999999999997e-06 2024-09-18T00:20:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622062+0000", 2024-09-18T00:20:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3749999999999992e-06 2024-09-18T00:20:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622068+0000", 2024-09-18T00:20:13.278 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5729999999999998e-06 2024-09-18T00:20:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622118+0000", 2024-09-18T00:20:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0696999999999999e-05 2024-09-18T00:20:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622222+0000", 2024-09-18T00:20:13.279 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103286 2024-09-18T00:20:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697061+0000", 2024-09-18T00:20:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074839141999999997 2024-09-18T00:20:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697102+0000", 2024-09-18T00:20:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1513e-05 2024-09-18T00:20:13.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697164+0000", 2024-09-18T00:20:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1446999999999996e-05 2024-09-18T00:20:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.281 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20246 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:20:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 79.094231058000005, 2024-09-18T00:20:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01018784, 2024-09-18T00:20:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20246 2024-09-18T00:20:13.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:20:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:20:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:20:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2599999999999998e-07 2024-09-18T00:20:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155735+0000", 2024-09-18T00:20:13.284 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2699999999999999e-06 2024-09-18T00:20:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155744+0000", 2024-09-18T00:20:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8629999999999997e-06 2024-09-18T00:20:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.285 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155749+0000", 2024-09-18T00:20:13.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0359999999999997e-06 2024-09-18T00:20:13.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165748+0000", 2024-09-18T00:20:13.286 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099991449999999992 2024-09-18T00:20:13.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.286 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.286 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.286 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165819+0000", 2024-09-18T00:20:13.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0964999999999996e-05 2024-09-18T00:20:13.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.287 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.287 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165920+0000", 2024-09-18T00:20:13.287 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101135 2024-09-18T00:20:13.287 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.287 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.287 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.287 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20247 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:13.287 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:20:13.287 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 79.094207716, 2024-09-18T00:20:13.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010443987, 2024-09-18T00:20:13.288 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.288 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.288 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:13.288 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.288 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.288 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.288 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20247 2024-09-18T00:20:13.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.288 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.288 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.288 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.288 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:20:13.288 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:20:13.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:20:13.289 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1899999999999998e-07 2024-09-18T00:20:13.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.289 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.289 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.289 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155760+0000", 2024-09-18T00:20:13.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6459999999999999e-06 2024-09-18T00:20:13.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155767+0000", 2024-09-18T00:20:13.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0350000000000004e-06 2024-09-18T00:20:13.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.290 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155774+0000", 2024-09-18T00:20:13.290 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:20:13.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.290 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.290 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165960+0000", 2024-09-18T00:20:13.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186051999999999 2024-09-18T00:20:13.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166048+0000", 2024-09-18T00:20:13.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8848e-05 2024-09-18T00:20:13.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.291 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.291 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.291 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166200+0000", 2024-09-18T00:20:13.291 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015147000000000001 2024-09-18T00:20:13.291 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.291 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.291 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.292 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.292 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20248 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:13.292 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:20:13.292 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 79.094182552000007, 2024-09-18T00:20:13.292 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010617721, 2024-09-18T00:20:13.292 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.292 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.292 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:13.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.292 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.292 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20248 2024-09-18T00:20:13.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.293 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:20:13.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.293 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:20:13.293 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.293 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.293 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155782+0000", 2024-09-18T00:20:13.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.84e-07 2024-09-18T00:20:13.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155784+0000", 2024-09-18T00:20:13.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2800000000000002e-06 2024-09-18T00:20:13.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.294 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.294 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155790+0000", 2024-09-18T00:20:13.294 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5679999999999996e-06 2024-09-18T00:20:13.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.294 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155795+0000", 2024-09-18T00:20:13.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6929999999999998e-06 2024-09-18T00:20:13.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166245+0000", 2024-09-18T00:20:13.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010449847999999999 2024-09-18T00:20:13.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.295 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.295 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166307+0000", 2024-09-18T00:20:13.295 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2287999999999997e-05 2024-09-18T00:20:13.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.295 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.296 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.296 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166399+0000", 2024-09-18T00:20:13.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.166e-05 2024-09-18T00:20:13.296 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.296 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.296 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.296 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.296 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20249 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:13.296 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:20:13.296 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 79.094162175999998, 2024-09-18T00:20:13.296 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010811983000000001, 2024-09-18T00:20:13.297 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.297 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.297 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:13.297 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.297 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.297 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.297 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20249 2024-09-18T00:20:13.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.297 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.297 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:20:13.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.298 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:20:13.298 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.298 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.298 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155802+0000", 2024-09-18T00:20:13.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:20:13.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.299 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.299 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155804+0000", 2024-09-18T00:20:13.299 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:20:13.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.299 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155810+0000", 2024-09-18T00:20:13.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0580000000000003e-06 2024-09-18T00:20:13.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.300 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.300 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155817+0000", 2024-09-18T00:20:13.300 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3980000000000004e-06 2024-09-18T00:20:13.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.300 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.301 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166451+0000", 2024-09-18T00:20:13.301 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010634825000000001 2024-09-18T00:20:13.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.301 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.301 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166520+0000", 2024-09-18T00:20:13.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8317000000000003e-05 2024-09-18T00:20:13.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.302 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.302 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.302 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166613+0000", 2024-09-18T00:20:13.302 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3801000000000005e-05 2024-09-18T00:20:13.302 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.302 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.303 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.303 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:21913 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:20:13.303 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:20:13.303 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 75.956317029999994, 2024-09-18T00:20:13.303 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090508916999999994, 2024-09-18T00:20:13.303 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.303 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.303 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.304 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.304 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 21913 2024-09-18T00:20:13.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.304 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.304 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.304 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:20:13.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.305 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.305 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:20:13.305 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.305 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293648+0000", 2024-09-18T00:20:13.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6400000000000001e-07 2024-09-18T00:20:13.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.306 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.306 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293653+0000", 2024-09-18T00:20:13.306 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3820000000000003e-06 2024-09-18T00:20:13.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.306 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293665+0000", 2024-09-18T00:20:13.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1778e-05 2024-09-18T00:20:13.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293673+0000", 2024-09-18T00:20:13.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4130000000000006e-06 2024-09-18T00:20:13.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.307 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.307 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.307 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293719+0000", 2024-09-18T00:20:13.307 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5534999999999997e-05 2024-09-18T00:20:13.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293773+0000", 2024-09-18T00:20:13.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4389999999999999e-05 2024-09-18T00:20:13.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384053+0000", 2024-09-18T00:20:13.308 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090279617000000006 2024-09-18T00:20:13.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.308 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.308 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.308 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384088+0000", 2024-09-18T00:20:13.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5466000000000002e-05 2024-09-18T00:20:13.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.309 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.309 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384156+0000", 2024-09-18T00:20:13.309 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7471999999999994e-05 2024-09-18T00:20:13.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.309 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.309 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.309 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.309 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22274 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:20:13.309 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:20:13.309 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 75.145200810000006, 2024-09-18T00:20:13.310 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.069820742000000005, 2024-09-18T00:20:13.310 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.310 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.310 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.310 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.310 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22274 2024-09-18T00:20:13.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.310 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.310 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.310 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.310 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:20:13.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:20:13.311 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.311 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.311 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.311 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104766+0000", 2024-09-18T00:20:13.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3859999999999999e-06 2024-09-18T00:20:13.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104783+0000", 2024-09-18T00:20:13.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6697e-05 2024-09-18T00:20:13.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.312 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.312 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.312 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104816+0000", 2024-09-18T00:20:13.312 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2786000000000001e-05 2024-09-18T00:20:13.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104833+0000", 2024-09-18T00:20:13.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7125999999999998e-05 2024-09-18T00:20:13.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.313 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.313 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104965+0000", 2024-09-18T00:20:13.313 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013215899999999999 2024-09-18T00:20:13.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.105103+0000", 2024-09-18T00:20:13.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138193 2024-09-18T00:20:13.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174496+0000", 2024-09-18T00:20:13.314 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.06939244 2024-09-18T00:20:13.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.314 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.314 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.314 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174530+0000", 2024-09-18T00:20:13.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4601999999999998e-05 2024-09-18T00:20:13.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.315 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.315 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174584+0000", 2024-09-18T00:20:13.315 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3353000000000002e-05 2024-09-18T00:20:13.315 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.315 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.315 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.315 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.315 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22485 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write 16521958~30847] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:20:13.316 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:20:13.316 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 74.296411800000001, 2024-09-18T00:20:13.316 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012574648000000001, 2024-09-18T00:20:13.316 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.316 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.316 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.316 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.316 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.316 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.316 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22485 2024-09-18T00:20:13.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.317 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:20:13.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.317 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.317 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:20:13.317 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.317 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953554+0000", 2024-09-18T00:20:13.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1150000000000001e-06 2024-09-18T00:20:13.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962036+0000", 2024-09-18T00:20:13.318 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0084817599999999996 2024-09-18T00:20:13.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.318 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.318 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.318 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962062+0000", 2024-09-18T00:20:13.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6499000000000002e-05 2024-09-18T00:20:13.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962074+0000", 2024-09-18T00:20:13.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2095e-05 2024-09-18T00:20:13.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.319 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962165+0000", 2024-09-18T00:20:13.319 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0703000000000002e-05 2024-09-18T00:20:13.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.319 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.319 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962276+0000", 2024-09-18T00:20:13.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111406 2024-09-18T00:20:13.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.320 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965925+0000", 2024-09-18T00:20:13.320 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036488919999999999 2024-09-18T00:20:13.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.320 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.320 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965992+0000", 2024-09-18T00:20:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6806999999999999e-05 2024-09-18T00:20:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.966127+0000", 2024-09-18T00:20:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013437099999999999 2024-09-18T00:20:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22681 3.4 3:22e76021:::rbd_data.112a948d5956.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:20:13.321 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:20:13.322 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 73.962412494999995, 2024-09-18T00:20:13.322 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076591189000000004, 2024-09-18T00:20:13.322 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.322 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.322 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.322 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.322 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22681 2024-09-18T00:20:13.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.322 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.322 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.322 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.322 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:20:13.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:20:13.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.323 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287554+0000", 2024-09-18T00:20:13.323 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.514e-06 2024-09-18T00:20:13.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.323 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.323 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287567+0000", 2024-09-18T00:20:13.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-18T00:20:13.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287597+0000", 2024-09-18T00:20:13.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0406e-05 2024-09-18T00:20:13.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.324 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.324 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287626+0000", 2024-09-18T00:20:13.324 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9306e-05 2024-09-18T00:20:13.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.324 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287801+0000", 2024-09-18T00:20:13.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017411400000000001 2024-09-18T00:20:13.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.288053+0000", 2024-09-18T00:20:13.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025223500000000001 2024-09-18T00:20:13.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.325 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.325 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.325 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364016+0000", 2024-09-18T00:20:13.325 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075963301999999996 2024-09-18T00:20:13.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364064+0000", 2024-09-18T00:20:13.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7886000000000002e-05 2024-09-18T00:20:13.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.326 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.326 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364142+0000", 2024-09-18T00:20:13.326 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8345999999999998e-05 2024-09-18T00:20:13.326 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.326 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.326 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.326 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.327 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:24700 3.4 3:23a8f5d4:::rbd_data.112a990280ef.0000000000000016:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:20:13.327 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:20:13.327 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 66.311946516000006, 2024-09-18T00:20:13.327 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091975649000000007, 2024-09-18T00:20:13.327 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.327 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.327 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.327 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.328 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 24700 2024-09-18T00:20:13.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.328 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:20:13.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.328 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.328 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.328 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:20:13.328 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938019+0000", 2024-09-18T00:20:13.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:20:13.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.329 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938030+0000", 2024-09-18T00:20:13.329 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0278000000000001e-05 2024-09-18T00:20:13.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.329 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.329 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938053+0000", 2024-09-18T00:20:13.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2946999999999999e-05 2024-09-18T00:20:13.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938062+0000", 2024-09-18T00:20:13.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8509999999999994e-06 2024-09-18T00:20:13.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.330 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.330 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.330 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938140+0000", 2024-09-18T00:20:13.330 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7664999999999996e-05 2024-09-18T00:20:13.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938314+0000", 2024-09-18T00:20:13.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017398500000000001 2024-09-18T00:20:13.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.331 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.331 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029896+0000", 2024-09-18T00:20:13.331 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091582078999999997 2024-09-18T00:20:13.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.331 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029930+0000", 2024-09-18T00:20:13.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3723999999999998e-05 2024-09-18T00:20:13.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.332 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.332 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.332 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029993+0000", 2024-09-18T00:20:13.332 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2901e-05 2024-09-18T00:20:13.332 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.332 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.332 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.333 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.333 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37799 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:13.333 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:20:13.333 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 52.839718365000003, 2024-09-18T00:20:13.333 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186016000000001, 2024-09-18T00:20:13.333 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.333 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.333 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:13.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.333 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.334 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37799 2024-09-18T00:20:13.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.334 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:20:13.334 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.334 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.334 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.334 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:20:13.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410246+0000", 2024-09-18T00:20:13.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.35e-07 2024-09-18T00:20:13.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.335 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.335 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.335 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410247+0000", 2024-09-18T00:20:13.335 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0389999999999999e-06 2024-09-18T00:20:13.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410250+0000", 2024-09-18T00:20:13.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3380000000000001e-06 2024-09-18T00:20:13.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.336 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.336 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410252+0000", 2024-09-18T00:20:13.336 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5150000000000001e-06 2024-09-18T00:20:13.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.336 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420198+0000", 2024-09-18T00:20:13.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099454360000000002 2024-09-18T00:20:13.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420284+0000", 2024-09-18T00:20:13.337 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5638999999999998e-05 2024-09-18T00:20:13.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.337 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.337 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.337 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420431+0000", 2024-09-18T00:20:13.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014781399999999999 2024-09-18T00:20:13.338 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.338 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.338 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.338 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.338 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37800 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:13.338 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:20:13.338 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 52.839708088999998, 2024-09-18T00:20:13.338 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010452559, 2024-09-18T00:20:13.338 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.338 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.339 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:13.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.339 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.339 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37800 2024-09-18T00:20:13.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.339 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.339 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.339 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.339 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:20:13.339 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:20:13.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.340 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.340 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:20:13.340 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2499999999999999e-07 2024-09-18T00:20:13.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.340 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410257+0000", 2024-09-18T00:20:13.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0750000000000001e-06 2024-09-18T00:20:13.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.341 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410260+0000", 2024-09-18T00:20:13.341 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.428e-06 2024-09-18T00:20:13.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.341 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.341 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410263+0000", 2024-09-18T00:20:13.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5339999999999998e-06 2024-09-18T00:20:13.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420467+0000", 2024-09-18T00:20:13.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010204615 2024-09-18T00:20:13.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.342 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.342 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.342 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420557+0000", 2024-09-18T00:20:13.342 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9183000000000003e-05 2024-09-18T00:20:13.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.343 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.343 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420708+0000", 2024-09-18T00:20:13.343 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015149900000000001 2024-09-18T00:20:13.343 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.343 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.343 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.343 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.343 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37801 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:13.343 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:20:13.344 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 52.839697751999999, 2024-09-18T00:20:13.344 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010623357, 2024-09-18T00:20:13.344 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.344 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.344 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:13.344 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.344 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.344 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.344 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37801 2024-09-18T00:20:13.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.344 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:20:13.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:20:13.345 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.345 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.345 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.345 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:20:13.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.23e-07 2024-09-18T00:20:13.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410271+0000", 2024-09-18T00:20:13.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8040000000000001e-06 2024-09-18T00:20:13.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.346 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.346 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410274+0000", 2024-09-18T00:20:13.346 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5310000000000002e-06 2024-09-18T00:20:13.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.346 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410277+0000", 2024-09-18T00:20:13.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.605e-06 2024-09-18T00:20:13.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420743+0000", 2024-09-18T00:20:13.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010465487000000001 2024-09-18T00:20:13.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.347 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.347 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.347 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420811+0000", 2024-09-18T00:20:13.347 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8845999999999998e-05 2024-09-18T00:20:13.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.348 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.348 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420889+0000", 2024-09-18T00:20:13.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7861000000000006e-05 2024-09-18T00:20:13.348 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.348 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.348 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.348 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.348 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37802 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:13.348 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:20:13.348 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 52.839683125999997, 2024-09-18T00:20:13.348 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010745463, 2024-09-18T00:20:13.348 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.348 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.349 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:13.349 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.349 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.349 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.349 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37802 2024-09-18T00:20:13.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.349 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.349 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:20:13.349 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.349 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.349 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:20:13.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:20:13.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6199999999999999e-07 2024-09-18T00:20:13.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.350 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.350 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.350 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410282+0000", 2024-09-18T00:20:13.350 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.085e-06 2024-09-18T00:20:13.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410286+0000", 2024-09-18T00:20:13.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6610000000000002e-06 2024-09-18T00:20:13.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410288+0000", 2024-09-18T00:20:13.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6410000000000002e-06 2024-09-18T00:20:13.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.351 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.351 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420910+0000", 2024-09-18T00:20:13.351 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010621479 2024-09-18T00:20:13.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420952+0000", 2024-09-18T00:20:13.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2497999999999999e-05 2024-09-18T00:20:13.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.352 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.352 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.421026+0000", 2024-09-18T00:20:13.352 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3837000000000005e-05 2024-09-18T00:20:13.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.352 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.352 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.352 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.353 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:59273 3.a 3:535385f0:::rbd_children:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-18T00:20:13.353 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:43.695268+0000", 2024-09-18T00:20:13.353 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 29.554695511999999, 2024-09-18T00:20:13.353 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.89624285599999998, 2024-09-18T00:20:13.353 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.353 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.353 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.353 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.353 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 59273 2024-09-18T00:20:13.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.353 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.353 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695268+0000", 2024-09-18T00:20:13.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695268+0000", 2024-09-18T00:20:13.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.354 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.354 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695270+0000", 2024-09-18T00:20:13.354 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1440000000000001e-06 2024-09-18T00:20:13.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.354 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695278+0000", 2024-09-18T00:20:13.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8839999999999994e-06 2024-09-18T00:20:13.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695296+0000", 2024-09-18T00:20:13.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7957000000000001e-05 2024-09-18T00:20:13.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.355 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.355 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.355 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695304+0000", 2024-09-18T00:20:13.355 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:20:13.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695372+0000", 2024-09-18T00:20:13.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8659999999999997e-05 2024-09-18T00:20:13.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-18T00:20:13.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695383+0000", 2024-09-18T00:20:13.356 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0852e-05 2024-09-18T00:20:13.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.356 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.356 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.356 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.586073+0000", 2024-09-18T00:20:13.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.89068997800000005 2024-09-18T00:20:13.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.586481+0000", 2024-09-18T00:20:13.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00040750900000000002 2024-09-18T00:20:13.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.357 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.357 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-18T00:20:13.357 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.586873+0000", 2024-09-18T00:20:13.357 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00039179999999999998 2024-09-18T00:20:13.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.588815+0000", 2024-09-18T00:20:13.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019426249999999999 2024-09-18T00:20:13.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.358 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:13.358 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.591259+0000", 2024-09-18T00:20:13.358 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0024436829999999999 2024-09-18T00:20:13.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.591399+0000", 2024-09-18T00:20:13.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139927 2024-09-18T00:20:13.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.359 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.359 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.359 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.591511+0000", 2024-09-18T00:20:13.359 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011215200000000001 2024-09-18T00:20:13.359 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.359 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.360 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.360 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:59788 3.9 e106/23 3:94325cfd:::rbd_header.112aa373d2ec:head v 106'325, mlcod=105'323)", 2024-09-18T00:20:13.360 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:48.641886+0000", 2024-09-18T00:20:13.360 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 24.608077667, 2024-09-18T00:20:13.360 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010208398, 2024-09-18T00:20:13.360 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.360 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.360 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.360 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.360 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641886+0000", 2024-09-18T00:20:13.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.361 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.361 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641886+0000", 2024-09-18T00:20:13.361 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641887+0000", 2024-09-18T00:20:13.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0999999999999997e-07 2024-09-18T00:20:13.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641924+0000", 2024-09-18T00:20:13.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7252000000000002e-05 2024-09-18T00:20:13.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.362 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.362 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641955+0000", 2024-09-18T00:20:13.362 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0372e-05 2024-09-18T00:20:13.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641974+0000", 2024-09-18T00:20:13.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9695999999999999e-05 2024-09-18T00:20:13.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.363 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.363 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.642107+0000", 2024-09-18T00:20:13.363 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013297999999999999 2024-09-18T00:20:13.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.642142+0000", 2024-09-18T00:20:13.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4993000000000002e-05 2024-09-18T00:20:13.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.364 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.652006+0000", 2024-09-18T00:20:13.364 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0098642109999999995 2024-09-18T00:20:13.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.364 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.365 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.652094+0000", 2024-09-18T00:20:13.365 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7984000000000003e-05 2024-09-18T00:20:13.365 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.365 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.365 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.365 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.365 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.365 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:13.367 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:20:13.367 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:13.364+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:13.468 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:13.464+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:13.531 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:13.531 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:20:13.531 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:13.535 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is unset 2024-09-18T00:20:13.557 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:20:13.568 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:13.564+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:13.669 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:13.668+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:13.671 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:20:13.671 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:20:13.671 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:13.686 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:20:13.771 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:13.768+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:13.856 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:20:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:20:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:20:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:20:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:20:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 121.052124037, 2024-09-18T00:20:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:20:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.856 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:20:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:20:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:20:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:20:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:20:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:20:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:20:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:20:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:20:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:20:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:20:13.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:20:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:20:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:20:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:20:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:13.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:20:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:20:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:20:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:20:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:13.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:20:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:20:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:20:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:20:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:20:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:20:13.862 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:20:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:20:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 121.035916164, 2024-09-18T00:20:13.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:20:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:20:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:20:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:20:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:20:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:20:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:20:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.865 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:20:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:20:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:20:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:20:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:20:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:20:13.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:20:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:20:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:20:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:20:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.868 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:20:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:20:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 108.008874844, 2024-09-18T00:20:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:20:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.869 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:20:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:20:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:20:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:20:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:20:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:20:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:20:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:20:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:20:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:20:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:20:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:20:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:20:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:20:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:20:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:20:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:20:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:20:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:20:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:20:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:20:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:20:13.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:20:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:20:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:20:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:20:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:20:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 107.987598292, 2024-09-18T00:20:13.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:20:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:20:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:20:13.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:20:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:20:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:20:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:20:13.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:20:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:20:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:20:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:20:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:20:13.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:20:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:20:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:20:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:20:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:20:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11460 3.19 3:9e578b6c:::rbd_data.112ab1fa102a.0000000000000153:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:20:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:20:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 98.585226840000004, 2024-09-18T00:20:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041989011999999999, 2024-09-18T00:20:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11460 2024-09-18T00:20:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.880 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:20:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:20:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267067+0000", 2024-09-18T00:20:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:20:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267088+0000", 2024-09-18T00:20:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0908999999999999e-05 2024-09-18T00:20:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267133+0000", 2024-09-18T00:20:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5232000000000003e-05 2024-09-18T00:20:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267157+0000", 2024-09-18T00:20:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3833e-05 2024-09-18T00:20:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267300+0000", 2024-09-18T00:20:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142282 2024-09-18T00:20:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267447+0000", 2024-09-18T00:20:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147851 2024-09-18T00:20:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308889+0000", 2024-09-18T00:20:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041441779999999998 2024-09-18T00:20:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308956+0000", 2024-09-18T00:20:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7136000000000002e-05 2024-09-18T00:20:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.309053+0000", 2024-09-18T00:20:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6552999999999995e-05 2024-09-18T00:20:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:16947 3.18 3:18b58c7b:::rbd_data.112a6efb7564.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:20:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:20:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 85.235449138999996, 2024-09-18T00:20:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.101692484, 2024-09-18T00:20:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 16947 2024-09-18T00:20:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:20:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:20:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616843+0000", 2024-09-18T00:20:13.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0780000000000001e-06 2024-09-18T00:20:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617785+0000", 2024-09-18T00:20:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094204100000000002 2024-09-18T00:20:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617801+0000", 2024-09-18T00:20:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6436999999999999e-05 2024-09-18T00:20:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617809+0000", 2024-09-18T00:20:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9049999999999997e-06 2024-09-18T00:20:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617904+0000", 2024-09-18T00:20:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5204999999999994e-05 2024-09-18T00:20:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.618071+0000", 2024-09-18T00:20:13.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016640000000000001 2024-09-18T00:20:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.624534+0000", 2024-09-18T00:20:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064634499999999999 2024-09-18T00:20:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718299+0000", 2024-09-18T00:20:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093764739 2024-09-18T00:20:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718375+0000", 2024-09-18T00:20:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6532999999999994e-05 2024-09-18T00:20:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718466+0000", 2024-09-18T00:20:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0581000000000006e-05 2024-09-18T00:20:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718534+0000", 2024-09-18T00:20:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8114999999999994e-05 2024-09-18T00:20:13.893 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:16947 3.18 e59/23 ondisk, result = 0)", 2024-09-18T00:20:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:20:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 85.210232445000003, 2024-09-18T00:20:13.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076488069000000006, 2024-09-18T00:20:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:20:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:20:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642060+0000", 2024-09-18T00:20:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6640000000000001e-06 2024-09-18T00:20:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642071+0000", 2024-09-18T00:20:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1204e-05 2024-09-18T00:20:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642091+0000", 2024-09-18T00:20:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0313999999999999e-05 2024-09-18T00:20:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642103+0000", 2024-09-18T00:20:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1214e-05 2024-09-18T00:20:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718350+0000", 2024-09-18T00:20:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076247681999999997 2024-09-18T00:20:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718366+0000", 2024-09-18T00:20:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5420000000000001e-05 2024-09-18T00:20:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718546+0000", 2024-09-18T00:20:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000180571 2024-09-18T00:20:13.898 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18686 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:20:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.903555075, 2024-09-18T00:20:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041132573999999998, 2024-09-18T00:20:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.899 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18686 2024-09-18T00:20:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:20:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:20:13.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948737+0000", 2024-09-18T00:20:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3769999999999999e-06 2024-09-18T00:20:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949968+0000", 2024-09-18T00:20:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001230493 2024-09-18T00:20:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.901 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949993+0000", 2024-09-18T00:20:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5282e-05 2024-09-18T00:20:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950011+0000", 2024-09-18T00:20:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8199999999999999e-05 2024-09-18T00:20:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950180+0000", 2024-09-18T00:20:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016867600000000001 2024-09-18T00:20:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950337+0000", 2024-09-18T00:20:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156902 2024-09-18T00:20:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950818+0000", 2024-09-18T00:20:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048192599999999998 2024-09-18T00:20:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978491+0000", 2024-09-18T00:20:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027672568000000002 2024-09-18T00:20:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989759+0000", 2024-09-18T00:20:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011268144000000001 2024-09-18T00:20:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989824+0000", 2024-09-18T00:20:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4556999999999999e-05 2024-09-18T00:20:13.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989868+0000", 2024-09-18T00:20:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4449000000000001e-05 2024-09-18T00:20:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18688 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:20:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.902125957999999, 2024-09-18T00:20:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040041259000000003, 2024-09-18T00:20:13.905 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18688 2024-09-18T00:20:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:20:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:20:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950166+0000", 2024-09-18T00:20:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:20:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951295+0000", 2024-09-18T00:20:13.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011293480000000001 2024-09-18T00:20:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951328+0000", 2024-09-18T00:20:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2524999999999998e-05 2024-09-18T00:20:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951346+0000", 2024-09-18T00:20:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8170999999999999e-05 2024-09-18T00:20:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978546+0000", 2024-09-18T00:20:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027200418000000001 2024-09-18T00:20:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978666+0000", 2024-09-18T00:20:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119913 2024-09-18T00:20:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.979098+0000", 2024-09-18T00:20:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00043169399999999999 2024-09-18T00:20:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990000+0000", 2024-09-18T00:20:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010901719000000001 2024-09-18T00:20:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990109+0000", 2024-09-18T00:20:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010964299999999999 2024-09-18T00:20:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990152+0000", 2024-09-18T00:20:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2877000000000003e-05 2024-09-18T00:20:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990206+0000", 2024-09-18T00:20:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3717000000000001e-05 2024-09-18T00:20:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18689 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:20:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.900914259999993, 2024-09-18T00:20:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051022338, 2024-09-18T00:20:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.911 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18689 2024-09-18T00:20:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:20:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:20:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951378+0000", 2024-09-18T00:20:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:20:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951385+0000", 2024-09-18T00:20:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-18T00:20:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951402+0000", 2024-09-18T00:20:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6538999999999999e-05 2024-09-18T00:20:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951565+0000", 2024-09-18T00:20:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016307999999999999 2024-09-18T00:20:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988232+0000", 2024-09-18T00:20:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036667656999999999 2024-09-18T00:20:13.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988318+0000", 2024-09-18T00:20:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5348999999999999e-05 2024-09-18T00:20:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988574+0000", 2024-09-18T00:20:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025670800000000002 2024-09-18T00:20:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002254+0000", 2024-09-18T00:20:13.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013679274 2024-09-18T00:20:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002326+0000", 2024-09-18T00:20:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2720999999999997e-05 2024-09-18T00:20:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002364+0000", 2024-09-18T00:20:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7314000000000002e-05 2024-09-18T00:20:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002399+0000", 2024-09-18T00:20:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5162e-05 2024-09-18T00:20:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18690 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:20:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.900697242000007, 2024-09-18T00:20:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050872134999999999, 2024-09-18T00:20:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.917 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18690 2024-09-18T00:20:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:20:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:20:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951595+0000", 2024-09-18T00:20:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4479999999999999e-06 2024-09-18T00:20:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952167+0000", 2024-09-18T00:20:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00057188400000000004 2024-09-18T00:20:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952192+0000", 2024-09-18T00:20:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4936999999999999e-05 2024-09-18T00:20:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952206+0000", 2024-09-18T00:20:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4565e-05 2024-09-18T00:20:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990303+0000", 2024-09-18T00:20:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038096954000000002 2024-09-18T00:20:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990403+0000", 2024-09-18T00:20:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9486000000000003e-05 2024-09-18T00:20:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990773+0000", 2024-09-18T00:20:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037037799999999998 2024-09-18T00:20:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002316+0000", 2024-09-18T00:20:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011542581 2024-09-18T00:20:13.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002407+0000", 2024-09-18T00:20:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0968e-05 2024-09-18T00:20:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002442+0000", 2024-09-18T00:20:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4962000000000002e-05 2024-09-18T00:20:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002466+0000", 2024-09-18T00:20:13.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3972e-05 2024-09-18T00:20:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18691 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:20:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.900064365000006, 2024-09-18T00:20:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051495530999999997, 2024-09-18T00:20:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.923 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18691 2024-09-18T00:20:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:20:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:20:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952228+0000", 2024-09-18T00:20:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:20:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952234+0000", 2024-09-18T00:20:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0310000000000004e-06 2024-09-18T00:20:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952248+0000", 2024-09-18T00:20:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4577e-05 2024-09-18T00:20:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952260+0000", 2024-09-18T00:20:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1659e-05 2024-09-18T00:20:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.000989+0000", 2024-09-18T00:20:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048729445000000003 2024-09-18T00:20:13.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001076+0000", 2024-09-18T00:20:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6323999999999995e-05 2024-09-18T00:20:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001295+0000", 2024-09-18T00:20:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021944299999999999 2024-09-18T00:20:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.927 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003502+0000", 2024-09-18T00:20:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022065209999999999 2024-09-18T00:20:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003581+0000", 2024-09-18T00:20:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9404000000000001e-05 2024-09-18T00:20:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003652+0000", 2024-09-18T00:20:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0506999999999994e-05 2024-09-18T00:20:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.928 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003722+0000", 2024-09-18T00:20:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0300000000000001e-05 2024-09-18T00:20:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18692 3.7 3:e69124d0:::rbd_data.112a6efb7564.0000000000000001:f [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:20:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.900016304000005, 2024-09-18T00:20:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050418114999999999, 2024-09-18T00:20:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.929 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18692 2024-09-18T00:20:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:20:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.930 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:20:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952275+0000", 2024-09-18T00:20:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4e-07 2024-09-18T00:20:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952281+0000", 2024-09-18T00:20:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2000000000000002e-06 2024-09-18T00:20:13.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952295+0000", 2024-09-18T00:20:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4731e-05 2024-09-18T00:20:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952307+0000", 2024-09-18T00:20:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1877e-05 2024-09-18T00:20:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002506+0000", 2024-09-18T00:20:13.932 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050199230999999997 2024-09-18T00:20:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002574+0000", 2024-09-18T00:20:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8076000000000003e-05 2024-09-18T00:20:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002693+0000", 2024-09-18T00:20:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011806 2024-09-18T00:20:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19383 3.5 3:a1859bb0:::rbd_data.112a7fe29ab4.000000000000015b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:20:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 80.230004707000006, 2024-09-18T00:20:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056760192000000001, 2024-09-18T00:20:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19383 2024-09-18T00:20:13.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:20:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:20:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.935 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622287+0000", 2024-09-18T00:20:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-18T00:20:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622292+0000", 2024-09-18T00:20:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6110000000000004e-06 2024-09-18T00:20:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622304+0000", 2024-09-18T00:20:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2106000000000001e-05 2024-09-18T00:20:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.936 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622309+0000", 2024-09-18T00:20:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.818e-06 2024-09-18T00:20:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622360+0000", 2024-09-18T00:20:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0430999999999999e-05 2024-09-18T00:20:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622438+0000", 2024-09-18T00:20:13.937 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8362999999999995e-05 2024-09-18T00:20:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678811+0000", 2024-09-18T00:20:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056372894 2024-09-18T00:20:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678906+0000", 2024-09-18T00:20:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5078e-05 2024-09-18T00:20:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.938 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.679046+0000", 2024-09-18T00:20:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139987 2024-09-18T00:20:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22468 3.5 3:a250f1ef:::rbd_data.112a948d5956.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:20:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:20:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 75.080923300999999, 2024-09-18T00:20:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.055599787999999997, 2024-09-18T00:20:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.939 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22468 2024-09-18T00:20:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:20:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:20:13.940 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771370+0000", 2024-09-18T00:20:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9629999999999998e-06 2024-09-18T00:20:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771385+0000", 2024-09-18T00:20:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5053e-05 2024-09-18T00:20:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771429+0000", 2024-09-18T00:20:13.941 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3192999999999998e-05 2024-09-18T00:20:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771453+0000", 2024-09-18T00:20:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3995999999999999e-05 2024-09-18T00:20:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771623+0000", 2024-09-18T00:20:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000170846 2024-09-18T00:20:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.942 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771838+0000", 2024-09-18T00:20:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021416600000000001 2024-09-18T00:20:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826866+0000", 2024-09-18T00:20:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.05502866 2024-09-18T00:20:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826906+0000", 2024-09-18T00:20:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9805999999999998e-05 2024-09-18T00:20:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.943 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826967+0000", 2024-09-18T00:20:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1105000000000005e-05 2024-09-18T00:20:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22470 3.2 3:474595b6:::rbd_data.112a948d5956.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:20:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:20:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 75.024732317000002, 2024-09-18T00:20:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088321282000000001, 2024-09-18T00:20:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.944 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22470 2024-09-18T00:20:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:20:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.945 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:20:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827560+0000", 2024-09-18T00:20:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.762e-06 2024-09-18T00:20:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827569+0000", 2024-09-18T00:20:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8740000000000001e-06 2024-09-18T00:20:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.946 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827585+0000", 2024-09-18T00:20:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5962000000000001e-05 2024-09-18T00:20:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827595+0000", 2024-09-18T00:20:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5910000000000002e-06 2024-09-18T00:20:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827811+0000", 2024-09-18T00:20:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000216858 2024-09-18T00:20:13.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:20:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827818+0000", 2024-09-18T00:20:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.99e-06 2024-09-18T00:20:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829799+0000", 2024-09-18T00:20:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019803849999999999 2024-09-18T00:20:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829884+0000", 2024-09-18T00:20:13.948 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4852999999999995e-05 2024-09-18T00:20:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915616+0000", 2024-09-18T00:20:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085732808999999993 2024-09-18T00:20:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915711+0000", 2024-09-18T00:20:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4578000000000001e-05 2024-09-18T00:20:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.949 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915880+0000", 2024-09-18T00:20:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016861999999999999 2024-09-18T00:20:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:54153 3.b 3:d3f7b2f3:::rbd_data.112a859a01c9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,zero 4596947~21151] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:20:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:20:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.999519581000001, 2024-09-18T00:20:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062612886000000006, 2024-09-18T00:20:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.950 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 54153 2024-09-18T00:20:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:20:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:20:13.951 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852773+0000", 2024-09-18T00:20:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.426e-06 2024-09-18T00:20:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855337+0000", 2024-09-18T00:20:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025647830000000002 2024-09-18T00:20:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.952 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855369+0000", 2024-09-18T00:20:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1300999999999997e-05 2024-09-18T00:20:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855387+0000", 2024-09-18T00:20:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7949999999999999e-05 2024-09-18T00:20:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855487+0000", 2024-09-18T00:20:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010062 2024-09-18T00:20:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855686+0000", 2024-09-18T00:20:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019901399999999999 2024-09-18T00:20:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858037+0000", 2024-09-18T00:20:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002351264 2024-09-18T00:20:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.896302+0000", 2024-09-18T00:20:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038264864000000003 2024-09-18T00:20:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915250+0000", 2024-09-18T00:20:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018947538999999999 2024-09-18T00:20:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915318+0000", 2024-09-18T00:20:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7838999999999999e-05 2024-09-18T00:20:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915384+0000", 2024-09-18T00:20:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6285999999999995e-05 2024-09-18T00:20:13.955 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:55488 3.18 3:1d5573d8:::rbd_data.112a859a01c9.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T00:20:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:20:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 42.266689372000002, 2024-09-18T00:20:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045116205999999999, 2024-09-18T00:20:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.956 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 55488 2024-09-18T00:20:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:20:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:20:13.957 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585602+0000", 2024-09-18T00:20:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4000000000000001e-07 2024-09-18T00:20:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585607+0000", 2024-09-18T00:20:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5240000000000002e-06 2024-09-18T00:20:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585618+0000", 2024-09-18T00:20:13.958 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1493e-05 2024-09-18T00:20:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585624+0000", 2024-09-18T00:20:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7180000000000001e-06 2024-09-18T00:20:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585672+0000", 2024-09-18T00:20:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8158000000000002e-05 2024-09-18T00:20:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.959 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585780+0000", 2024-09-18T00:20:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010781399999999999 2024-09-18T00:20:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.588202+0000", 2024-09-18T00:20:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002422156 2024-09-18T00:20:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630375+0000", 2024-09-18T00:20:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042172820999999999 2024-09-18T00:20:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630426+0000", 2024-09-18T00:20:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1369000000000002e-05 2024-09-18T00:20:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630581+0000", 2024-09-18T00:20:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000155041 2024-09-18T00:20:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630718+0000", 2024-09-18T00:20:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000136372 2024-09-18T00:20:13.961 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:62587 3.8 3:1527150c:::rbd_data.112a13ec2d6c.0000000000000003:head [delete] snapc b=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-18T00:20:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:07.612383+0000", 2024-09-18T00:20:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 6.2399080280000003, 2024-09-18T00:20:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.88651563099999997, 2024-09-18T00:20:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.962 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 62587 2024-09-18T00:20:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612383+0000", 2024-09-18T00:20:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612383+0000", 2024-09-18T00:20:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.963 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612385+0000", 2024-09-18T00:20:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9489999999999999e-06 2024-09-18T00:20:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612395+0000", 2024-09-18T00:20:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9340000000000001e-06 2024-09-18T00:20:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612417+0000", 2024-09-18T00:20:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2633000000000002e-05 2024-09-18T00:20:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612430+0000", 2024-09-18T00:20:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2639e-05 2024-09-18T00:20:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612529+0000", 2024-09-18T00:20:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9133000000000001e-05 2024-09-18T00:20:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-18T00:20:13.965 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612543+0000", 2024-09-18T00:20:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.412e-05 2024-09-18T00:20:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.490391+0000", 2024-09-18T00:20:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.87784777000000003 2024-09-18T00:20:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.966 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.490724+0000", 2024-09-18T00:20:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033358900000000002 2024-09-18T00:20:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-18T00:20:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.491039+0000", 2024-09-18T00:20:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000314548 2024-09-18T00:20:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.493889+0000", 2024-09-18T00:20:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028495339999999999 2024-09-18T00:20:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.967 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.498625+0000", 2024-09-18T00:20:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047369689999999997 2024-09-18T00:20:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.498794+0000", 2024-09-18T00:20:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000168006 2024-09-18T00:20:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.498898+0000", 2024-09-18T00:20:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104807 2024-09-18T00:20:13.968 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:62673 3.8 3:10f5bea1:::rbd_object_map.112a7b1589a:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-18T00:20:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:09.519309+0000", 2024-09-18T00:20:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 4.3329813599999998, 2024-09-18T00:20:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.995633447, 2024-09-18T00:20:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:13.969 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 62673 2024-09-18T00:20:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:09.519309+0000", 2024-09-18T00:20:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:09.519309+0000", 2024-09-18T00:20:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.970 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:09.519311+0000", 2024-09-18T00:20:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.689e-06 2024-09-18T00:20:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:09.519319+0000", 2024-09-18T00:20:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3510000000000002e-06 2024-09-18T00:20:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:09.519336+0000", 2024-09-18T00:20:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7053000000000001e-05 2024-09-18T00:20:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.971 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:09.519345+0000", 2024-09-18T00:20:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8890000000000008e-06 2024-09-18T00:20:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:09.519420+0000", 2024-09-18T00:20:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4296999999999997e-05 2024-09-18T00:20:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-18T00:20:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:09.519430+0000", 2024-09-18T00:20:13.972 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0284e-05 2024-09-18T00:20:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:10.513433+0000", 2024-09-18T00:20:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.99400258900000005 2024-09-18T00:20:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:10.513613+0000", 2024-09-18T00:20:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000180736 2024-09-18T00:20:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:13.973 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:10.514801+0000", 2024-09-18T00:20:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011874629999999999 2024-09-18T00:20:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:10.514859+0000", 2024-09-18T00:20:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8669999999999999e-05 2024-09-18T00:20:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:10.514943+0000", 2024-09-18T00:20:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3425999999999997e-05 2024-09-18T00:20:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.974 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.975 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:13.975 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:13.975 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:13.976 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:20:13.977 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:13.972+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:14.074 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:20:14.077 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:14.072+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:14.099 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:20:14.100 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:20:14.107 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:14.108 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:14.108 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:20:14.108 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:14.125 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:20:14.134 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:20:14.135 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:20:14.167 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:20:14.167 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:20:14.178 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:20:14.176+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:20:14.193 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:20:14.194 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:20:14.223 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:20:14.223 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:20:14.242 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:14.242 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:14.251 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:20:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:20:14.256 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:20:14.257 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:20:14.258 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:20:14.302 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:14.300+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:14.379 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:14.376+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:14.423 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:14.423 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:14.438 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:20:14.480 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:14.476+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:14.566 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:14.567 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:14.581 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:14.576+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:14.583 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:20:14.684 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:14.680+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:14.706 INFO:teuthology.orchestra.run.smithi097.stdout: marking collection for removal 2024-09-18T00:20:14.707 INFO:teuthology.orchestra.run.smithi097.stdout:setting '_remove' omap key 2024-09-18T00:20:14.707 INFO:teuthology.orchestra.run.smithi097.stdout:finish_remove_pgs 3.6_head removing 3.6 2024-09-18T00:20:14.709 INFO:teuthology.orchestra.run.smithi097.stdout:Remove successful 2024-09-18T00:20:14.761 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:14.761 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:14.775 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:20:14.822 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:14.820+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:14.884 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:14.880+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:14.898 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:14.898 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:14.913 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:20:14.998 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:14.996+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:15.079 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:15.080 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:15.086 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:15.084+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:15.093 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:20:15.187 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:15.184+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:15.237 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.6.1 2024-09-18T00:20:15.259 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:15.260 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:15.273 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:20:15.288 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:15.284+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:15.390 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:20:15.388+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:20:15.435 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:15.436 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:15.450 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:20:15.494 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:15.492+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:15.563 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T00:20:15.575 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:15.576 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:15.591 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:20:15.591 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:15.588+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:15.692 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:15.688+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:15.765 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:15.765 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:15.779 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:20:15.793 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:20:15.788+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:20:15.894 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:15.892+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:15.945 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:15.945 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:15.959 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T00:20:15.995 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:15.992+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:16.088 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:16.088 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:16.095 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:16.092+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:16.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T00:20:16.196 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:20:16.192+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:20:16.224 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:16.224 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:16.237 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T00:20:16.258 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T00:20:16.297 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:20:16.292+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:20:16.398 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:20:16.396+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:20:16.401 INFO:teuthology.orchestra.run.smithi097.stdout:ERROR: (22) Invalid argument 2024-09-18T00:20:16.402 INFO:teuthology.orchestra.run.smithi097.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T00:20:16.417 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T00:20:16.422 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0: {} 2024-09-18T00:20:16.422 INFO:teuthology.orchestra.run.smithi097.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T00:20:16.422 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-18T00:20:16.439 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2: {} 2024-09-18T00:20:16.440 INFO:teuthology.orchestra.run.smithi097.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T00:20:16.457 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3: {} 2024-09-18T00:20:16.457 INFO:teuthology.orchestra.run.smithi097.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T00:20:16.477 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4: {} 2024-09-18T00:20:16.478 INFO:teuthology.orchestra.run.smithi097.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T00:20:16.497 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5: {} 2024-09-18T00:20:16.498 INFO:teuthology.orchestra.run.smithi097.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T00:20:16.499 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:16.496+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:16.518 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6: {} 2024-09-18T00:20:16.518 INFO:teuthology.orchestra.run.smithi097.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T00:20:16.532 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:16.532 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:16.532 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:20:16.532 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:16.539 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7: {} 2024-09-18T00:20:16.539 INFO:teuthology.orchestra.run.smithi097.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T00:20:16.543 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-18T00:20:16.544 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-18T00:20:16.547 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T00:20:16.557 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is set 2024-09-18T00:20:16.600 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-18T00:20:16.596+0000 7f18c58d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26526) UID: 0 2024-09-18T00:20:16.689 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:16.690 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:16.690 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:20:16.690 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:20:16.690 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:16.700 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:16.696+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:16.705 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T00:20:16.801 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-18T00:20:16.796+0000 7f73d767f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32328) UID: 0 2024-09-18T00:20:16.825 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:20:16.825 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:20:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:20:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:5722 3.4 e41/23 3:20381d86:::rbd_data.112a3a39fc45.000000000000001d:head v 41'107, mlcod=40'105)", 2024-09-18T00:20:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:20:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.48619002, 2024-09-18T00:20:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01185172, 2024-09-18T00:20:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.826 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:20:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335496+0000", 2024-09-18T00:20:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335497+0000", 2024-09-18T00:20:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.505e-06 2024-09-18T00:20:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.827 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335945+0000", 2024-09-18T00:20:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000447687 2024-09-18T00:20:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335961+0000", 2024-09-18T00:20:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6430999999999999e-05 2024-09-18T00:20:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.335969+0000", 2024-09-18T00:20:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:20:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.828 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336030+0000", 2024-09-18T00:20:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1255999999999998e-05 2024-09-18T00:20:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.336061+0000", 2024-09-18T00:20:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0579e-05 2024-09-18T00:20:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347296+0000", 2024-09-18T00:20:16.829 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011235663 2024-09-18T00:20:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.347347+0000", 2024-09-18T00:20:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0914000000000001e-05 2024-09-18T00:20:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11457 3.0 3:0214e7b3:::rbd_data.112ab1fa102a.0000000000000150:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:20:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:20:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 101.561824082, 2024-09-18T00:20:16.830 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.028499165, 2024-09-18T00:20:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11457 2024-09-18T00:20:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:20:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259862+0000", 2024-09-18T00:20:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259865+0000", 2024-09-18T00:20:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7230000000000002e-06 2024-09-18T00:20:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259883+0000", 2024-09-18T00:20:16.832 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8067000000000002e-05 2024-09-18T00:20:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259924+0000", 2024-09-18T00:20:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.0550999999999999e-05 2024-09-18T00:20:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.259943+0000", 2024-09-18T00:20:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:20:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260054+0000", 2024-09-18T00:20:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011075 2024-09-18T00:20:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.260138+0000", 2024-09-18T00:20:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4549000000000007e-05 2024-09-18T00:20:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.287926+0000", 2024-09-18T00:20:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027787642000000001 2024-09-18T00:20:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.834 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288090+0000", 2024-09-18T00:20:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016402499999999999 2024-09-18T00:20:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.288361+0000", 2024-09-18T00:20:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00027097800000000001 2024-09-18T00:20:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18661 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:16.835 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:20:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.887056216999994, 2024-09-18T00:20:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.020008037999999999, 2024-09-18T00:20:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18661 2024-09-18T00:20:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.836 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:20:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934629+0000", 2024-09-18T00:20:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.934633+0000", 2024-09-18T00:20:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8e-06 2024-09-18T00:20:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.837 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935511+0000", 2024-09-18T00:20:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00087785700000000001 2024-09-18T00:20:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935561+0000", 2024-09-18T00:20:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9472000000000003e-05 2024-09-18T00:20:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935586+0000", 2024-09-18T00:20:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5616e-05 2024-09-18T00:20:16.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935838+0000", 2024-09-18T00:20:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025202699999999999 2024-09-18T00:20:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.936023+0000", 2024-09-18T00:20:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018480300000000001 2024-09-18T00:20:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:16.839 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953545+0000", 2024-09-18T00:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.017522157 2024-09-18T00:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.953723+0000", 2024-09-18T00:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017753800000000001 2024-09-18T00:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954637+0000", 2024-09-18T00:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00091476800000000005 2024-09-18T00:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.840 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18662 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.886051280000004, 2024-09-18T00:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.019388843999999999, 2024-09-18T00:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.841 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18662 2024-09-18T00:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935634+0000", 2024-09-18T00:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.842 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935636+0000", 2024-09-18T00:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.663e-06 2024-09-18T00:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935646+0000", 2024-09-18T00:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6700000000000006e-06 2024-09-18T00:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935672+0000", 2024-09-18T00:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5868999999999999e-05 2024-09-18T00:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.843 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.935692+0000", 2024-09-18T00:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0604000000000001e-05 2024-09-18T00:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950738+0000", 2024-09-18T00:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.015045868 2024-09-18T00:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950992+0000", 2024-09-18T00:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025368400000000001 2024-09-18T00:20:16.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954674+0000", 2024-09-18T00:20:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.003682154 2024-09-18T00:20:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.954806+0000", 2024-09-18T00:20:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013252000000000001 2024-09-18T00:20:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.955023+0000", 2024-09-18T00:20:16.845 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021681200000000001 2024-09-18T00:20:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19382 3.4 3:22f09d3a:::rbd_data.112a7fe29ab4.000000000000015a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:20:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.199641229999997, 2024-09-18T00:20:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075119302999999998, 2024-09-18T00:20:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:16.846 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19382 2024-09-18T00:20:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:20:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622044+0000", 2024-09-18T00:20:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622045+0000", 2024-09-18T00:20:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0020000000000001e-06 2024-09-18T00:20:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622053+0000", 2024-09-18T00:20:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2679999999999997e-06 2024-09-18T00:20:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622062+0000", 2024-09-18T00:20:16.848 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3749999999999992e-06 2024-09-18T00:20:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622068+0000", 2024-09-18T00:20:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.5729999999999998e-06 2024-09-18T00:20:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622118+0000", 2024-09-18T00:20:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0696999999999999e-05 2024-09-18T00:20:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.849 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622222+0000", 2024-09-18T00:20:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000103286 2024-09-18T00:20:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697061+0000", 2024-09-18T00:20:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.074839141999999997 2024-09-18T00:20:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697102+0000", 2024-09-18T00:20:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1513e-05 2024-09-18T00:20:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.850 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.697164+0000", 2024-09-18T00:20:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1446999999999996e-05 2024-09-18T00:20:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20246 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:20:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 82.665953037999998, 2024-09-18T00:20:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.01018784, 2024-09-18T00:20:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.851 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20246 2024-09-18T00:20:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:20:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.852 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:20:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155733+0000", 2024-09-18T00:20:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2599999999999998e-07 2024-09-18T00:20:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155735+0000", 2024-09-18T00:20:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2699999999999999e-06 2024-09-18T00:20:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.853 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155744+0000", 2024-09-18T00:20:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8629999999999997e-06 2024-09-18T00:20:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155749+0000", 2024-09-18T00:20:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0359999999999997e-06 2024-09-18T00:20:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165748+0000", 2024-09-18T00:20:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099991449999999992 2024-09-18T00:20:16.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165819+0000", 2024-09-18T00:20:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0964999999999996e-05 2024-09-18T00:20:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165920+0000", 2024-09-18T00:20:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000101135 2024-09-18T00:20:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20247 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:20:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 82.665929696000006, 2024-09-18T00:20:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010443987, 2024-09-18T00:20:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20247 2024-09-18T00:20:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.856 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:20:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:20:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155756+0000", 2024-09-18T00:20:16.857 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1899999999999998e-07 2024-09-18T00:20:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155760+0000", 2024-09-18T00:20:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6459999999999999e-06 2024-09-18T00:20:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155767+0000", 2024-09-18T00:20:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0350000000000004e-06 2024-09-18T00:20:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.858 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155774+0000", 2024-09-18T00:20:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5169999999999996e-06 2024-09-18T00:20:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.165960+0000", 2024-09-18T00:20:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186051999999999 2024-09-18T00:20:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166048+0000", 2024-09-18T00:20:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8848e-05 2024-09-18T00:20:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.859 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166200+0000", 2024-09-18T00:20:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015147000000000001 2024-09-18T00:20:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20248 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:20:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 82.665904531999999, 2024-09-18T00:20:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010617721, 2024-09-18T00:20:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.860 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20248 2024-09-18T00:20:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:20:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.861 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155781+0000", 2024-09-18T00:20:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155782+0000", 2024-09-18T00:20:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.84e-07 2024-09-18T00:20:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155784+0000", 2024-09-18T00:20:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2800000000000002e-06 2024-09-18T00:20:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.862 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155790+0000", 2024-09-18T00:20:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.5679999999999996e-06 2024-09-18T00:20:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155795+0000", 2024-09-18T00:20:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6929999999999998e-06 2024-09-18T00:20:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.863 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166245+0000", 2024-09-18T00:20:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010449847999999999 2024-09-18T00:20:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166307+0000", 2024-09-18T00:20:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2287999999999997e-05 2024-09-18T00:20:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166399+0000", 2024-09-18T00:20:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.166e-05 2024-09-18T00:20:16.864 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:20249 3.0 3:00029ba9:::rbd_data.112a5ba20d45.0000000000000006:10 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:20:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 82.665884156000004, 2024-09-18T00:20:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010811983000000001, 2024-09-18T00:20:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.865 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 20249 2024-09-18T00:20:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:20:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155801+0000", 2024-09-18T00:20:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.866 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155802+0000", 2024-09-18T00:20:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.1699999999999999e-07 2024-09-18T00:20:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155804+0000", 2024-09-18T00:20:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1670000000000002e-06 2024-09-18T00:20:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155810+0000", 2024-09-18T00:20:16.867 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0580000000000003e-06 2024-09-18T00:20:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.155817+0000", 2024-09-18T00:20:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3980000000000004e-06 2024-09-18T00:20:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166451+0000", 2024-09-18T00:20:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010634825000000001 2024-09-18T00:20:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.868 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166520+0000", 2024-09-18T00:20:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8317000000000003e-05 2024-09-18T00:20:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:54.166613+0000", 2024-09-18T00:20:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.3801000000000005e-05 2024-09-18T00:20:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:21913 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:20:16.869 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:20:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 79.528039010000001, 2024-09-18T00:20:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090508916999999994, 2024-09-18T00:20:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 21913 2024-09-18T00:20:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.870 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:20:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293647+0000", 2024-09-18T00:20:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293648+0000", 2024-09-18T00:20:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6400000000000001e-07 2024-09-18T00:20:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.871 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293653+0000", 2024-09-18T00:20:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3820000000000003e-06 2024-09-18T00:20:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293665+0000", 2024-09-18T00:20:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1778e-05 2024-09-18T00:20:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293673+0000", 2024-09-18T00:20:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4130000000000006e-06 2024-09-18T00:20:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.872 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293719+0000", 2024-09-18T00:20:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5534999999999997e-05 2024-09-18T00:20:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.293773+0000", 2024-09-18T00:20:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.4389999999999999e-05 2024-09-18T00:20:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384053+0000", 2024-09-18T00:20:16.873 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.090279617000000006 2024-09-18T00:20:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384088+0000", 2024-09-18T00:20:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5466000000000002e-05 2024-09-18T00:20:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:57.384156+0000", 2024-09-18T00:20:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7471999999999994e-05 2024-09-18T00:20:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22274 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:20:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:20:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.716922789999998, 2024-09-18T00:20:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.069820742000000005, 2024-09-18T00:20:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22274 2024-09-18T00:20:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.875 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:20:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104763+0000", 2024-09-18T00:20:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104766+0000", 2024-09-18T00:20:16.876 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3859999999999999e-06 2024-09-18T00:20:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104783+0000", 2024-09-18T00:20:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6697e-05 2024-09-18T00:20:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104816+0000", 2024-09-18T00:20:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2786000000000001e-05 2024-09-18T00:20:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.877 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104833+0000", 2024-09-18T00:20:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7125999999999998e-05 2024-09-18T00:20:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.104965+0000", 2024-09-18T00:20:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013215899999999999 2024-09-18T00:20:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.105103+0000", 2024-09-18T00:20:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000138193 2024-09-18T00:20:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.878 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174496+0000", 2024-09-18T00:20:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.06939244 2024-09-18T00:20:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174530+0000", 2024-09-18T00:20:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4601999999999998e-05 2024-09-18T00:20:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.174584+0000", 2024-09-18T00:20:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3353000000000002e-05 2024-09-18T00:20:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.879 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22485 3.4 3:223db924:::rbd_data.112a948d5956.0000000000000007:head [write 16521958~30847] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:20:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:20:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.868133779999994, 2024-09-18T00:20:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.012574648000000001, 2024-09-18T00:20:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.880 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22485 2024-09-18T00:20:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:20:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.881 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953552+0000", 2024-09-18T00:20:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.953554+0000", 2024-09-18T00:20:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1150000000000001e-06 2024-09-18T00:20:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.882 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962036+0000", 2024-09-18T00:20:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0084817599999999996 2024-09-18T00:20:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962062+0000", 2024-09-18T00:20:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6499000000000002e-05 2024-09-18T00:20:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962074+0000", 2024-09-18T00:20:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2095e-05 2024-09-18T00:20:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.883 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962165+0000", 2024-09-18T00:20:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0703000000000002e-05 2024-09-18T00:20:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.962276+0000", 2024-09-18T00:20:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000111406 2024-09-18T00:20:16.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965925+0000", 2024-09-18T00:20:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0036488919999999999 2024-09-18T00:20:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.965992+0000", 2024-09-18T00:20:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6806999999999999e-05 2024-09-18T00:20:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.966127+0000", 2024-09-18T00:20:16.885 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013437099999999999 2024-09-18T00:20:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22681 3.4 3:22e76021:::rbd_data.112a948d5956.0000000000000001:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:20:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:20:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 77.534134475000002, 2024-09-18T00:20:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076591189000000004, 2024-09-18T00:20:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.886 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22681 2024-09-18T00:20:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:20:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287551+0000", 2024-09-18T00:20:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287554+0000", 2024-09-18T00:20:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.514e-06 2024-09-18T00:20:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287567+0000", 2024-09-18T00:20:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.308e-05 2024-09-18T00:20:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.888 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287597+0000", 2024-09-18T00:20:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0406e-05 2024-09-18T00:20:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287626+0000", 2024-09-18T00:20:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9306e-05 2024-09-18T00:20:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.287801+0000", 2024-09-18T00:20:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017411400000000001 2024-09-18T00:20:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.889 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.288053+0000", 2024-09-18T00:20:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025223500000000001 2024-09-18T00:20:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364016+0000", 2024-09-18T00:20:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.075963301999999996 2024-09-18T00:20:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364064+0000", 2024-09-18T00:20:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.7886000000000002e-05 2024-09-18T00:20:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:59.364142+0000", 2024-09-18T00:20:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8345999999999998e-05 2024-09-18T00:20:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:24700 3.4 3:23a8f5d4:::rbd_data.112a990280ef.0000000000000016:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:20:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:20:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 69.883668495999999, 2024-09-18T00:20:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091975649000000007, 2024-09-18T00:20:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.891 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 24700 2024-09-18T00:20:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:20:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.892 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938017+0000", 2024-09-18T00:20:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938019+0000", 2024-09-18T00:20:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2189999999999998e-06 2024-09-18T00:20:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938030+0000", 2024-09-18T00:20:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0278000000000001e-05 2024-09-18T00:20:16.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938053+0000", 2024-09-18T00:20:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2946999999999999e-05 2024-09-18T00:20:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938062+0000", 2024-09-18T00:20:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.8509999999999994e-06 2024-09-18T00:20:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938140+0000", 2024-09-18T00:20:16.894 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7664999999999996e-05 2024-09-18T00:20:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:06.938314+0000", 2024-09-18T00:20:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00017398500000000001 2024-09-18T00:20:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029896+0000", 2024-09-18T00:20:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091582078999999997 2024-09-18T00:20:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:16.895 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029930+0000", 2024-09-18T00:20:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3723999999999998e-05 2024-09-18T00:20:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:07.029993+0000", 2024-09-18T00:20:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.2901e-05 2024-09-18T00:20:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37799 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:16.896 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:20:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 56.411440345000003, 2024-09-18T00:20:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010186016000000001, 2024-09-18T00:20:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37799 2024-09-18T00:20:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.897 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:20:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410245+0000", 2024-09-18T00:20:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410246+0000", 2024-09-18T00:20:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.35e-07 2024-09-18T00:20:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.898 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410247+0000", 2024-09-18T00:20:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0389999999999999e-06 2024-09-18T00:20:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410250+0000", 2024-09-18T00:20:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3380000000000001e-06 2024-09-18T00:20:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410252+0000", 2024-09-18T00:20:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5150000000000001e-06 2024-09-18T00:20:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.899 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420198+0000", 2024-09-18T00:20:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0099454360000000002 2024-09-18T00:20:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420284+0000", 2024-09-18T00:20:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5638999999999998e-05 2024-09-18T00:20:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420431+0000", 2024-09-18T00:20:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014781399999999999 2024-09-18T00:20:16.900 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37800 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:20:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 56.411430068999998, 2024-09-18T00:20:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010452559, 2024-09-18T00:20:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.901 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37800 2024-09-18T00:20:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:20:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:20:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410256+0000", 2024-09-18T00:20:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2499999999999999e-07 2024-09-18T00:20:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410257+0000", 2024-09-18T00:20:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0750000000000001e-06 2024-09-18T00:20:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410260+0000", 2024-09-18T00:20:16.903 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.428e-06 2024-09-18T00:20:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410263+0000", 2024-09-18T00:20:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5339999999999998e-06 2024-09-18T00:20:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420467+0000", 2024-09-18T00:20:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010204615 2024-09-18T00:20:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.904 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420557+0000", 2024-09-18T00:20:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.9183000000000003e-05 2024-09-18T00:20:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420708+0000", 2024-09-18T00:20:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00015149900000000001 2024-09-18T00:20:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37801 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:20:16.905 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 56.411419731999999, 2024-09-18T00:20:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010623357, 2024-09-18T00:20:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37801 2024-09-18T00:20:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:20:16.906 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:20:16.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410266+0000", 2024-09-18T00:20:16.907 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.23e-07 2024-09-18T00:20:16.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.907 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.907 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.907 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410271+0000", 2024-09-18T00:20:16.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8040000000000001e-06 2024-09-18T00:20:16.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410274+0000", 2024-09-18T00:20:16.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5310000000000002e-06 2024-09-18T00:20:16.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.908 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410277+0000", 2024-09-18T00:20:16.908 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.605e-06 2024-09-18T00:20:16.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.908 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.908 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420743+0000", 2024-09-18T00:20:16.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010465487000000001 2024-09-18T00:20:16.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420811+0000", 2024-09-18T00:20:16.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8845999999999998e-05 2024-09-18T00:20:16.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.909 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.909 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.909 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420889+0000", 2024-09-18T00:20:16.909 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7861000000000006e-05 2024-09-18T00:20:16.909 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.909 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.910 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.910 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.910 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:37802 3.4 3:21014da1:::rbd_data.112a3496573c.0000000000000005:17 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:16.910 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:20:16.910 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 56.411405105999997, 2024-09-18T00:20:16.910 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010745463, 2024-09-18T00:20:16.910 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.910 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.910 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:16.910 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.910 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.910 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.911 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 37802 2024-09-18T00:20:16.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.911 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:20:16.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.911 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.911 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:20:16.911 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.911 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410281+0000", 2024-09-18T00:20:16.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6199999999999999e-07 2024-09-18T00:20:16.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410282+0000", 2024-09-18T00:20:16.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.085e-06 2024-09-18T00:20:16.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.912 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.912 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.912 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410286+0000", 2024-09-18T00:20:16.912 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.6610000000000002e-06 2024-09-18T00:20:16.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.410288+0000", 2024-09-18T00:20:16.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.6410000000000002e-06 2024-09-18T00:20:16.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420910+0000", 2024-09-18T00:20:16.913 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010621479 2024-09-18T00:20:16.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.913 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.913 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.913 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.420952+0000", 2024-09-18T00:20:16.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2497999999999999e-05 2024-09-18T00:20:16.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.914 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.914 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:20.421026+0000", 2024-09-18T00:20:16.914 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.3837000000000005e-05 2024-09-18T00:20:16.914 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.914 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.914 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.914 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.914 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:59273 3.a 3:535385f0:::rbd_children:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-18T00:20:16.914 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:43.695268+0000", 2024-09-18T00:20:16.914 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 33.126417492000002, 2024-09-18T00:20:16.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.89624285599999998, 2024-09-18T00:20:16.915 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.915 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.915 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:16.915 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:16.915 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:16.915 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:16.915 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 59273 2024-09-18T00:20:16.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.915 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.915 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.915 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.915 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695268+0000", 2024-09-18T00:20:16.915 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695268+0000", 2024-09-18T00:20:16.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.916 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695270+0000", 2024-09-18T00:20:16.916 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1440000000000001e-06 2024-09-18T00:20:16.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.916 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.916 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695278+0000", 2024-09-18T00:20:16.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8839999999999994e-06 2024-09-18T00:20:16.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695296+0000", 2024-09-18T00:20:16.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7957000000000001e-05 2024-09-18T00:20:16.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.917 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.917 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.917 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695304+0000", 2024-09-18T00:20:16.917 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6850000000000001e-06 2024-09-18T00:20:16.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695372+0000", 2024-09-18T00:20:16.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8659999999999997e-05 2024-09-18T00:20:16.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-18T00:20:16.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:43.695383+0000", 2024-09-18T00:20:16.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0852e-05 2024-09-18T00:20:16.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.918 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.918 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.918 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.586073+0000", 2024-09-18T00:20:16.918 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.89068997800000005 2024-09-18T00:20:16.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.586481+0000", 2024-09-18T00:20:16.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00040750900000000002 2024-09-18T00:20:16.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 3", 2024-09-18T00:20:16.919 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.586873+0000", 2024-09-18T00:20:16.919 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00039179999999999998 2024-09-18T00:20:16.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.919 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.919 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:16.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.588815+0000", 2024-09-18T00:20:16.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019426249999999999 2024-09-18T00:20:16.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:16.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.591259+0000", 2024-09-18T00:20:16.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0024436829999999999 2024-09-18T00:20:16.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.920 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:16.920 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.591399+0000", 2024-09-18T00:20:16.920 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139927 2024-09-18T00:20:16.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.920 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.921 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.921 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:44.591511+0000", 2024-09-18T00:20:16.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011215200000000001 2024-09-18T00:20:16.921 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.921 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.921 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.921 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.921 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop(client.4394.0:59788 3.9 e106/23 3:94325cfd:::rbd_header.112aa373d2ec:head v 106'325, mlcod=105'323)", 2024-09-18T00:20:16.921 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:48.641886+0000", 2024-09-18T00:20:16.921 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 28.179799646999999, 2024-09-18T00:20:16.921 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010208398, 2024-09-18T00:20:16.921 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:16.921 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:16.922 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:16.922 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:16.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:16.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641886+0000", 2024-09-18T00:20:16.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:16.922 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641886+0000", 2024-09-18T00:20:16.922 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:16.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.922 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:16.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641887+0000", 2024-09-18T00:20:16.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0999999999999997e-07 2024-09-18T00:20:16.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:16.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641924+0000", 2024-09-18T00:20:16.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7252000000000002e-05 2024-09-18T00:20:16.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.923 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:16.923 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641955+0000", 2024-09-18T00:20:16.923 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.0372e-05 2024-09-18T00:20:16.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.923 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:16.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.641974+0000", 2024-09-18T00:20:16.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9695999999999999e-05 2024-09-18T00:20:16.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:16.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.642107+0000", 2024-09-18T00:20:16.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00013297999999999999 2024-09-18T00:20:16.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.924 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.924 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:16.924 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.642142+0000", 2024-09-18T00:20:16.924 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4993000000000002e-05 2024-09-18T00:20:16.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:16.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.652006+0000", 2024-09-18T00:20:16.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0098642109999999995 2024-09-18T00:20:16.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:16.925 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:16.925 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:16.925 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:48.652094+0000", 2024-09-18T00:20:16.925 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.7984000000000003e-05 2024-09-18T00:20:16.925 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.925 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.925 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.925 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:16.925 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:16.926 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:16.927 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T00:20:16.927 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-18T00:20:16.924+0000 7fafbb240640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26523) UID: 0 2024-09-18T00:20:16.943 INFO:teuthology.orchestra.run.smithi097.stdout:get_pg_num_history pg_num_history pg_num_history(e126 pg_nums {1={16=1},2={19=8},3={23=32}} deleted_pools ) 2024-09-18T00:20:16.944 INFO:teuthology.orchestra.run.smithi097.stdout:Importing pgid 3.6 2024-09-18T00:20:16.949 INFO:teuthology.orchestra.run.smithi097.stdout:Write #3:60e10a89:::rbd_data.112a4712e9d2.0000000000000027:head# 2024-09-18T00:20:16.949 INFO:teuthology.orchestra.run.smithi097.stdout:snapset c=[]:{} 2024-09-18T00:20:16.949 INFO:teuthology.orchestra.run.smithi097.stdout:Write #3:6104537f:::rbd_data.112a960351d8.0000000000000004:head# 2024-09-18T00:20:16.959 INFO:teuthology.orchestra.run.smithi097.stdout:snapset 0=[]:{} 2024-09-18T00:20:17.027 INFO:teuthology.orchestra.run.smithi097.stdout:Write #3:617842cf:::rbd_data.112a5ba20d45.000000000000003f:head# 2024-09-18T00:20:17.027 INFO:teuthology.orchestra.run.smithi097.stdout:snapset 10=[]:{} 2024-09-18T00:20:17.028 INFO:teuthology.orchestra.run.smithi097.stdout:Write #3:62e7f20b:::rbd_data.112a9f4259df.0000000000000001:head# 2024-09-18T00:20:17.028 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:17.024+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:17.028 INFO:teuthology.orchestra.run.smithi097.stdout:snapset 0=[]:{} 2024-09-18T00:20:17.032 INFO:teuthology.orchestra.run.smithi097.stdout:Write #3:6404e862:::rbd_data.112ab1fa102a.0000000000000004:head# 2024-09-18T00:20:17.032 INFO:teuthology.orchestra.run.smithi097.stdout:snapset 0=[]:{} 2024-09-18T00:20:17.034 INFO:teuthology.orchestra.run.smithi097.stdout:Write #3:6437f226:::rbd_data.112aa7ba4578.0000000000000145:head# 2024-09-18T00:20:17.034 INFO:teuthology.orchestra.run.smithi097.stdout:snapset 0=[]:{} 2024-09-18T00:20:17.035 INFO:teuthology.orchestra.run.smithi097.stdout:Write #3:649263c7:::rbd_data.112aa7ba4578.0000000000000140:head# 2024-09-18T00:20:17.035 INFO:teuthology.orchestra.run.smithi097.stdout:snapset 0=[]:{} 2024-09-18T00:20:17.037 INFO:teuthology.orchestra.run.smithi097.stdout:Write #3:64d23a70:::rbd_data.112a4712e9d2.0000000000000020:head# 2024-09-18T00:20:17.037 INFO:teuthology.orchestra.run.smithi097.stdout:snapset c=[]:{} 2024-09-18T00:20:17.037 INFO:teuthology.orchestra.run.smithi097.stdout:Write #3:65cb5a01:::rbd_data.112a960351d8.000000000000000f:head# 2024-09-18T00:20:17.037 INFO:teuthology.orchestra.run.smithi097.stdout:snapset 0=[]:{} 2024-09-18T00:20:17.038 INFO:teuthology.orchestra.run.smithi097.stdout:Write #3:66496aed:::rbd_object_map.112a960351d8:head# 2024-09-18T00:20:17.038 INFO:teuthology.orchestra.run.smithi097.stdout:snapset 0=[]:{} 2024-09-18T00:20:17.038 INFO:teuthology.orchestra.run.smithi097.stdout:Write #3:67857717:::rbd_data.112a9f4259df.0000000000000004:head# 2024-09-18T00:20:17.038 INFO:teuthology.orchestra.run.smithi097.stdout:snapset 0=[]:{} 2024-09-18T00:20:17.040 INFO:teuthology.orchestra.run.smithi097.stdout:write_pg epoch 126 info 3.6( v 126'321 (0'0,126'321] local-lis/les=124/125 n=11 ec=23/23 lis/c=124/124 les/c/f=125/125/0 sis=124) 2024-09-18T00:20:17.042 INFO:teuthology.orchestra.run.smithi097.stdout:Import successful 2024-09-18T00:20:17.059 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:17.059 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:17.059 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:20:17.060 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:17.074 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T00:20:17.162 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:17.160+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:17.230 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-18T00:20:17.228+0000 7f6e6d00d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26530) UID: 0 2024-09-18T00:20:17.237 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:17.237 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:17.237 INFO:teuthology.orchestra.run.smithi097.stdout: "complaint_time": 30, 2024-09-18T00:20:17.237 INFO:teuthology.orchestra.run.smithi097.stdout: "num_blocked_ops": 0 2024-09-18T00:20:17.237 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:17.252 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T00:20:17.338 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-09-18T00:20:17.336+0000 7fc688b4e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26442) UID: 0 2024-09-18T00:20:17.417 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: "size": 20, 2024-09-18T00:20:17.417 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 600, 2024-09-18T00:20:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [ 2024-09-18T00:20:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:845 3.15 3:adfa7c87:::rbd_data.112ae20746f5.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-18T00:20:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:20:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 124.611880693, 2024-09-18T00:20:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.096122131999999999, 2024-09-18T00:20:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.418 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 845 2024-09-18T00:20:17.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:20:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.420 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800167+0000", 2024-09-18T00:20:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.421 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800169+0000", 2024-09-18T00:20:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2709999999999999e-06 2024-09-18T00:20:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.422 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800180+0000", 2024-09-18T00:20:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1272e-05 2024-09-18T00:20:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800203+0000", 2024-09-18T00:20:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2904999999999998e-05 2024-09-18T00:20:17.423 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800213+0000", 2024-09-18T00:20:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0271999999999999e-05 2024-09-18T00:20:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.424 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800291+0000", 2024-09-18T00:20:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.7768999999999994e-05 2024-09-18T00:20:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.425 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.800490+0000", 2024-09-18T00:20:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000199058 2024-09-18T00:20:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:17.426 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.804342+0000", 2024-09-18T00:20:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0038514600000000001 2024-09-18T00:20:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896092+0000", 2024-09-18T00:20:17.427 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.091750565000000006 2024-09-18T00:20:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896163+0000", 2024-09-18T00:20:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0612999999999996e-05 2024-09-18T00:20:17.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896219+0000", 2024-09-18T00:20:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.6518000000000003e-05 2024-09-18T00:20:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.429 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896289+0000", 2024-09-18T00:20:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9429000000000002e-05 2024-09-18T00:20:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.430 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:845 3.15 e30/27 ondisk, result = 0)", 2024-09-18T00:20:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:20:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 124.59567282, 2024-09-18T00:20:17.431 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079927756000000003, 2024-09-18T00:20:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.432 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:20:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816375+0000", 2024-09-18T00:20:17.433 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816376+0000", 2024-09-18T00:20:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:20:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.434 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816382+0000", 2024-09-18T00:20:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.703e-06 2024-09-18T00:20:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.435 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816404+0000", 2024-09-18T00:20:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.1693999999999999e-05 2024-09-18T00:20:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.816416+0000", 2024-09-18T00:20:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2054e-05 2024-09-18T00:20:17.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896147+0000", 2024-09-18T00:20:17.437 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.079731064000000004 2024-09-18T00:20:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896157+0000", 2024-09-18T00:20:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5140000000000007e-06 2024-09-18T00:20:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.438 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:12.896302+0000", 2024-09-18T00:20:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00014551700000000001 2024-09-18T00:20:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:6753 3.7 3:e100dc1c:::rbd_data.112a3a39fc45.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:20:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:20:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.5686315, 2024-09-18T00:20:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.11020611600000001, 2024-09-18T00:20:17.439 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 6753 2024-09-18T00:20:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:20:17.440 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843416+0000", 2024-09-18T00:20:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843420+0000", 2024-09-18T00:20:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.8229999999999997e-06 2024-09-18T00:20:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.441 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843440+0000", 2024-09-18T00:20:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0789999999999999e-05 2024-09-18T00:20:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843474+0000", 2024-09-18T00:20:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3853000000000003e-05 2024-09-18T00:20:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843496+0000", 2024-09-18T00:20:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.173e-05 2024-09-18T00:20:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843578+0000", 2024-09-18T00:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.1447999999999997e-05 2024-09-18T00:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.843642+0000", 2024-09-18T00:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4492000000000005e-05 2024-09-18T00:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:17.443 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.849346+0000", 2024-09-18T00:20:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0057038740000000003 2024-09-18T00:20:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953407+0000", 2024-09-18T00:20:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.104060895 2024-09-18T00:20:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953484+0000", 2024-09-18T00:20:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6811999999999996e-05 2024-09-18T00:20:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953589+0000", 2024-09-18T00:20:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000105398 2024-09-18T00:20:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953622+0000", 2024-09-18T00:20:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.3000999999999998e-05 2024-09-18T00:20:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.445 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:6753 3.7 e42/32 ondisk, result = 0)", 2024-09-18T00:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 111.54735494800001, 2024-09-18T00:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088935705000000004, 2024-09-18T00:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.446 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:20:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864692+0000", 2024-09-18T00:20:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864693+0000", 2024-09-18T00:20:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0079999999999999e-06 2024-09-18T00:20:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.447 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864698+0000", 2024-09-18T00:20:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.9799999999999998e-06 2024-09-18T00:20:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864718+0000", 2024-09-18T00:20:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9528e-05 2024-09-18T00:20:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.448 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.864728+0000", 2024-09-18T00:20:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0175e-05 2024-09-18T00:20:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953433+0000", 2024-09-18T00:20:17.449 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088705330999999998 2024-09-18T00:20:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953440+0000", 2024-09-18T00:20:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.3060000000000003e-06 2024-09-18T00:20:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:25.953628+0000", 2024-09-18T00:20:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00018837699999999999 2024-09-18T00:20:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.451 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:11460 3.19 3:9e578b6c:::rbd_data.112ab1fa102a.0000000000000153:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T00:20:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:20:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 102.14498349599999, 2024-09-18T00:20:17.452 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041989011999999999, 2024-09-18T00:20:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 11460 2024-09-18T00:20:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:20:17.453 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267064+0000", 2024-09-18T00:20:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267067+0000", 2024-09-18T00:20:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4359999999999998e-06 2024-09-18T00:20:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.454 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267088+0000", 2024-09-18T00:20:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0908999999999999e-05 2024-09-18T00:20:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267133+0000", 2024-09-18T00:20:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5232000000000003e-05 2024-09-18T00:20:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267157+0000", 2024-09-18T00:20:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3833e-05 2024-09-18T00:20:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267300+0000", 2024-09-18T00:20:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000142282 2024-09-18T00:20:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.267447+0000", 2024-09-18T00:20:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000147851 2024-09-18T00:20:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.456 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308889+0000", 2024-09-18T00:20:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041441779999999998 2024-09-18T00:20:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.308956+0000", 2024-09-18T00:20:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7136000000000002e-05 2024-09-18T00:20:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:35.309053+0000", 2024-09-18T00:20:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.6552999999999995e-05 2024-09-18T00:20:17.457 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:16947 3.18 3:18b58c7b:::rbd_data.112a6efb7564.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:20:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:20:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.795205795000001, 2024-09-18T00:20:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.101692484, 2024-09-18T00:20:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.458 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 16947 2024-09-18T00:20:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:20:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.459 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616842+0000", 2024-09-18T00:20:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.616843+0000", 2024-09-18T00:20:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0780000000000001e-06 2024-09-18T00:20:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617785+0000", 2024-09-18T00:20:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00094204100000000002 2024-09-18T00:20:17.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617801+0000", 2024-09-18T00:20:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6436999999999999e-05 2024-09-18T00:20:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617809+0000", 2024-09-18T00:20:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9049999999999997e-06 2024-09-18T00:20:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.461 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.617904+0000", 2024-09-18T00:20:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5204999999999994e-05 2024-09-18T00:20:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.618071+0000", 2024-09-18T00:20:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016640000000000001 2024-09-18T00:20:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.624534+0000", 2024-09-18T00:20:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0064634499999999999 2024-09-18T00:20:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.462 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718299+0000", 2024-09-18T00:20:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.093764739 2024-09-18T00:20:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718375+0000", 2024-09-18T00:20:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.6532999999999994e-05 2024-09-18T00:20:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718466+0000", 2024-09-18T00:20:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0581000000000006e-05 2024-09-18T00:20:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.463 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718534+0000", 2024-09-18T00:20:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8114999999999994e-05 2024-09-18T00:20:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_repop_reply(client.4394.0:16947 3.18 e59/23 ondisk, result = 0)", 2024-09-18T00:20:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:20:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 88.769989100999993, 2024-09-18T00:20:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076488069000000006, 2024-09-18T00:20:17.464 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:20:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642058+0000", 2024-09-18T00:20:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.465 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642060+0000", 2024-09-18T00:20:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6640000000000001e-06 2024-09-18T00:20:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642071+0000", 2024-09-18T00:20:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1204e-05 2024-09-18T00:20:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642091+0000", 2024-09-18T00:20:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.0313999999999999e-05 2024-09-18T00:20:17.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.642103+0000", 2024-09-18T00:20:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1214e-05 2024-09-18T00:20:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718350+0000", 2024-09-18T00:20:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.076247681999999997 2024-09-18T00:20:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.467 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718366+0000", 2024-09-18T00:20:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5420000000000001e-05 2024-09-18T00:20:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:48.718546+0000", 2024-09-18T00:20:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000180571 2024-09-18T00:20:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18686 3.7 3:e2f288b3:::rbd_data.112a5ba20d45.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:20:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 84.463311731000005, 2024-09-18T00:20:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.041132573999999998, 2024-09-18T00:20:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18686 2024-09-18T00:20:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.469 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:20:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948736+0000", 2024-09-18T00:20:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.948737+0000", 2024-09-18T00:20:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3769999999999999e-06 2024-09-18T00:20:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949968+0000", 2024-09-18T00:20:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.001230493 2024-09-18T00:20:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.949993+0000", 2024-09-18T00:20:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.5282e-05 2024-09-18T00:20:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950011+0000", 2024-09-18T00:20:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8199999999999999e-05 2024-09-18T00:20:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950180+0000", 2024-09-18T00:20:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016867600000000001 2024-09-18T00:20:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950337+0000", 2024-09-18T00:20:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000156902 2024-09-18T00:20:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950818+0000", 2024-09-18T00:20:17.472 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00048192599999999998 2024-09-18T00:20:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978491+0000", 2024-09-18T00:20:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027672568000000002 2024-09-18T00:20:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989759+0000", 2024-09-18T00:20:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011268144000000001 2024-09-18T00:20:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.473 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989824+0000", 2024-09-18T00:20:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.4556999999999999e-05 2024-09-18T00:20:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.989868+0000", 2024-09-18T00:20:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.4449000000000001e-05 2024-09-18T00:20:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.474 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18688 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:20:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 84.461882614000004, 2024-09-18T00:20:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.040041259000000003, 2024-09-18T00:20:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.475 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18688 2024-09-18T00:20:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:20:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950165+0000", 2024-09-18T00:20:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.476 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.950166+0000", 2024-09-18T00:20:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.234e-06 2024-09-18T00:20:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951295+0000", 2024-09-18T00:20:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011293480000000001 2024-09-18T00:20:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.477 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951328+0000", 2024-09-18T00:20:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.2524999999999998e-05 2024-09-18T00:20:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951346+0000", 2024-09-18T00:20:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.8170999999999999e-05 2024-09-18T00:20:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978546+0000", 2024-09-18T00:20:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.027200418000000001 2024-09-18T00:20:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.478 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.978666+0000", 2024-09-18T00:20:17.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000119913 2024-09-18T00:20:17.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:17.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.979098+0000", 2024-09-18T00:20:17.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00043169399999999999 2024-09-18T00:20:17.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.479 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.479 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:17.479 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990000+0000", 2024-09-18T00:20:17.479 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.010901719000000001 2024-09-18T00:20:17.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:17.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990109+0000", 2024-09-18T00:20:17.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010964299999999999 2024-09-18T00:20:17.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:17.480 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990152+0000", 2024-09-18T00:20:17.480 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.2877000000000003e-05 2024-09-18T00:20:17.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.480 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.481 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990206+0000", 2024-09-18T00:20:17.481 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.3717000000000001e-05 2024-09-18T00:20:17.481 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.481 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.481 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.481 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.481 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18689 3.7 3:e4221217:::rbd_data.112a5ba20d45.0000000000000003:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:17.481 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:20:17.481 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 84.460670915999998, 2024-09-18T00:20:17.482 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051022338, 2024-09-18T00:20:17.482 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.482 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.482 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.482 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.482 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.482 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.482 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18689 2024-09-18T00:20:17.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.482 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.482 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.482 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.482 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:20:17.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951376+0000", 2024-09-18T00:20:17.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.483 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.483 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.483 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951378+0000", 2024-09-18T00:20:17.483 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5880000000000001e-06 2024-09-18T00:20:17.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951385+0000", 2024-09-18T00:20:17.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.9460000000000001e-06 2024-09-18T00:20:17.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.484 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951402+0000", 2024-09-18T00:20:17.484 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.6538999999999999e-05 2024-09-18T00:20:17.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.484 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.484 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951565+0000", 2024-09-18T00:20:17.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016307999999999999 2024-09-18T00:20:17.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988232+0000", 2024-09-18T00:20:17.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.036667656999999999 2024-09-18T00:20:17.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.485 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.485 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988318+0000", 2024-09-18T00:20:17.485 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.5348999999999999e-05 2024-09-18T00:20:17.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:17.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.988574+0000", 2024-09-18T00:20:17.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00025670800000000002 2024-09-18T00:20:17.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:17.486 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002254+0000", 2024-09-18T00:20:17.486 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.013679274 2024-09-18T00:20:17.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.486 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.486 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:17.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002326+0000", 2024-09-18T00:20:17.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.2720999999999997e-05 2024-09-18T00:20:17.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:17.487 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002364+0000", 2024-09-18T00:20:17.487 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.7314000000000002e-05 2024-09-18T00:20:17.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.487 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.487 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.488 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002399+0000", 2024-09-18T00:20:17.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.5162e-05 2024-09-18T00:20:17.488 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.488 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.488 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.488 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.488 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18690 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:17.488 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:20:17.488 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 84.460453897999997, 2024-09-18T00:20:17.488 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050872134999999999, 2024-09-18T00:20:17.489 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.489 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.489 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.489 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.489 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.489 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.489 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18690 2024-09-18T00:20:17.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.489 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.489 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.489 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.489 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:20:17.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951593+0000", 2024-09-18T00:20:17.490 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.490 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.490 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.490 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.951595+0000", 2024-09-18T00:20:17.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4479999999999999e-06 2024-09-18T00:20:17.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952167+0000", 2024-09-18T00:20:17.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00057188400000000004 2024-09-18T00:20:17.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.491 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.491 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.491 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952192+0000", 2024-09-18T00:20:17.491 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.4936999999999999e-05 2024-09-18T00:20:17.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952206+0000", 2024-09-18T00:20:17.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4565e-05 2024-09-18T00:20:17.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.492 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990303+0000", 2024-09-18T00:20:17.492 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038096954000000002 2024-09-18T00:20:17.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.492 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990403+0000", 2024-09-18T00:20:17.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9486000000000003e-05 2024-09-18T00:20:17.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:17.493 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.990773+0000", 2024-09-18T00:20:17.493 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00037037799999999998 2024-09-18T00:20:17.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.493 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.493 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:17.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002316+0000", 2024-09-18T00:20:17.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.011542581 2024-09-18T00:20:17.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.494 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:17.494 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002407+0000", 2024-09-18T00:20:17.494 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0968e-05 2024-09-18T00:20:17.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:17.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002442+0000", 2024-09-18T00:20:17.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.4962000000000002e-05 2024-09-18T00:20:17.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.495 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.495 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.495 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002466+0000", 2024-09-18T00:20:17.495 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3972e-05 2024-09-18T00:20:17.495 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.495 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.495 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.496 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.496 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18691 3.7 3:e6ea7e5c:::rbd_data.112a5ba20d45.0000000000000009:head [delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:17.496 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:20:17.496 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 84.459821020999996, 2024-09-18T00:20:17.496 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.051495530999999997, 2024-09-18T00:20:17.497 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.497 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.497 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.497 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.497 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.497 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.497 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18691 2024-09-18T00:20:17.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.497 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.497 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.497 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:20:17.497 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952226+0000", 2024-09-18T00:20:17.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952228+0000", 2024-09-18T00:20:17.498 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.3200000000000001e-06 2024-09-18T00:20:17.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.498 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.498 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.498 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952234+0000", 2024-09-18T00:20:17.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.0310000000000004e-06 2024-09-18T00:20:17.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952248+0000", 2024-09-18T00:20:17.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4577e-05 2024-09-18T00:20:17.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952260+0000", 2024-09-18T00:20:17.499 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1659e-05 2024-09-18T00:20:17.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.499 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.499 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.499 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.000989+0000", 2024-09-18T00:20:17.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.048729445000000003 2024-09-18T00:20:17.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001076+0000", 2024-09-18T00:20:17.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.6323999999999995e-05 2024-09-18T00:20:17.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 7", 2024-09-18T00:20:17.500 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.001295+0000", 2024-09-18T00:20:17.500 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021944299999999999 2024-09-18T00:20:17.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.500 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.500 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:17.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003502+0000", 2024-09-18T00:20:17.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0022065209999999999 2024-09-18T00:20:17.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:17.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003581+0000", 2024-09-18T00:20:17.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.9404000000000001e-05 2024-09-18T00:20:17.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.501 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.501 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:17.501 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003652+0000", 2024-09-18T00:20:17.501 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0506999999999994e-05 2024-09-18T00:20:17.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.502 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.502 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.003722+0000", 2024-09-18T00:20:17.502 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.0300000000000001e-05 2024-09-18T00:20:17.502 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.502 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.502 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.502 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:18692 3.7 3:e69124d0:::rbd_data.112a6efb7564.0000000000000001:f [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:17.503 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:20:17.503 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 84.459772959999995, 2024-09-18T00:20:17.503 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050418114999999999, 2024-09-18T00:20:17.503 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.503 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.503 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "started", 2024-09-18T00:20:17.503 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.503 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.503 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.503 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 18692 2024-09-18T00:20:17.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.503 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.503 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.503 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:20:17.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952274+0000", 2024-09-18T00:20:17.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.504 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.504 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.504 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952275+0000", 2024-09-18T00:20:17.504 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4e-07 2024-09-18T00:20:17.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952281+0000", 2024-09-18T00:20:17.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.2000000000000002e-06 2024-09-18T00:20:17.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952295+0000", 2024-09-18T00:20:17.505 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.4731e-05 2024-09-18T00:20:17.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.505 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.505 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.505 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:52.952307+0000", 2024-09-18T00:20:17.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1877e-05 2024-09-18T00:20:17.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002506+0000", 2024-09-18T00:20:17.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.050199230999999997 2024-09-18T00:20:17.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.506 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002574+0000", 2024-09-18T00:20:17.506 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.8076000000000003e-05 2024-09-18T00:20:17.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.506 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.506 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.507 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.002693+0000", 2024-09-18T00:20:17.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00011806 2024-09-18T00:20:17.507 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.507 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.507 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.507 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.507 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:19383 3.5 3:a1859bb0:::rbd_data.112a7fe29ab4.000000000000015b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-18T00:20:17.507 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:20:17.507 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 83.789761362999997, 2024-09-18T00:20:17.507 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056760192000000001, 2024-09-18T00:20:17.507 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.508 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.508 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.508 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.508 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.508 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.508 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 19383 2024-09-18T00:20:17.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.508 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.508 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.508 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:20:17.508 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622286+0000", 2024-09-18T00:20:17.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622287+0000", 2024-09-18T00:20:17.509 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.0400000000000005e-07 2024-09-18T00:20:17.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.509 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.509 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.509 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622292+0000", 2024-09-18T00:20:17.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.6110000000000004e-06 2024-09-18T00:20:17.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622304+0000", 2024-09-18T00:20:17.510 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2106000000000001e-05 2024-09-18T00:20:17.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.510 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.510 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.510 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622309+0000", 2024-09-18T00:20:17.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.818e-06 2024-09-18T00:20:17.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.511 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.511 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.511 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622360+0000", 2024-09-18T00:20:17.511 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.0430999999999999e-05 2024-09-18T00:20:17.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.512 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.512 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.622438+0000", 2024-09-18T00:20:17.512 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.8362999999999995e-05 2024-09-18T00:20:17.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.512 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:17.513 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678811+0000", 2024-09-18T00:20:17.513 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.056372894 2024-09-18T00:20:17.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.513 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.513 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:17.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.678906+0000", 2024-09-18T00:20:17.514 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5078e-05 2024-09-18T00:20:17.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.514 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.514 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.514 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:53.679046+0000", 2024-09-18T00:20:17.515 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000139987 2024-09-18T00:20:17.515 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.515 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.515 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.515 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.516 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22468 3.5 3:a250f1ef:::rbd_data.112a948d5956.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:20:17.516 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:20:17.516 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.640679957000003, 2024-09-18T00:20:17.516 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.055599787999999997, 2024-09-18T00:20:17.516 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.516 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.517 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.517 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.517 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.517 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.517 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22468 2024-09-18T00:20:17.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.518 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:20:17.518 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.518 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.518 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.518 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771367+0000", 2024-09-18T00:20:17.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771370+0000", 2024-09-18T00:20:17.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.9629999999999998e-06 2024-09-18T00:20:17.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.519 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.519 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771385+0000", 2024-09-18T00:20:17.519 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5053e-05 2024-09-18T00:20:17.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.519 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771429+0000", 2024-09-18T00:20:17.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.3192999999999998e-05 2024-09-18T00:20:17.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771453+0000", 2024-09-18T00:20:17.520 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.3995999999999999e-05 2024-09-18T00:20:17.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.520 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.520 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.520 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771623+0000", 2024-09-18T00:20:17.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000170846 2024-09-18T00:20:17.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.771838+0000", 2024-09-18T00:20:17.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00021416600000000001 2024-09-18T00:20:17.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.521 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.521 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:17.521 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826866+0000", 2024-09-18T00:20:17.521 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.05502866 2024-09-18T00:20:17.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:17.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826906+0000", 2024-09-18T00:20:17.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.9805999999999998e-05 2024-09-18T00:20:17.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.522 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.522 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.522 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.826967+0000", 2024-09-18T00:20:17.522 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.1105000000000005e-05 2024-09-18T00:20:17.522 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.522 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.522 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.523 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.523 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:22470 3.2 3:474595b6:::rbd_data.112a948d5956.0000000000000004:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-18T00:20:17.523 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:20:17.523 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 78.584488973000006, 2024-09-18T00:20:17.523 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.088321282000000001, 2024-09-18T00:20:17.523 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.523 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.523 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.523 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.523 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.523 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.524 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 22470 2024-09-18T00:20:17.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.524 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:20:17.524 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.524 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.524 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.524 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827558+0000", 2024-09-18T00:20:17.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827560+0000", 2024-09-18T00:20:17.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.762e-06 2024-09-18T00:20:17.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.525 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.525 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827569+0000", 2024-09-18T00:20:17.525 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8740000000000001e-06 2024-09-18T00:20:17.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.525 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827585+0000", 2024-09-18T00:20:17.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.5962000000000001e-05 2024-09-18T00:20:17.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827595+0000", 2024-09-18T00:20:17.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.5910000000000002e-06 2024-09-18T00:20:17.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.526 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.526 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.526 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827811+0000", 2024-09-18T00:20:17.526 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000216858 2024-09-18T00:20:17.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op must wait for map", 2024-09-18T00:20:17.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.827818+0000", 2024-09-18T00:20:17.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.99e-06 2024-09-18T00:20:17.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.527 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829799+0000", 2024-09-18T00:20:17.527 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0019803849999999999 2024-09-18T00:20:17.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.527 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.527 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.829884+0000", 2024-09-18T00:20:17.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.4852999999999995e-05 2024-09-18T00:20:17.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:17.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915616+0000", 2024-09-18T00:20:17.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.085732808999999993 2024-09-18T00:20:17.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.528 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.528 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:17.528 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915711+0000", 2024-09-18T00:20:17.528 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.4578000000000001e-05 2024-09-18T00:20:17.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.529 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.529 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:18:58.915880+0000", 2024-09-18T00:20:17.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00016861999999999999 2024-09-18T00:20:17.529 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.529 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.529 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.529 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.529 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:54153 3.b 3:d3f7b2f3:::rbd_data.112a859a01c9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,zero 4596947~21151] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-18T00:20:17.529 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:20:17.529 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 46.559276236999999, 2024-09-18T00:20:17.529 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.062612886000000006, 2024-09-18T00:20:17.530 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.530 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.530 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.530 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.530 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.530 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.530 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 54153 2024-09-18T00:20:17.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.530 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.530 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.530 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:20:17.530 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.530 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852771+0000", 2024-09-18T00:20:17.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.852773+0000", 2024-09-18T00:20:17.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.426e-06 2024-09-18T00:20:17.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.531 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.531 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855337+0000", 2024-09-18T00:20:17.531 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0025647830000000002 2024-09-18T00:20:17.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855369+0000", 2024-09-18T00:20:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 3.1300999999999997e-05 2024-09-18T00:20:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855387+0000", 2024-09-18T00:20:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7949999999999999e-05 2024-09-18T00:20:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855487+0000", 2024-09-18T00:20:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010062 2024-09-18T00:20:17.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.855686+0000", 2024-09-18T00:20:17.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00019901399999999999 2024-09-18T00:20:17.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:17.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.858037+0000", 2024-09-18T00:20:17.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002351264 2024-09-18T00:20:17.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.533 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.533 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:17.533 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.896302+0000", 2024-09-18T00:20:17.533 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.038264864000000003 2024-09-18T00:20:17.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:17.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915250+0000", 2024-09-18T00:20:17.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.018947538999999999 2024-09-18T00:20:17.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:17.534 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915318+0000", 2024-09-18T00:20:17.534 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.7838999999999999e-05 2024-09-18T00:20:17.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.534 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.534 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.535 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:30.915384+0000", 2024-09-18T00:20:17.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 6.6285999999999995e-05 2024-09-18T00:20:17.535 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.535 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.535 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.535 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.535 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:55488 3.18 3:1d5573d8:::rbd_data.112a859a01c9.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T00:20:17.535 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:20:17.535 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 45.826446027999999, 2024-09-18T00:20:17.535 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.045116205999999999, 2024-09-18T00:20:17.535 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.536 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.536 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.536 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.536 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.536 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.536 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 55488 2024-09-18T00:20:17.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.536 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.536 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.536 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:20:17.536 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.536 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585601+0000", 2024-09-18T00:20:17.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585602+0000", 2024-09-18T00:20:17.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4000000000000001e-07 2024-09-18T00:20:17.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.537 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.537 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.537 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585607+0000", 2024-09-18T00:20:17.537 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.5240000000000002e-06 2024-09-18T00:20:17.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585618+0000", 2024-09-18T00:20:17.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.1493e-05 2024-09-18T00:20:17.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585624+0000", 2024-09-18T00:20:17.538 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.7180000000000001e-06 2024-09-18T00:20:17.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.538 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.538 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585672+0000", 2024-09-18T00:20:17.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 4.8158000000000002e-05 2024-09-18T00:20:17.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.585780+0000", 2024-09-18T00:20:17.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00010781399999999999 2024-09-18T00:20:17.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.539 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:17.539 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.588202+0000", 2024-09-18T00:20:17.539 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.002422156 2024-09-18T00:20:17.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.539 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:17.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630375+0000", 2024-09-18T00:20:17.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.042172820999999999 2024-09-18T00:20:17.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.540 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.540 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:17.540 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630426+0000", 2024-09-18T00:20:17.540 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.1369000000000002e-05 2024-09-18T00:20:17.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:17.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630581+0000", 2024-09-18T00:20:17.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000155041 2024-09-18T00:20:17.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.541 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.541 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.541 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:19:31.630718+0000", 2024-09-18T00:20:17.541 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000136372 2024-09-18T00:20:17.541 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.541 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.541 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.542 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.542 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:62587 3.8 3:1527150c:::rbd_data.112a13ec2d6c.0000000000000003:head [delete] snapc b=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-18T00:20:17.542 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:07.612383+0000", 2024-09-18T00:20:17.542 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 9.7996646839999997, 2024-09-18T00:20:17.542 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.88651563099999997, 2024-09-18T00:20:17.542 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.542 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.542 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.542 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.542 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.542 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.542 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 62587 2024-09-18T00:20:17.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.543 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612383+0000", 2024-09-18T00:20:17.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612383+0000", 2024-09-18T00:20:17.543 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.543 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.543 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.543 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612385+0000", 2024-09-18T00:20:17.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.9489999999999999e-06 2024-09-18T00:20:17.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612395+0000", 2024-09-18T00:20:17.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9340000000000001e-06 2024-09-18T00:20:17.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.544 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612417+0000", 2024-09-18T00:20:17.544 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 2.2633000000000002e-05 2024-09-18T00:20:17.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.544 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.544 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612430+0000", 2024-09-18T00:20:17.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.2639e-05 2024-09-18T00:20:17.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612529+0000", 2024-09-18T00:20:17.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 9.9133000000000001e-05 2024-09-18T00:20:17.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.545 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.545 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-18T00:20:17.545 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:07.612543+0000", 2024-09-18T00:20:17.545 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.412e-05 2024-09-18T00:20:17.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.490391+0000", 2024-09-18T00:20:17.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.87784777000000003 2024-09-18T00:20:17.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.490724+0000", 2024-09-18T00:20:17.546 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.00033358900000000002 2024-09-18T00:20:17.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.546 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.546 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for subops from 1", 2024-09-18T00:20:17.546 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.491039+0000", 2024-09-18T00:20:17.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000314548 2024-09-18T00:20:17.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:17.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.493889+0000", 2024-09-18T00:20:17.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0028495339999999999 2024-09-18T00:20:17.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:17.547 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.498625+0000", 2024-09-18T00:20:17.547 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0047369689999999997 2024-09-18T00:20:17.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.547 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:17.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.498794+0000", 2024-09-18T00:20:17.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000168006 2024-09-18T00:20:17.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.548 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.548 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:08.498898+0000", 2024-09-18T00:20:17.548 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000104807 2024-09-18T00:20:17.548 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.548 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.548 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.548 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.548 INFO:teuthology.orchestra.run.smithi097.stdout: "description": "osd_op(client.4394.0:62673 3.8 3:10f5bea1:::rbd_object_map.112a7b1589a:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-18T00:20:17.549 INFO:teuthology.orchestra.run.smithi097.stdout: "initiated_at": "2024-09-18T00:20:09.519309+0000", 2024-09-18T00:20:17.549 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 7.892738016, 2024-09-18T00:20:17.549 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.995633447, 2024-09-18T00:20:17.549 INFO:teuthology.orchestra.run.smithi097.stdout: "continuous": false, 2024-09-18T00:20:17.549 INFO:teuthology.orchestra.run.smithi097.stdout: "type_data": { 2024-09-18T00:20:17.549 INFO:teuthology.orchestra.run.smithi097.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.549 INFO:teuthology.orchestra.run.smithi097.stdout: "client_info": { 2024-09-18T00:20:17.549 INFO:teuthology.orchestra.run.smithi097.stdout: "client": "client.4394", 2024-09-18T00:20:17.549 INFO:teuthology.orchestra.run.smithi097.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.549 INFO:teuthology.orchestra.run.smithi097.stdout: "tid": 62673 2024-09-18T00:20:17.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.550 INFO:teuthology.orchestra.run.smithi097.stdout: "events": [ 2024-09-18T00:20:17.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.550 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "initiated", 2024-09-18T00:20:17.550 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:09.519309+0000", 2024-09-18T00:20:17.550 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.550 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "header_read", 2024-09-18T00:20:17.551 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:09.519309+0000", 2024-09-18T00:20:17.551 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0 2024-09-18T00:20:17.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.551 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "throttled", 2024-09-18T00:20:17.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:09.519311+0000", 2024-09-18T00:20:17.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.689e-06 2024-09-18T00:20:17.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "all_read", 2024-09-18T00:20:17.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:09.519319+0000", 2024-09-18T00:20:17.552 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3510000000000002e-06 2024-09-18T00:20:17.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.552 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.552 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "dispatched", 2024-09-18T00:20:17.552 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:09.519336+0000", 2024-09-18T00:20:17.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.7053000000000001e-05 2024-09-18T00:20:17.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:09.519345+0000", 2024-09-18T00:20:17.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.8890000000000008e-06 2024-09-18T00:20:17.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.553 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.553 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:09.519420+0000", 2024-09-18T00:20:17.553 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 7.4296999999999997e-05 2024-09-18T00:20:17.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.553 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "waiting for peered", 2024-09-18T00:20:17.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:09.519430+0000", 2024-09-18T00:20:17.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 1.0284e-05 2024-09-18T00:20:17.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "reached_pg", 2024-09-18T00:20:17.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:10.513433+0000", 2024-09-18T00:20:17.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.99400258900000005 2024-09-18T00:20:17.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.554 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.554 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "started", 2024-09-18T00:20:17.554 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:10.513613+0000", 2024-09-18T00:20:17.554 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.000180736 2024-09-18T00:20:17.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "op_commit", 2024-09-18T00:20:17.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:10.514801+0000", 2024-09-18T00:20:17.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 0.0011874629999999999 2024-09-18T00:20:17.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "commit_sent", 2024-09-18T00:20:17.555 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:10.514859+0000", 2024-09-18T00:20:17.555 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 5.8669999999999999e-05 2024-09-18T00:20:17.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-09-18T00:20:17.555 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-09-18T00:20:17.555 INFO:teuthology.orchestra.run.smithi097.stdout: "event": "done", 2024-09-18T00:20:17.556 INFO:teuthology.orchestra.run.smithi097.stdout: "time": "2024-09-18T00:20:10.514943+0000", 2024-09-18T00:20:17.556 INFO:teuthology.orchestra.run.smithi097.stdout: "duration": 8.3425999999999997e-05 2024-09-18T00:20:17.556 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.556 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.556 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.556 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-09-18T00:20:17.556 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-18T00:20:17.556 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:17.558 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T00:20:17.558 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:17.556+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:17.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /var/log/ceph/exp.3.6.1 2024-09-18T00:20:17.686 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-09-18T00:20:17.684+0000 7f8f3dcf3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26439) UID: 0 2024-09-18T00:20:17.700 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:17.700 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:20:17.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.701 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:63978 3.1d 3:bf8d1506:::rbd_object_map.112a45c8e167:head [call lock.unlock in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-18T00:20:17.701 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:20:17.699466+0000", 2024-09-18T00:20:17.701 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.00098848199999999995, 2024-09-18T00:20:17.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001037102, 2024-09-18T00:20:17.701 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:17.701 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:17.701 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "waiting for sub ops", 2024-09-18T00:20:17.701 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:17.701 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:17.701 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.701 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 63978 2024-09-18T00:20:17.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.702 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:17.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:17.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:20:17.699466+0000", 2024-09-18T00:20:17.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:17.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:17.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:20:17.699466+0000", 2024-09-18T00:20:17.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:17.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:17.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:20:17.699467+0000", 2024-09-18T00:20:17.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-18T00:20:17.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:17.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:20:17.699471+0000", 2024-09-18T00:20:17.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2560000000000004e-06 2024-09-18T00:20:17.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:17.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:20:17.699481+0000", 2024-09-18T00:20:17.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5179999999999993e-06 2024-09-18T00:20:17.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:20:17.699486+0000", 2024-09-18T00:20:17.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0409999999999997e-06 2024-09-18T00:20:17.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:17.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:20:17.699524+0000", 2024-09-18T00:20:17.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8127000000000003e-05 2024-09-18T00:20:17.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:17.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:20:17.699578+0000", 2024-09-18T00:20:17.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.452e-05 2024-09-18T00:20:17.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 5", 2024-09-18T00:20:17.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:20:17.699678+0000", 2024-09-18T00:20:17.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9581999999999996e-05 2024-09-18T00:20:17.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:17.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:20:17.700429+0000", 2024-09-18T00:20:17.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00075141100000000001 2024-09-18T00:20:17.705 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:17.705 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:17.705 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:17.705 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:17.706 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-18T00:20:17.706 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 1 2024-09-18T00:20:17.706 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:17.716 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T00:20:17.720 INFO:teuthology.orchestra.run.smithi097.stdout:All operations completed A-OK! 2024-09-18T00:20:17.760 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-09-18T00:20:17.756+0000 7f2b91033640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26435) UID: 0 2024-09-18T00:20:17.796 DEBUG:teuthology.parallel:result is None 2024-09-18T00:20:17.796 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-18T00:20:17.804 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-18T00:20:17.845 INFO:tasks.thrashosds:joining thrashosds 2024-09-18T00:20:17.845 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:17.846 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:17.846 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:20:17.846 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:20:17.846 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:17.852 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T00:20:17.979 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:17.979 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:20:17.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:20:17.979 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:20:17.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.979 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2418 3.1a 3:59a8b40d:::rbd_data.112aa7ba4578.00000000000001b0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T00:20:17.979 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:20:17.979 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 117.24029195, 2024-09-18T00:20:17.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10488562999999999, 2024-09-18T00:20:17.980 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:17.980 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:17.980 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.980 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:17.980 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:17.980 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.980 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2418 2024-09-18T00:20:17.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.980 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:17.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:17.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:20:17.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:17.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:17.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735589+0000", 2024-09-18T00:20:17.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:17.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:17.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735592+0000", 2024-09-18T00:20:17.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5229999999999998e-06 2024-09-18T00:20:17.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:17.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735604+0000", 2024-09-18T00:20:17.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2143e-05 2024-09-18T00:20:17.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:17.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735629+0000", 2024-09-18T00:20:17.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.497e-05 2024-09-18T00:20:17.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735648+0000", 2024-09-18T00:20:17.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8896999999999999e-05 2024-09-18T00:20:17.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.983 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:17.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.735775+0000", 2024-09-18T00:20:17.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126935 2024-09-18T00:20:17.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.983 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:17.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.736118+0000", 2024-09-18T00:20:17.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034330800000000001 2024-09-18T00:20:17.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.983 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-18T00:20:17.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.741633+0000", 2024-09-18T00:20:17.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055144139999999996 2024-09-18T00:20:17.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:17.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840134+0000", 2024-09-18T00:20:17.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.098501618999999999 2024-09-18T00:20:17.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:17.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840184+0000", 2024-09-18T00:20:17.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9953000000000001e-05 2024-09-18T00:20:17.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:17.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840336+0000", 2024-09-18T00:20:17.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151476 2024-09-18T00:20:17.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:17.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.840475+0000", 2024-09-18T00:20:17.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013939200000000001 2024-09-18T00:20:17.985 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:17.985 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:17.985 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:17.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.986 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:22272 3.3 3:c617f123:::rbd_data.112a948d5956.0000000000000009:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T00:20:17.986 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:20:17.986 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 79.984783750999995, 2024-09-18T00:20:17.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10752577300000001, 2024-09-18T00:20:17.986 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:17.986 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:17.986 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.986 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:17.986 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:17.987 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.987 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 22272 2024-09-18T00:20:17.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.987 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:17.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:17.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:20:17.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:17.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:17.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991098+0000", 2024-09-18T00:20:17.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:17.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:17.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991102+0000", 2024-09-18T00:20:17.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9369999999999997e-06 2024-09-18T00:20:17.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:17.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991118+0000", 2024-09-18T00:20:17.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6396e-05 2024-09-18T00:20:17.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:17.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991150+0000", 2024-09-18T00:20:17.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2132999999999999e-05 2024-09-18T00:20:17.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991170+0000", 2024-09-18T00:20:17.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0085000000000002e-05 2024-09-18T00:20:17.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:17.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991370+0000", 2024-09-18T00:20:17.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019963700000000001 2024-09-18T00:20:17.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:17.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.991518+0000", 2024-09-18T00:20:17.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000148424 2024-09-18T00:20:17.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:17.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:57.998850+0000", 2024-09-18T00:20:17.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073315480000000002 2024-09-18T00:20:17.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:17.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098178+0000", 2024-09-18T00:20:17.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.099328067000000006 2024-09-18T00:20:17.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:17.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098314+0000", 2024-09-18T00:20:17.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000135678 2024-09-18T00:20:17.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:17.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098498+0000", 2024-09-18T00:20:17.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018479600000000001 2024-09-18T00:20:17.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:17.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:58.098623+0000", 2024-09-18T00:20:17.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000125072 2024-09-18T00:20:17.992 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:17.992 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:17.992 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:17.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.992 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:23689 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-18T00:20:17.992 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:20:17.992 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.123458972999998, 2024-09-18T00:20:17.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9562625809999998, 2024-09-18T00:20:17.993 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:17.993 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:17.993 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:17.993 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:17.993 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:17.993 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:17.993 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 23689 2024-09-18T00:20:17.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.993 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:17.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:17.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:20:17.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:17.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:17.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852422+0000", 2024-09-18T00:20:17.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:17.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:17.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852423+0000", 2024-09-18T00:20:17.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9999999999999995e-07 2024-09-18T00:20:17.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:17.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852429+0000", 2024-09-18T00:20:17.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3279999999999996e-06 2024-09-18T00:20:17.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:17.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852442+0000", 2024-09-18T00:20:17.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3052e-05 2024-09-18T00:20:17.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:17.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852448+0000", 2024-09-18T00:20:17.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2779999999999999e-06 2024-09-18T00:20:17.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:17.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852523+0000", 2024-09-18T00:20:17.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4463999999999997e-05 2024-09-18T00:20:17.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:17.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:01.852538+0000", 2024-09-18T00:20:17.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5466999999999999e-05 2024-09-18T00:20:17.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:17.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793478+0000", 2024-09-18T00:20:17.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.94094012699999996 2024-09-18T00:20:17.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:17.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:02.793494+0000", 2024-09-18T00:20:17.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5807999999999998e-05 2024-09-18T00:20:17.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:17.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.804875+0000", 2024-09-18T00:20:17.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0113811939999999 2024-09-18T00:20:17.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:17.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805359+0000", 2024-09-18T00:20:17.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000484248 2024-09-18T00:20:17.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-18T00:20:17.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.805872+0000", 2024-09-18T00:20:17.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00051218100000000001 2024-09-18T00:20:17.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:17.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.807790+0000", 2024-09-18T00:20:17.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001918763 2024-09-18T00:20:17.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:17.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808242+0000", 2024-09-18T00:20:17.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045126200000000002 2024-09-18T00:20:17.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:17.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808428+0000", 2024-09-18T00:20:17.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018633000000000001 2024-09-18T00:20:17.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:17.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:17.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:03.808685+0000", 2024-09-18T00:20:17.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025707900000000001 2024-09-18T00:20:17.999 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:17.999 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:17.999 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:17.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:17.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.000 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47925 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7864320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:18.000 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:20:18.000 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.943296871999998, 2024-09-18T00:20:18.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086484302999999998, 2024-09-18T00:20:18.000 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.000 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.000 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.000 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.000 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.000 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.000 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47925 2024-09-18T00:20:18.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.000 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:20:18.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:20:18.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032585+0000", 2024-09-18T00:20:18.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.82e-07 2024-09-18T00:20:18.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032587+0000", 2024-09-18T00:20:18.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.058e-06 2024-09-18T00:20:18.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032593+0000", 2024-09-18T00:20:18.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1380000000000004e-06 2024-09-18T00:20:18.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032598+0000", 2024-09-18T00:20:18.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9819999999999999e-06 2024-09-18T00:20:18.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118873+0000", 2024-09-18T00:20:18.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086274609000000002 2024-09-18T00:20:18.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.118950+0000", 2024-09-18T00:20:18.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6687999999999995e-05 2024-09-18T00:20:18.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119069+0000", 2024-09-18T00:20:18.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011934600000000001 2024-09-18T00:20:18.004 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.004 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.004 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.004 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47926 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7897088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:18.004 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032604+0000", 2024-09-18T00:20:18.004 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.943276996000002, 2024-09-18T00:20:18.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086712168000000006, 2024-09-18T00:20:18.004 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.004 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.004 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.004 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.005 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.005 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.005 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47926 2024-09-18T00:20:18.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.005 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032604+0000", 2024-09-18T00:20:18.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032604+0000", 2024-09-18T00:20:18.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032605+0000", 2024-09-18T00:20:18.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:20:18.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032611+0000", 2024-09-18T00:20:18.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4860000000000002e-06 2024-09-18T00:20:18.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032618+0000", 2024-09-18T00:20:18.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6699999999999997e-06 2024-09-18T00:20:18.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032623+0000", 2024-09-18T00:20:18.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0860000000000001e-06 2024-09-18T00:20:18.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119111+0000", 2024-09-18T00:20:18.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086488067000000002 2024-09-18T00:20:18.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119185+0000", 2024-09-18T00:20:18.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3758999999999996e-05 2024-09-18T00:20:18.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119317+0000", 2024-09-18T00:20:18.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013159599999999999 2024-09-18T00:20:18.008 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.008 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.008 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.009 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47927 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7929856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:18.009 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:20:18.009 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.943251654999997, 2024-09-18T00:20:18.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086929532000000004, 2024-09-18T00:20:18.013 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.013 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.013 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.013 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.013 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.013 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.013 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47927 2024-09-18T00:20:18.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.013 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:20:18.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:20:18.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032630+0000", 2024-09-18T00:20:18.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-18T00:20:18.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032632+0000", 2024-09-18T00:20:18.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1320000000000001e-06 2024-09-18T00:20:18.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032639+0000", 2024-09-18T00:20:18.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1840000000000004e-06 2024-09-18T00:20:18.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032644+0000", 2024-09-18T00:20:18.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2519999999999999e-06 2024-09-18T00:20:18.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119358+0000", 2024-09-18T00:20:18.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086714075000000002 2024-09-18T00:20:18.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119438+0000", 2024-09-18T00:20:18.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9756e-05 2024-09-18T00:20:18.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119559+0000", 2024-09-18T00:20:18.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121579 2024-09-18T00:20:18.017 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.018 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.018 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.018 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47928 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7962624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:18.018 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032650+0000", 2024-09-18T00:20:18.018 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.943231230999999, 2024-09-18T00:20:18.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087152732999999996, 2024-09-18T00:20:18.018 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.018 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.018 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.019 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.019 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.019 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.019 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47928 2024-09-18T00:20:18.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.019 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032650+0000", 2024-09-18T00:20:18.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032650+0000", 2024-09-18T00:20:18.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032651+0000", 2024-09-18T00:20:18.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:20:18.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032653+0000", 2024-09-18T00:20:18.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.182e-06 2024-09-18T00:20:18.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032659+0000", 2024-09-18T00:20:18.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2319999999999998e-06 2024-09-18T00:20:18.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032664+0000", 2024-09-18T00:20:18.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0420000000000002e-06 2024-09-18T00:20:18.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119602+0000", 2024-09-18T00:20:18.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.086938213 2024-09-18T00:20:18.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119676+0000", 2024-09-18T00:20:18.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3708000000000006e-05 2024-09-18T00:20:18.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119803+0000", 2024-09-18T00:20:18.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126852 2024-09-18T00:20:18.022 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.022 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.022 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.023 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47929 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 7995392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:18.023 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032670+0000", 2024-09-18T00:20:18.023 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.943210972999999, 2024-09-18T00:20:18.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087373082000000005, 2024-09-18T00:20:18.023 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.023 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.023 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.023 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.023 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.024 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.024 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47929 2024-09-18T00:20:18.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.024 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032670+0000", 2024-09-18T00:20:18.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032670+0000", 2024-09-18T00:20:18.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032671+0000", 2024-09-18T00:20:18.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:20:18.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032673+0000", 2024-09-18T00:20:18.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.928e-06 2024-09-18T00:20:18.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032679+0000", 2024-09-18T00:20:18.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.02e-06 2024-09-18T00:20:18.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032684+0000", 2024-09-18T00:20:18.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1660000000000002e-06 2024-09-18T00:20:18.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119844+0000", 2024-09-18T00:20:18.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087160248999999995 2024-09-18T00:20:18.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.119920+0000", 2024-09-18T00:20:18.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5181999999999999e-05 2024-09-18T00:20:18.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120044+0000", 2024-09-18T00:20:18.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012403300000000001 2024-09-18T00:20:18.027 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.027 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.027 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.027 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47930 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8028160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:18.028 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032690+0000", 2024-09-18T00:20:18.028 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.943191120999998, 2024-09-18T00:20:18.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087595941999999996, 2024-09-18T00:20:18.028 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.028 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.028 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.028 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.028 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.028 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.028 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47930 2024-09-18T00:20:18.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.028 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032690+0000", 2024-09-18T00:20:18.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032690+0000", 2024-09-18T00:20:18.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032691+0000", 2024-09-18T00:20:18.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9399999999999995e-07 2024-09-18T00:20:18.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032693+0000", 2024-09-18T00:20:18.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1299999999999999e-06 2024-09-18T00:20:18.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032699+0000", 2024-09-18T00:20:18.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1879999999999999e-06 2024-09-18T00:20:18.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032704+0000", 2024-09-18T00:20:18.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0939999999999998e-06 2024-09-18T00:20:18.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120087+0000", 2024-09-18T00:20:18.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087383049000000004 2024-09-18T00:20:18.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120164+0000", 2024-09-18T00:20:18.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6679000000000002e-05 2024-09-18T00:20:18.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120286+0000", 2024-09-18T00:20:18.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012230800000000001 2024-09-18T00:20:18.032 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.032 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.032 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.032 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47931 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8060928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:18.032 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:20:18.032 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.943170877999997, 2024-09-18T00:20:18.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087823688999999996, 2024-09-18T00:20:18.032 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.032 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.032 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.033 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.033 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.033 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.033 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47931 2024-09-18T00:20:18.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.033 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:20:18.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:20:18.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032711+0000", 2024-09-18T00:20:18.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9200000000000001e-07 2024-09-18T00:20:18.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032713+0000", 2024-09-18T00:20:18.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1239999999999999e-06 2024-09-18T00:20:18.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032719+0000", 2024-09-18T00:20:18.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1659999999999999e-06 2024-09-18T00:20:18.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032724+0000", 2024-09-18T00:20:18.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9359999999999998e-06 2024-09-18T00:20:18.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120330+0000", 2024-09-18T00:20:18.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087605559 2024-09-18T00:20:18.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120403+0000", 2024-09-18T00:20:18.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-05 2024-09-18T00:20:18.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120534+0000", 2024-09-18T00:20:18.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013121199999999999 2024-09-18T00:20:18.036 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.036 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.036 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.037 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47932 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8093696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:18.037 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032730+0000", 2024-09-18T00:20:18.037 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.943150979999999, 2024-09-18T00:20:18.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088052486999999999, 2024-09-18T00:20:18.037 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.037 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.037 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.037 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.037 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.037 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.037 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47932 2024-09-18T00:20:18.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.037 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032730+0000", 2024-09-18T00:20:18.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032730+0000", 2024-09-18T00:20:18.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032731+0000", 2024-09-18T00:20:18.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2200000000000004e-07 2024-09-18T00:20:18.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032743+0000", 2024-09-18T00:20:18.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1796e-05 2024-09-18T00:20:18.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032749+0000", 2024-09-18T00:20:18.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6959999999999999e-06 2024-09-18T00:20:18.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032755+0000", 2024-09-18T00:20:18.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.22e-06 2024-09-18T00:20:18.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120578+0000", 2024-09-18T00:20:18.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.087823017000000003 2024-09-18T00:20:18.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120654+0000", 2024-09-18T00:20:18.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6476000000000004e-05 2024-09-18T00:20:18.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120783+0000", 2024-09-18T00:20:18.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012876000000000001 2024-09-18T00:20:18.041 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.041 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.041 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.041 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47933 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8126464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:18.041 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032761+0000", 2024-09-18T00:20:18.041 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.943119961999997, 2024-09-18T00:20:18.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088263233999999996, 2024-09-18T00:20:18.041 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.041 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.042 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.042 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.042 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.042 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.042 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47933 2024-09-18T00:20:18.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.042 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032761+0000", 2024-09-18T00:20:18.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032761+0000", 2024-09-18T00:20:18.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032762+0000", 2024-09-18T00:20:18.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-18T00:20:18.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032764+0000", 2024-09-18T00:20:18.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1519999999999999e-06 2024-09-18T00:20:18.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032770+0000", 2024-09-18T00:20:18.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1700000000000002e-06 2024-09-18T00:20:18.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032778+0000", 2024-09-18T00:20:18.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2400000000000001e-06 2024-09-18T00:20:18.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120825+0000", 2024-09-18T00:20:18.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088047046000000004 2024-09-18T00:20:18.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.120898+0000", 2024-09-18T00:20:18.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3011999999999998e-05 2024-09-18T00:20:18.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121025+0000", 2024-09-18T00:20:18.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012710999999999999 2024-09-18T00:20:18.046 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.046 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.046 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.046 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47934 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8159232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:18.046 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032784+0000", 2024-09-18T00:20:18.046 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.943097410999997, 2024-09-18T00:20:18.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088479458999999996, 2024-09-18T00:20:18.046 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.046 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.047 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.047 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.047 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.047 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.047 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47934 2024-09-18T00:20:18.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.047 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032784+0000", 2024-09-18T00:20:18.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032784+0000", 2024-09-18T00:20:18.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032785+0000", 2024-09-18T00:20:18.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9599999999999999e-07 2024-09-18T00:20:18.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032787+0000", 2024-09-18T00:20:18.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.052e-06 2024-09-18T00:20:18.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032793+0000", 2024-09-18T00:20:18.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2439999999999998e-06 2024-09-18T00:20:18.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032798+0000", 2024-09-18T00:20:18.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9080000000000003e-06 2024-09-18T00:20:18.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121066+0000", 2024-09-18T00:20:18.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088267795999999996 2024-09-18T00:20:18.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121139+0000", 2024-09-18T00:20:18.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2910999999999994e-05 2024-09-18T00:20:18.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121263+0000", 2024-09-18T00:20:18.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000124852 2024-09-18T00:20:18.050 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.050 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.050 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.051 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47935 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8192000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:18.051 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032841+0000", 2024-09-18T00:20:18.051 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.943040330999999, 2024-09-18T00:20:18.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088666069, 2024-09-18T00:20:18.051 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.051 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.051 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.051 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.051 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.051 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.051 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47935 2024-09-18T00:20:18.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.051 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032841+0000", 2024-09-18T00:20:18.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032841+0000", 2024-09-18T00:20:18.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032842+0000", 2024-09-18T00:20:18.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6400000000000002e-07 2024-09-18T00:20:18.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032845+0000", 2024-09-18T00:20:18.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8959999999999999e-06 2024-09-18T00:20:18.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032852+0000", 2024-09-18T00:20:18.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1640000000000004e-06 2024-09-18T00:20:18.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032857+0000", 2024-09-18T00:20:18.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3340000000000001e-06 2024-09-18T00:20:18.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121304+0000", 2024-09-18T00:20:18.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088447001999999997 2024-09-18T00:20:18.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121376+0000", 2024-09-18T00:20:18.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2121999999999998e-05 2024-09-18T00:20:18.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121507+0000", 2024-09-18T00:20:18.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130987 2024-09-18T00:20:18.055 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.055 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.055 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.055 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47936 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8224768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:18.055 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:20:18.055 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.943017609999998, 2024-09-18T00:20:18.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088878349999999995, 2024-09-18T00:20:18.055 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.055 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.055 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.055 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.056 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.056 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.056 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47936 2024-09-18T00:20:18.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.056 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:20:18.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:20:18.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032864+0000", 2024-09-18T00:20:18.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8999999999999997e-07 2024-09-18T00:20:18.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032866+0000", 2024-09-18T00:20:18.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.142e-06 2024-09-18T00:20:18.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032873+0000", 2024-09-18T00:20:18.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2199999999999997e-06 2024-09-18T00:20:18.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032878+0000", 2024-09-18T00:20:18.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1220000000000003e-06 2024-09-18T00:20:18.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121548+0000", 2024-09-18T00:20:18.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088670530999999997 2024-09-18T00:20:18.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121621+0000", 2024-09-18T00:20:18.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2941999999999994e-05 2024-09-18T00:20:18.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121742+0000", 2024-09-18T00:20:18.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120903 2024-09-18T00:20:18.059 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.059 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.059 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.059 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47937 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8257536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:18.059 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:20:18.060 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.942997501999997, 2024-09-18T00:20:18.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089098572000000001, 2024-09-18T00:20:18.060 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.060 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.060 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.060 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.060 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.060 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.060 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47937 2024-09-18T00:20:18.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.060 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:20:18.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:20:18.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032884+0000", 2024-09-18T00:20:18.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9200000000000001e-07 2024-09-18T00:20:18.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032886+0000", 2024-09-18T00:20:18.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9999999999999999e-06 2024-09-18T00:20:18.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032893+0000", 2024-09-18T00:20:18.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2020000000000001e-06 2024-09-18T00:20:18.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032898+0000", 2024-09-18T00:20:18.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9860000000000002e-06 2024-09-18T00:20:18.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121785+0000", 2024-09-18T00:20:18.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.088887318000000007 2024-09-18T00:20:18.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121858+0000", 2024-09-18T00:20:18.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2900999999999999e-05 2024-09-18T00:20:18.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.121983+0000", 2024-09-18T00:20:18.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000124673 2024-09-18T00:20:18.063 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.063 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.064 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.064 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47938 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8290304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:18.064 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:20:18.064 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.942977718000002, 2024-09-18T00:20:18.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089335780000000004, 2024-09-18T00:20:18.064 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.064 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.064 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.064 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.064 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.064 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.065 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47938 2024-09-18T00:20:18.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.065 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:20:18.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:20:18.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032904+0000", 2024-09-18T00:20:18.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2799999999999996e-07 2024-09-18T00:20:18.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032923+0000", 2024-09-18T00:20:18.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8218000000000001e-05 2024-09-18T00:20:18.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032930+0000", 2024-09-18T00:20:18.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5440000000000001e-06 2024-09-18T00:20:18.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032936+0000", 2024-09-18T00:20:18.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4720000000000003e-06 2024-09-18T00:20:18.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122023+0000", 2024-09-18T00:20:18.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089087728000000005 2024-09-18T00:20:18.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122117+0000", 2024-09-18T00:20:18.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3295999999999996e-05 2024-09-18T00:20:18.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122240+0000", 2024-09-18T00:20:18.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012289399999999999 2024-09-18T00:20:18.068 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.068 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.068 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.068 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47939 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8323072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:18.068 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032943+0000", 2024-09-18T00:20:18.068 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.942938165000001, 2024-09-18T00:20:18.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089547929999999998, 2024-09-18T00:20:18.068 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.068 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.068 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.069 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.069 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.069 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.069 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47939 2024-09-18T00:20:18.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.069 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032943+0000", 2024-09-18T00:20:18.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032943+0000", 2024-09-18T00:20:18.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032944+0000", 2024-09-18T00:20:18.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6199999999999998e-07 2024-09-18T00:20:18.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032946+0000", 2024-09-18T00:20:18.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2840000000000001e-06 2024-09-18T00:20:18.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032952+0000", 2024-09-18T00:20:18.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2700000000000001e-06 2024-09-18T00:20:18.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032957+0000", 2024-09-18T00:20:18.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0200000000000002e-06 2024-09-18T00:20:18.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122280+0000", 2024-09-18T00:20:18.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089322330000000005 2024-09-18T00:20:18.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122354+0000", 2024-09-18T00:20:18.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3794000000000005e-05 2024-09-18T00:20:18.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122491+0000", 2024-09-18T00:20:18.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013767 2024-09-18T00:20:18.072 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.072 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.072 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.073 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:47940 3.1a 3:58c5ae96:::rbd_data.112a4b380b1a.0000000000000004:19 [read 8355840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:18.073 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:20:18.073 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.942917737000002, 2024-09-18T00:20:18.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089758663000000002, 2024-09-18T00:20:18.073 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.073 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.073 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.073 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.073 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.073 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.073 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 47940 2024-09-18T00:20:18.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:20:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:20:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032964+0000", 2024-09-18T00:20:18.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8599999999999998e-07 2024-09-18T00:20:18.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032966+0000", 2024-09-18T00:20:18.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1160000000000002e-06 2024-09-18T00:20:18.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032972+0000", 2024-09-18T00:20:18.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1639999999999998e-06 2024-09-18T00:20:18.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.032978+0000", 2024-09-18T00:20:18.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1320000000000002e-06 2024-09-18T00:20:18.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122520+0000", 2024-09-18T00:20:18.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.089541930000000006 2024-09-18T00:20:18.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122601+0000", 2024-09-18T00:20:18.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1207999999999999e-05 2024-09-18T00:20:18.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.122722+0000", 2024-09-18T00:20:18.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012162700000000001 2024-09-18T00:20:18.077 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.077 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.077 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.077 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:61476 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-18T00:20:18.077 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:57.749097+0000", 2024-09-18T00:20:18.077 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.226784687999999, 2024-09-18T00:20:18.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99209296800000002, 2024-09-18T00:20:18.077 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.077 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.077 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.078 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.078 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.078 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.078 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 61476 2024-09-18T00:20:18.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.078 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:57.749097+0000", 2024-09-18T00:20:18.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:57.749097+0000", 2024-09-18T00:20:18.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:57.749100+0000", 2024-09-18T00:20:18.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.903e-06 2024-09-18T00:20:18.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:57.749113+0000", 2024-09-18T00:20:18.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3654e-05 2024-09-18T00:20:18.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:57.749146+0000", 2024-09-18T00:20:18.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2815999999999999e-05 2024-09-18T00:20:18.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:57.749161+0000", 2024-09-18T00:20:18.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4893e-05 2024-09-18T00:20:18.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:57.749283+0000", 2024-09-18T00:20:18.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012157100000000001 2024-09-18T00:20:18.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:18.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:57.749304+0000", 2024-09-18T00:20:18.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0999999999999999e-05 2024-09-18T00:20:18.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:58.740510+0000", 2024-09-18T00:20:18.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99120623299999999 2024-09-18T00:20:18.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:58.740839+0000", 2024-09-18T00:20:18.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032949999999999999 2024-09-18T00:20:18.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:58.741190+0000", 2024-09-18T00:20:18.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00035039799999999997 2024-09-18T00:20:18.082 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.082 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.082 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.082 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.082 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.082 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:18.083 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T00:20:18.212 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:18.212 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:18.212 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:20:18.212 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:18.226 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T00:20:18.349 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:18.350 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:18.350 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:20:18.350 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:20:18.350 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:18.364 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T00:20:18.484 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:18.485 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:20:18.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:20:18.485 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:20:18.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.485 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2229 3.10 3:09d04be5:::rbd_data.112aa7ba4578.000000000000014e:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:20:18.485 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:20:18.485 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.218406868, 2024-09-18T00:20:18.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048730086999999998, 2024-09-18T00:20:18.485 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.485 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.485 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:18.485 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.485 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.486 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.486 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2229 2024-09-18T00:20:18.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.486 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:20:18.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263359+0000", 2024-09-18T00:20:18.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263362+0000", 2024-09-18T00:20:18.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3869999999999999e-06 2024-09-18T00:20:18.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263381+0000", 2024-09-18T00:20:18.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9153000000000001e-05 2024-09-18T00:20:18.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263418+0000", 2024-09-18T00:20:18.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6346000000000001e-05 2024-09-18T00:20:18.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263435+0000", 2024-09-18T00:20:18.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7309999999999999e-05 2024-09-18T00:20:18.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263589+0000", 2024-09-18T00:20:18.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015451199999999999 2024-09-18T00:20:18.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263902+0000", 2024-09-18T00:20:18.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031291799999999997 2024-09-18T00:20:18.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:18.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.265995+0000", 2024-09-18T00:20:18.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020931809999999999 2024-09-18T00:20:18.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:18.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311788+0000", 2024-09-18T00:20:18.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045792078 2024-09-18T00:20:18.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:18.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311903+0000", 2024-09-18T00:20:18.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011510699999999999 2024-09-18T00:20:18.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:18.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312009+0000", 2024-09-18T00:20:18.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010630599999999999 2024-09-18T00:20:18.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312089+0000", 2024-09-18T00:20:18.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9789000000000005e-05 2024-09-18T00:20:18.491 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.491 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.491 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.491 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2229 3.10 e36/23 ondisk, result = 0)", 2024-09-18T00:20:18.491 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:20:18.491 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.199261209, 2024-09-18T00:20:18.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029600955000000002, 2024-09-18T00:20:18.491 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.491 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.491 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.492 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:20:18.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282504+0000", 2024-09-18T00:20:18.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282507+0000", 2024-09-18T00:20:18.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9160000000000001e-06 2024-09-18T00:20:18.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282522+0000", 2024-09-18T00:20:18.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5e-05 2024-09-18T00:20:18.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282558+0000", 2024-09-18T00:20:18.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5694000000000001e-05 2024-09-18T00:20:18.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.282580+0000", 2024-09-18T00:20:18.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2323e-05 2024-09-18T00:20:18.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311873+0000", 2024-09-18T00:20:18.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029293013999999999 2024-09-18T00:20:18.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.311892+0000", 2024-09-18T00:20:18.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8556e-05 2024-09-18T00:20:18.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.312105+0000", 2024-09-18T00:20:18.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021345199999999999 2024-09-18T00:20:18.495 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.495 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.495 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.495 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:5714 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3801088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-18T00:20:18.495 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:20:18.495 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.171558706, 2024-09-18T00:20:18.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027448727999999999, 2024-09-18T00:20:18.495 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.495 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.496 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:18.496 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.496 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.496 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.496 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5714 2024-09-18T00:20:18.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.496 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:20:18.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310207+0000", 2024-09-18T00:20:18.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.310211+0000", 2024-09-18T00:20:18.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5639999999999998e-06 2024-09-18T00:20:18.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312678+0000", 2024-09-18T00:20:18.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002466761 2024-09-18T00:20:18.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312746+0000", 2024-09-18T00:20:18.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8046000000000005e-05 2024-09-18T00:20:18.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312774+0000", 2024-09-18T00:20:18.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7813e-05 2024-09-18T00:20:18.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.312927+0000", 2024-09-18T00:20:18.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015328499999999999 2024-09-18T00:20:18.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313214+0000", 2024-09-18T00:20:18.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028683099999999998 2024-09-18T00:20:18.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:18.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.313708+0000", 2024-09-18T00:20:18.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049379100000000004 2024-09-18T00:20:18.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:18.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.336895+0000", 2024-09-18T00:20:18.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023186801 2024-09-18T00:20:18.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:18.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337454+0000", 2024-09-18T00:20:18.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055979399999999996 2024-09-18T00:20:18.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:18.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337571+0000", 2024-09-18T00:20:18.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116821 2024-09-18T00:20:18.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:25.337656+0000", 2024-09-18T00:20:18.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4221000000000006e-05 2024-09-18T00:20:18.501 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.501 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.501 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.501 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:6801 3.10 3:0f273ec3:::rbd_data.112a3a39fc45.0000000000000019:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-18T00:20:18.501 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:20:18.501 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 112.444722258, 2024-09-18T00:20:18.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051953602000000002, 2024-09-18T00:20:18.501 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.501 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.501 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:18.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.502 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6801 2024-09-18T00:20:18.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.502 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:20:18.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037043+0000", 2024-09-18T00:20:18.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037046+0000", 2024-09-18T00:20:18.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.19e-06 2024-09-18T00:20:18.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037062+0000", 2024-09-18T00:20:18.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586999999999998e-05 2024-09-18T00:20:18.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037096+0000", 2024-09-18T00:20:18.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4112000000000001e-05 2024-09-18T00:20:18.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037120+0000", 2024-09-18T00:20:18.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.355e-05 2024-09-18T00:20:18.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037246+0000", 2024-09-18T00:20:18.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012652200000000001 2024-09-18T00:20:18.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.037385+0000", 2024-09-18T00:20:18.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013924599999999999 2024-09-18T00:20:18.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-18T00:20:18.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.040881+0000", 2024-09-18T00:20:18.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034954830000000002 2024-09-18T00:20:18.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:18.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088794+0000", 2024-09-18T00:20:18.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047912897000000003 2024-09-18T00:20:18.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:18.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088828+0000", 2024-09-18T00:20:18.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3880000000000001e-05 2024-09-18T00:20:18.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:18.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088916+0000", 2024-09-18T00:20:18.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.865e-05 2024-09-18T00:20:18.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088997+0000", 2024-09-18T00:20:18.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0484999999999999e-05 2024-09-18T00:20:18.507 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.507 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.507 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.507 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:6801 3.10 e42/23 ondisk, result = 0)", 2024-09-18T00:20:18.507 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:20:18.507 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 112.430914848, 2024-09-18T00:20:18.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037960394000000001, 2024-09-18T00:20:18.507 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.507 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.508 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.508 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:20:18.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050851+0000", 2024-09-18T00:20:18.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050852+0000", 2024-09-18T00:20:18.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7799999999999999e-06 2024-09-18T00:20:18.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050860+0000", 2024-09-18T00:20:18.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7729999999999999e-06 2024-09-18T00:20:18.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050891+0000", 2024-09-18T00:20:18.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0617000000000002e-05 2024-09-18T00:20:18.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.050910+0000", 2024-09-18T00:20:18.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.944e-05 2024-09-18T00:20:18.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088766+0000", 2024-09-18T00:20:18.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037855633999999999 2024-09-18T00:20:18.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088784+0000", 2024-09-18T00:20:18.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7767000000000001e-05 2024-09-18T00:20:18.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:26.088811+0000", 2024-09-18T00:20:18.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7382999999999999e-05 2024-09-18T00:20:18.511 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.511 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.511 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.511 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12594 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:18.511 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:20:18.511 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 102.09013831, 2024-09-18T00:20:18.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026492600000000002, 2024-09-18T00:20:18.512 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.512 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.512 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.512 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12594 2024-09-18T00:20:18.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.512 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:20:18.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391627+0000", 2024-09-18T00:20:18.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391629+0000", 2024-09-18T00:20:18.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.86e-06 2024-09-18T00:20:18.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391637+0000", 2024-09-18T00:20:18.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3999999999999992e-06 2024-09-18T00:20:18.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391664+0000", 2024-09-18T00:20:18.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6536000000000001e-05 2024-09-18T00:20:18.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391681+0000", 2024-09-18T00:20:18.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6563e-05 2024-09-18T00:20:18.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417791+0000", 2024-09-18T00:20:18.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026110936000000001 2024-09-18T00:20:18.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.417933+0000", 2024-09-18T00:20:18.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014112099999999999 2024-09-18T00:20:18.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418120+0000", 2024-09-18T00:20:18.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018718399999999999 2024-09-18T00:20:18.516 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.516 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.516 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.516 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12595 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:18.516 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:20:18.516 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 102.090062112, 2024-09-18T00:20:18.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026819428999999999, 2024-09-18T00:20:18.516 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.516 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.516 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.516 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.516 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.517 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.517 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12595 2024-09-18T00:20:18.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.517 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:20:18.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391703+0000", 2024-09-18T00:20:18.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391705+0000", 2024-09-18T00:20:18.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3999999999999999e-06 2024-09-18T00:20:18.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391711+0000", 2024-09-18T00:20:18.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7459999999999997e-06 2024-09-18T00:20:18.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391733+0000", 2024-09-18T00:20:18.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2923000000000001e-05 2024-09-18T00:20:18.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391748+0000", 2024-09-18T00:20:18.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4790000000000001e-05 2024-09-18T00:20:18.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418180+0000", 2024-09-18T00:20:18.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026431888000000001 2024-09-18T00:20:18.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418328+0000", 2024-09-18T00:20:18.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000147725 2024-09-18T00:20:18.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418523+0000", 2024-09-18T00:20:18.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000194957 2024-09-18T00:20:18.520 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.520 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.520 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.520 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12596 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:18.520 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:20:18.521 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 102.089999144, 2024-09-18T00:20:18.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027220296000000001, 2024-09-18T00:20:18.521 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.521 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.521 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.521 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.521 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.521 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.521 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12596 2024-09-18T00:20:18.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.521 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:20:18.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391766+0000", 2024-09-18T00:20:18.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391768+0000", 2024-09-18T00:20:18.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.283e-06 2024-09-18T00:20:18.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391773+0000", 2024-09-18T00:20:18.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3970000000000002e-06 2024-09-18T00:20:18.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391795+0000", 2024-09-18T00:20:18.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1835e-05 2024-09-18T00:20:18.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391809+0000", 2024-09-18T00:20:18.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4107000000000001e-05 2024-09-18T00:20:18.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418582+0000", 2024-09-18T00:20:18.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.026772918999999999 2024-09-18T00:20:18.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418780+0000", 2024-09-18T00:20:18.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000198042 2024-09-18T00:20:18.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.418987+0000", 2024-09-18T00:20:18.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000206713 2024-09-18T00:20:18.525 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.525 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.525 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.525 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12597 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:18.525 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:20:18.525 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 102.08993862600001, 2024-09-18T00:20:18.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027612714, 2024-09-18T00:20:18.525 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.525 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.525 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.526 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.526 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12597 2024-09-18T00:20:18.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.526 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:20:18.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391827+0000", 2024-09-18T00:20:18.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391828+0000", 2024-09-18T00:20:18.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.203e-06 2024-09-18T00:20:18.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391834+0000", 2024-09-18T00:20:18.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6529999999999999e-06 2024-09-18T00:20:18.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391856+0000", 2024-09-18T00:20:18.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2195999999999999e-05 2024-09-18T00:20:18.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391870+0000", 2024-09-18T00:20:18.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4063000000000001e-05 2024-09-18T00:20:18.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419049+0000", 2024-09-18T00:20:18.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027178751000000001 2024-09-18T00:20:18.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419239+0000", 2024-09-18T00:20:18.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019039400000000001 2024-09-18T00:20:18.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419440+0000", 2024-09-18T00:20:18.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000200454 2024-09-18T00:20:18.529 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.529 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.530 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.530 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12598 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:18.530 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:20:18.530 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 102.08987557499999, 2024-09-18T00:20:18.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028269056000000001, 2024-09-18T00:20:18.530 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.530 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.530 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.530 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.530 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.530 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.531 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12598 2024-09-18T00:20:18.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.531 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:20:18.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391890+0000", 2024-09-18T00:20:18.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391891+0000", 2024-09-18T00:20:18.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.257e-06 2024-09-18T00:20:18.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391897+0000", 2024-09-18T00:20:18.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9560000000000002e-06 2024-09-18T00:20:18.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391920+0000", 2024-09-18T00:20:18.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3025999999999999e-05 2024-09-18T00:20:18.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391935+0000", 2024-09-18T00:20:18.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4853e-05 2024-09-18T00:20:18.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419527+0000", 2024-09-18T00:20:18.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027592378000000001 2024-09-18T00:20:18.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.419681+0000", 2024-09-18T00:20:18.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015316 2024-09-18T00:20:18.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420159+0000", 2024-09-18T00:20:18.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000478426 2024-09-18T00:20:18.534 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.534 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.534 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.534 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12599 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:18.534 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:20:18.534 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 102.08981172, 2024-09-18T00:20:18.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028649857000000001, 2024-09-18T00:20:18.534 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.534 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.535 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.535 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.535 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.535 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.535 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12599 2024-09-18T00:20:18.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.535 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:20:18.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391954+0000", 2024-09-18T00:20:18.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391955+0000", 2024-09-18T00:20:18.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-06 2024-09-18T00:20:18.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391961+0000", 2024-09-18T00:20:18.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7100000000000004e-06 2024-09-18T00:20:18.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391982+0000", 2024-09-18T00:20:18.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1492e-05 2024-09-18T00:20:18.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.391996+0000", 2024-09-18T00:20:18.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4194e-05 2024-09-18T00:20:18.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420237+0000", 2024-09-18T00:20:18.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028240818000000001 2024-09-18T00:20:18.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420430+0000", 2024-09-18T00:20:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019228299999999999 2024-09-18T00:20:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420604+0000", 2024-09-18T00:20:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000174117 2024-09-18T00:20:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12600 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 270336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:20:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 102.08975072600001, 2024-09-18T00:20:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028964586, 2024-09-18T00:20:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12600 2024-09-18T00:20:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:20:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392015+0000", 2024-09-18T00:20:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392016+0000", 2024-09-18T00:20:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.31e-06 2024-09-18T00:20:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392022+0000", 2024-09-18T00:20:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5600000000000001e-06 2024-09-18T00:20:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392045+0000", 2024-09-18T00:20:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3595999999999999e-05 2024-09-18T00:20:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392060+0000", 2024-09-18T00:20:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4727e-05 2024-09-18T00:20:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420669+0000", 2024-09-18T00:20:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028608855999999998 2024-09-18T00:20:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420813+0000", 2024-09-18T00:20:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014429500000000001 2024-09-18T00:20:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.420979+0000", 2024-09-18T00:20:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016624200000000001 2024-09-18T00:20:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12601 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 311296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:20:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 102.089686821, 2024-09-18T00:20:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029292832000000001, 2024-09-18T00:20:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12601 2024-09-18T00:20:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:20:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392079+0000", 2024-09-18T00:20:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392080+0000", 2024-09-18T00:20:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2100000000000001e-06 2024-09-18T00:20:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392086+0000", 2024-09-18T00:20:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7570000000000001e-06 2024-09-18T00:20:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392105+0000", 2024-09-18T00:20:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9415999999999998e-05 2024-09-18T00:20:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392132+0000", 2024-09-18T00:20:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7019e-05 2024-09-18T00:20:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421049+0000", 2024-09-18T00:20:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.028916659000000001 2024-09-18T00:20:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421204+0000", 2024-09-18T00:20:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155224 2024-09-18T00:20:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421372+0000", 2024-09-18T00:20:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016754699999999999 2024-09-18T00:20:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12602 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:20:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 102.08961419000001, 2024-09-18T00:20:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029612833000000002, 2024-09-18T00:20:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12602 2024-09-18T00:20:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:20:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392151+0000", 2024-09-18T00:20:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392152+0000", 2024-09-18T00:20:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0470000000000001e-06 2024-09-18T00:20:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392158+0000", 2024-09-18T00:20:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9599999999999997e-06 2024-09-18T00:20:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392181+0000", 2024-09-18T00:20:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2629e-05 2024-09-18T00:20:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392222+0000", 2024-09-18T00:20:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0772000000000002e-05 2024-09-18T00:20:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421441+0000", 2024-09-18T00:20:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029219327 2024-09-18T00:20:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421608+0000", 2024-09-18T00:20:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000166813 2024-09-18T00:20:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421764+0000", 2024-09-18T00:20:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015628500000000001 2024-09-18T00:20:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12603 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:20:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 102.08952379599999, 2024-09-18T00:20:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029902524, 2024-09-18T00:20:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12603 2024-09-18T00:20:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:20:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392242+0000", 2024-09-18T00:20:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392243+0000", 2024-09-18T00:20:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:20:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392260+0000", 2024-09-18T00:20:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6860000000000001e-05 2024-09-18T00:20:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392282+0000", 2024-09-18T00:20:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2713000000000001e-05 2024-09-18T00:20:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392297+0000", 2024-09-18T00:20:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4375e-05 2024-09-18T00:20:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421830+0000", 2024-09-18T00:20:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029532668000000002 2024-09-18T00:20:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.421985+0000", 2024-09-18T00:20:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000155613 2024-09-18T00:20:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422144+0000", 2024-09-18T00:20:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000159102 2024-09-18T00:20:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12604 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:20:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 102.089448745, 2024-09-18T00:20:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030228695, 2024-09-18T00:20:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12604 2024-09-18T00:20:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:20:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392317+0000", 2024-09-18T00:20:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392318+0000", 2024-09-18T00:20:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1930000000000001e-06 2024-09-18T00:20:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392337+0000", 2024-09-18T00:20:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8879999999999999e-05 2024-09-18T00:20:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392364+0000", 2024-09-18T00:20:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7288999999999999e-05 2024-09-18T00:20:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392379+0000", 2024-09-18T00:20:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:20:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422209+0000", 2024-09-18T00:20:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.029829050999999999 2024-09-18T00:20:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422388+0000", 2024-09-18T00:20:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017988699999999999 2024-09-18T00:20:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422545+0000", 2024-09-18T00:20:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015701199999999999 2024-09-18T00:20:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12605 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:20:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 102.089364184, 2024-09-18T00:20:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030488674, 2024-09-18T00:20:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12605 2024-09-18T00:20:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:20:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392401+0000", 2024-09-18T00:20:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392403+0000", 2024-09-18T00:20:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.883e-06 2024-09-18T00:20:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392421+0000", 2024-09-18T00:20:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7526000000000001e-05 2024-09-18T00:20:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392450+0000", 2024-09-18T00:20:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8972e-05 2024-09-18T00:20:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392488+0000", 2024-09-18T00:20:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8522000000000001e-05 2024-09-18T00:20:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422614+0000", 2024-09-18T00:20:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.030125945000000001 2024-09-18T00:20:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422752+0000", 2024-09-18T00:20:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000138178 2024-09-18T00:20:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.422890+0000", 2024-09-18T00:20:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013764800000000001 2024-09-18T00:20:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:12606 3.10 3:0eaf4bd9:::rbd_data.112ab1fa102a.0000000000000005:9 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T00:20:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:20:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 102.089255524, 2024-09-18T00:20:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033727073000000003, 2024-09-18T00:20:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12606 2024-09-18T00:20:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:20:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392510+0000", 2024-09-18T00:20:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392511+0000", 2024-09-18T00:20:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3090000000000001e-06 2024-09-18T00:20:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392517+0000", 2024-09-18T00:20:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.993e-06 2024-09-18T00:20:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392543+0000", 2024-09-18T00:20:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6152999999999998e-05 2024-09-18T00:20:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.392558+0000", 2024-09-18T00:20:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4610000000000001e-05 2024-09-18T00:20:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.425959+0000", 2024-09-18T00:20:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.033400484000000001 2024-09-18T00:20:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426106+0000", 2024-09-18T00:20:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014779800000000001 2024-09-18T00:20:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:36.426237+0000", 2024-09-18T00:20:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013072600000000001 2024-09-18T00:20:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17562 3.a 3:5578626c:::rbd_data.112a6efb7564.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T00:20:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:20:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 88.523790363000003, 2024-09-18T00:20:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052733453999999999, 2024-09-18T00:20:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17562 2024-09-18T00:20:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:20:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957975+0000", 2024-09-18T00:20:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957977+0000", 2024-09-18T00:20:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.119e-06 2024-09-18T00:20:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.957988+0000", 2024-09-18T00:20:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0557000000000001e-05 2024-09-18T00:20:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958017+0000", 2024-09-18T00:20:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9156e-05 2024-09-18T00:20:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958036+0000", 2024-09-18T00:20:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8919e-05 2024-09-18T00:20:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958144+0000", 2024-09-18T00:20:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000107905 2024-09-18T00:20:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.958284+0000", 2024-09-18T00:20:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140216 2024-09-18T00:20:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.960572+0000", 2024-09-18T00:20:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022881860000000002 2024-09-18T00:20:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010537+0000", 2024-09-18T00:20:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049964664999999998 2024-09-18T00:20:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010590+0000", 2024-09-18T00:20:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3094999999999999e-05 2024-09-18T00:20:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010642+0000", 2024-09-18T00:20:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2073000000000001e-05 2024-09-18T00:20:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010709+0000", 2024-09-18T00:20:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6563000000000006e-05 2024-09-18T00:20:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17562 3.a e60/23 ondisk, result = 0)", 2024-09-18T00:20:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:20:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 88.513586992, 2024-09-18T00:20:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042539307999999998, 2024-09-18T00:20:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:20:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968179+0000", 2024-09-18T00:20:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968180+0000", 2024-09-18T00:20:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-18T00:20:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968188+0000", 2024-09-18T00:20:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7419999999999997e-06 2024-09-18T00:20:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968217+0000", 2024-09-18T00:20:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8926999999999999e-05 2024-09-18T00:20:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.968231+0000", 2024-09-18T00:20:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4168999999999999e-05 2024-09-18T00:20:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010575+0000", 2024-09-18T00:20:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.042343923999999998 2024-09-18T00:20:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010584+0000", 2024-09-18T00:20:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-18T00:20:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:50.010718+0000", 2024-09-18T00:20:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013344499999999999 2024-09-18T00:20:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.581 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:18.582 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T00:20:18.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T00:20:18.709 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:18.709 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:18.709 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:20:18.709 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:18.723 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T00:20:18.844 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:18.844 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:18.844 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:20:18.844 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:20:18.845 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:18.858 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T00:20:18.984 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:18.984 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:20:18.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:20:18.984 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:20:18.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.984 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2228 3.15 3:aa2fc059:::rbd_data.112aa7ba4578.000000000000014a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T00:20:18.984 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:20:18.984 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.717566718, 2024-09-18T00:20:18.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.068966377999999995, 2024-09-18T00:20:18.985 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.985 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.985 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:18.985 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.985 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.985 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.985 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2228 2024-09-18T00:20:18.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.985 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:20:18.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263097+0000", 2024-09-18T00:20:18.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263099+0000", 2024-09-18T00:20:18.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2359999999999999e-06 2024-09-18T00:20:18.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263115+0000", 2024-09-18T00:20:18.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5996999999999999e-05 2024-09-18T00:20:18.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263156+0000", 2024-09-18T00:20:18.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1066000000000003e-05 2024-09-18T00:20:18.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263183+0000", 2024-09-18T00:20:18.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7233000000000002e-05 2024-09-18T00:20:18.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263304+0000", 2024-09-18T00:20:18.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121209 2024-09-18T00:20:18.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.263528+0000", 2024-09-18T00:20:18.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022401800000000001 2024-09-18T00:20:18.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:20:18.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.266458+0000", 2024-09-18T00:20:18.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029296980000000001 2024-09-18T00:20:18.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:18.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331628+0000", 2024-09-18T00:20:18.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.065170045999999995 2024-09-18T00:20:18.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:18.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331819+0000", 2024-09-18T00:20:18.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000190819 2024-09-18T00:20:18.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:18.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.331971+0000", 2024-09-18T00:20:18.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000151972 2024-09-18T00:20:18.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:20.332063+0000", 2024-09-18T00:20:18.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2083999999999995e-05 2024-09-18T00:20:18.990 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.990 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.990 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.991 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:2854 3.f 3:f02b0bca:::rbd_data.112aa7ba4578.0000000000000135:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-18T00:20:18.991 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:20:18.991 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 117.110594228, 2024-09-18T00:20:18.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.082582802999999996, 2024-09-18T00:20:18.991 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.991 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.991 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:18.991 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:18.991 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:18.991 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:18.991 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 2854 2024-09-18T00:20:18.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.992 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:20:18.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870069+0000", 2024-09-18T00:20:18.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870071+0000", 2024-09-18T00:20:18.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.227e-06 2024-09-18T00:20:18.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870083+0000", 2024-09-18T00:20:18.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1909999999999999e-05 2024-09-18T00:20:18.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870114+0000", 2024-09-18T00:20:18.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0576e-05 2024-09-18T00:20:18.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870129+0000", 2024-09-18T00:20:18.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.485e-05 2024-09-18T00:20:18.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:18.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870256+0000", 2024-09-18T00:20:18.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012691300000000001 2024-09-18T00:20:18.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:18.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.870493+0000", 2024-09-18T00:20:18.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023737399999999999 2024-09-18T00:20:18.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:18.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.874933+0000", 2024-09-18T00:20:18.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044398299999999996 2024-09-18T00:20:18.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:18.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952452+0000", 2024-09-18T00:20:18.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.077518918000000006 2024-09-18T00:20:18.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:18.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952486+0000", 2024-09-18T00:20:18.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4035999999999997e-05 2024-09-18T00:20:18.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:18.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952575+0000", 2024-09-18T00:20:18.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.954e-05 2024-09-18T00:20:18.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:18.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952652+0000", 2024-09-18T00:20:18.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6629000000000001e-05 2024-09-18T00:20:18.996 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.996 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:18.996 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:18.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.997 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:2854 3.f e38/32 ondisk, result = 0)", 2024-09-18T00:20:18.997 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:20:18.997 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 117.093063413, 2024-09-18T00:20:18.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064868625999999999, 2024-09-18T00:20:18.997 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:18.997 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:18.997 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:18.997 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:18.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:18.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:20:18.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:18.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887600+0000", 2024-09-18T00:20:18.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:18.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:18.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887602+0000", 2024-09-18T00:20:18.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.677e-06 2024-09-18T00:20:18.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:18.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887609+0000", 2024-09-18T00:20:18.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6729999999999992e-06 2024-09-18T00:20:18.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:18.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887637+0000", 2024-09-18T00:20:18.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8206000000000001e-05 2024-09-18T00:20:18.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:18.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:18.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:18.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.887651+0000", 2024-09-18T00:20:18.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3387000000000001e-05 2024-09-18T00:20:18.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952422+0000", 2024-09-18T00:20:19.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064771366999999996 2024-09-18T00:20:19.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952441+0000", 2024-09-18T00:20:19.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8737e-05 2024-09-18T00:20:19.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:21.952469+0000", 2024-09-18T00:20:19.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7579000000000001e-05 2024-09-18T00:20:19.001 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.001 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.001 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.001 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15879 3.16 3:692b47b6:::rbd_object_map.112a13ec2d6c.000000000000000b:head [call] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:20:19.001 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:20:19.001 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.325899505999999, 2024-09-18T00:20:19.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.011184018, 2024-09-18T00:20:19.001 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.001 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.001 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.001 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.002 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.002 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.002 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15879 2024-09-18T00:20:19.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.002 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:20:19.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654764+0000", 2024-09-18T00:20:19.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654768+0000", 2024-09-18T00:20:19.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9129999999999996e-06 2024-09-18T00:20:19.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654786+0000", 2024-09-18T00:20:19.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8695999999999999e-05 2024-09-18T00:20:19.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654832+0000", 2024-09-18T00:20:19.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5735000000000002e-05 2024-09-18T00:20:19.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654860+0000", 2024-09-18T00:20:19.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7789000000000001e-05 2024-09-18T00:20:19.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655048+0000", 2024-09-18T00:20:19.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018781900000000001 2024-09-18T00:20:19.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:20:19.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655070+0000", 2024-09-18T00:20:19.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1916000000000002e-05 2024-09-18T00:20:19.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663335+0000", 2024-09-18T00:20:19.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0082651210000000003 2024-09-18T00:20:19.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:19.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663358+0000", 2024-09-18T00:20:19.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2986e-05 2024-09-18T00:20:19.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663739+0000", 2024-09-18T00:20:19.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0003814820000001 2024-09-18T00:20:19.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.663920+0000", 2024-09-18T00:20:19.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018116900000000001 2024-09-18T00:20:19.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.665948+0000", 2024-09-18T00:20:19.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020273919999999998 2024-09-18T00:20:19.007 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.007 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.007 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.007 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:15881 3.16 3:6cc14920:::rbd_header.112a3285400b:head [watch reconnect cookie 140685950184608 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-18T00:20:19.007 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:20:19.007 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.325738510999997, 2024-09-18T00:20:19.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0138723599999999, 2024-09-18T00:20:19.007 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.007 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.007 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.007 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.008 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.008 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.008 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 15881 2024-09-18T00:20:19.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.008 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:20:19.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654925+0000", 2024-09-18T00:20:19.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654926+0000", 2024-09-18T00:20:19.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6470000000000001e-06 2024-09-18T00:20:19.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654935+0000", 2024-09-18T00:20:19.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8859999999999993e-06 2024-09-18T00:20:19.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654960+0000", 2024-09-18T00:20:19.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4689999999999999e-05 2024-09-18T00:20:19.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.654976+0000", 2024-09-18T00:20:19.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6319000000000001e-05 2024-09-18T00:20:19.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655238+0000", 2024-09-18T00:20:19.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026145999999999998 2024-09-18T00:20:19.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:19.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.655255+0000", 2024-09-18T00:20:19.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7249999999999999e-05 2024-09-18T00:20:19.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663416+0000", 2024-09-18T00:20:19.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081607959999999997 2024-09-18T00:20:19.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:19.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:44.663436+0000", 2024-09-18T00:20:19.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0384e-05 2024-09-18T00:20:19.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666184+0000", 2024-09-18T00:20:19.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002748121 2024-09-18T00:20:19.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.666452+0000", 2024-09-18T00:20:19.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026801700000000001 2024-09-18T00:20:19.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:45.668797+0000", 2024-09-18T00:20:19.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023447899999999998 2024-09-18T00:20:19.012 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.013 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.013 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.013 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:17089 3.f 3:f7681386:::rbd_data.112a6efb7564.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-18T00:20:19.013 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:20:19.013 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 90.100811414999995, 2024-09-18T00:20:19.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.12573889399999999, 2024-09-18T00:20:19.013 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.013 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.013 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:19.013 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.013 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.014 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.014 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 17089 2024-09-18T00:20:19.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.014 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:20:19.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879852+0000", 2024-09-18T00:20:19.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879854+0000", 2024-09-18T00:20:19.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5170000000000002e-06 2024-09-18T00:20:19.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879867+0000", 2024-09-18T00:20:19.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3103e-05 2024-09-18T00:20:19.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879898+0000", 2024-09-18T00:20:19.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0103e-05 2024-09-18T00:20:19.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879912+0000", 2024-09-18T00:20:19.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4636e-05 2024-09-18T00:20:19.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.879992+0000", 2024-09-18T00:20:19.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9814000000000005e-05 2024-09-18T00:20:19.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.880101+0000", 2024-09-18T00:20:19.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109172 2024-09-18T00:20:19.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:19.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.887269+0000", 2024-09-18T00:20:19.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071679300000000003 2024-09-18T00:20:19.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:19.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005269+0000", 2024-09-18T00:20:19.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.117999621 2024-09-18T00:20:19.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:19.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005317+0000", 2024-09-18T00:20:19.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8279000000000003e-05 2024-09-18T00:20:19.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:19.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005464+0000", 2024-09-18T00:20:19.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014661200000000001 2024-09-18T00:20:19.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005591+0000", 2024-09-18T00:20:19.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000127107 2024-09-18T00:20:19.018 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.019 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.019 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.019 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:17089 3.f e59/32 ondisk, result = 0)", 2024-09-18T00:20:19.019 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:20:19.019 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 90.076704550000002, 2024-09-18T00:20:19.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101334403, 2024-09-18T00:20:19.019 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.019 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.019 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.019 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:20:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903959+0000", 2024-09-18T00:20:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903960+0000", 2024-09-18T00:20:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.33e-06 2024-09-18T00:20:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903966+0000", 2024-09-18T00:20:19.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5929999999999996e-06 2024-09-18T00:20:19.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903986+0000", 2024-09-18T00:20:19.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.018e-05 2024-09-18T00:20:19.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:48.903997+0000", 2024-09-18T00:20:19.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0988999999999999e-05 2024-09-18T00:20:19.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005171+0000", 2024-09-18T00:20:19.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101174267 2024-09-18T00:20:19.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005200+0000", 2024-09-18T00:20:19.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8540000000000001e-05 2024-09-18T00:20:19.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:18:49.005293+0000", 2024-09-18T00:20:19.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504000000000005e-05 2024-09-18T00:20:19.023 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.023 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.023 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.023 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24529 3.16 3:6857ce1a:::rbd_data.112a990280ef.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:20:19.023 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:20:19.023 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 72.343893139000002, 2024-09-18T00:20:19.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.10693805200000001, 2024-09-18T00:20:19.023 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.023 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.024 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:19.024 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.024 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.024 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.024 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24529 2024-09-18T00:20:19.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.024 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:20:19.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636770+0000", 2024-09-18T00:20:19.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636772+0000", 2024-09-18T00:20:19.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1100000000000001e-06 2024-09-18T00:20:19.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636783+0000", 2024-09-18T00:20:19.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1005999999999999e-05 2024-09-18T00:20:19.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636814+0000", 2024-09-18T00:20:19.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1047000000000003e-05 2024-09-18T00:20:19.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636829+0000", 2024-09-18T00:20:19.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4363e-05 2024-09-18T00:20:19.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.636980+0000", 2024-09-18T00:20:19.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015181099999999999 2024-09-18T00:20:19.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.637237+0000", 2024-09-18T00:20:19.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000256352 2024-09-18T00:20:19.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:19.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743536+0000", 2024-09-18T00:20:19.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.106298825 2024-09-18T00:20:19.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:19.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743599+0000", 2024-09-18T00:20:19.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3449999999999997e-05 2024-09-18T00:20:19.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:06.743708+0000", 2024-09-18T00:20:19.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109088 2024-09-18T00:20:19.028 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.028 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.028 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.028 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24805 3.16 3:6de5e15d:::rbd_data.112a990280ef.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:20:19.028 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:20:19.028 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 71.758089906999999, 2024-09-18T00:20:19.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14775107700000001, 2024-09-18T00:20:19.029 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.029 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.029 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:19.029 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.029 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.029 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.029 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24805 2024-09-18T00:20:19.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.029 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:20:19.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222573+0000", 2024-09-18T00:20:19.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222574+0000", 2024-09-18T00:20:19.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0210000000000001e-06 2024-09-18T00:20:19.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222581+0000", 2024-09-18T00:20:19.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.144e-06 2024-09-18T00:20:19.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222595+0000", 2024-09-18T00:20:19.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4567e-05 2024-09-18T00:20:19.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222603+0000", 2024-09-18T00:20:19.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6529999999999994e-06 2024-09-18T00:20:19.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222687+0000", 2024-09-18T00:20:19.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4587999999999997e-05 2024-09-18T00:20:19.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.222828+0000", 2024-09-18T00:20:19.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000140178 2024-09-18T00:20:19.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:19.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370231+0000", 2024-09-18T00:20:19.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.14740378700000001 2024-09-18T00:20:19.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:19.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370269+0000", 2024-09-18T00:20:19.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7886999999999998e-05 2024-09-18T00:20:19.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.370324+0000", 2024-09-18T00:20:19.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5251999999999999e-05 2024-09-18T00:20:19.033 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.033 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.033 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.033 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:24852 3.13 3:cc64f0fd:::rbd_data.112a990280ef.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-18T00:20:19.033 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:20:19.034 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 71.525350437, 2024-09-18T00:20:19.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.066392943999999995, 2024-09-18T00:20:19.034 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.034 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.034 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:19.034 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.034 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.034 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.034 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 24852 2024-09-18T00:20:19.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.034 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:20:19.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455313+0000", 2024-09-18T00:20:19.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455315+0000", 2024-09-18T00:20:19.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.705e-06 2024-09-18T00:20:19.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455324+0000", 2024-09-18T00:20:19.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8120000000000002e-06 2024-09-18T00:20:19.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455349+0000", 2024-09-18T00:20:19.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4250999999999999e-05 2024-09-18T00:20:19.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455361+0000", 2024-09-18T00:20:19.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2805e-05 2024-09-18T00:20:19.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455425+0000", 2024-09-18T00:20:19.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3981999999999998e-05 2024-09-18T00:20:19.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.455480+0000", 2024-09-18T00:20:19.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4792e-05 2024-09-18T00:20:19.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:19.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.460671+0000", 2024-09-18T00:20:19.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051907300000000002 2024-09-18T00:20:19.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:19.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521355+0000", 2024-09-18T00:20:19.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060684208000000003 2024-09-18T00:20:19.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:19.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521493+0000", 2024-09-18T00:20:19.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013803200000000001 2024-09-18T00:20:19.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:19.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521590+0000", 2024-09-18T00:20:19.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6339e-05 2024-09-18T00:20:19.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:07.521706+0000", 2024-09-18T00:20:19.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116288 2024-09-18T00:20:19.043 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.043 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.043 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.043 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:25269 3.f 3:f6adfd48:::rbd_data.112aa34d8ec3.0000000000000145:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T00:20:19.043 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:20:19.043 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.375760475999996, 2024-09-18T00:20:19.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.075209085999999994, 2024-09-18T00:20:19.044 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.044 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.044 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:19.044 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.044 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.044 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.044 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 25269 2024-09-18T00:20:19.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.044 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:20:19.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604903+0000", 2024-09-18T00:20:19.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604904+0000", 2024-09-18T00:20:19.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0890000000000001e-06 2024-09-18T00:20:19.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604910+0000", 2024-09-18T00:20:19.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.964e-06 2024-09-18T00:20:19.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604925+0000", 2024-09-18T00:20:19.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4894e-05 2024-09-18T00:20:19.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604932+0000", 2024-09-18T00:20:19.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-18T00:20:19.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.604990+0000", 2024-09-18T00:20:19.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8143000000000002e-05 2024-09-18T00:20:19.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.605104+0000", 2024-09-18T00:20:19.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011404399999999999 2024-09-18T00:20:19.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:19.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.608755+0000", 2024-09-18T00:20:19.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003651081 2024-09-18T00:20:19.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:19.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679920+0000", 2024-09-18T00:20:19.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.071164543999999996 2024-09-18T00:20:19.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:19.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679978+0000", 2024-09-18T00:20:19.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8585000000000002e-05 2024-09-18T00:20:19.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:19.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680036+0000", 2024-09-18T00:20:19.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7978e-05 2024-09-18T00:20:19.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680112+0000", 2024-09-18T00:20:19.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5527999999999999e-05 2024-09-18T00:20:19.049 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.049 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.049 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.049 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:25269 3.f e72/32 ondisk, result = 0)", 2024-09-18T00:20:19.049 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:20:19.049 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.362550091000003, 2024-09-18T00:20:19.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062014761000000002, 2024-09-18T00:20:19.049 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.049 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.050 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.050 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:20:19.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618113+0000", 2024-09-18T00:20:19.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618114+0000", 2024-09-18T00:20:19.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1650000000000001e-06 2024-09-18T00:20:19.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618119+0000", 2024-09-18T00:20:19.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-18T00:20:19.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618134+0000", 2024-09-18T00:20:19.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5770999999999999e-05 2024-09-18T00:20:19.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.618143+0000", 2024-09-18T00:20:19.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.551e-06 2024-09-18T00:20:19.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679967+0000", 2024-09-18T00:20:19.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061823937000000002 2024-09-18T00:20:19.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.679975+0000", 2024-09-18T00:20:19.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7340000000000008e-06 2024-09-18T00:20:19.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:08.680128+0000", 2024-09-18T00:20:19.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015338299999999999 2024-09-18T00:20:19.053 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.053 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.053 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.053 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48484 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 15704064~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:19.053 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:20:19.053 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.613347982000001, 2024-09-18T00:20:19.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058450876999999998, 2024-09-18T00:20:19.054 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.054 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.054 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.054 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.054 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.054 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.054 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48484 2024-09-18T00:20:19.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.054 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:20:19.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.367315+0000", 2024-09-18T00:20:19.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370429+0000", 2024-09-18T00:20:19.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031138530000000002 2024-09-18T00:20:19.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370435+0000", 2024-09-18T00:20:19.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6269999999999997e-06 2024-09-18T00:20:19.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370455+0000", 2024-09-18T00:20:19.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0426999999999999e-05 2024-09-18T00:20:19.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.370465+0000", 2024-09-18T00:20:19.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0118999999999999e-05 2024-09-18T00:20:19.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425525+0000", 2024-09-18T00:20:19.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055059943 2024-09-18T00:20:19.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425645+0000", 2024-09-18T00:20:19.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119392 2024-09-18T00:20:19.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.425766+0000", 2024-09-18T00:20:19.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012151599999999999 2024-09-18T00:20:19.057 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.057 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.058 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.058 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48503 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16326656~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:19.058 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:20:19.058 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.609467580999997, 2024-09-18T00:20:19.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058310395000000001, 2024-09-18T00:20:19.058 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.058 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.058 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.058 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.058 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.058 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.058 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48503 2024-09-18T00:20:19.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.059 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:20:19.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:20:19.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371196+0000", 2024-09-18T00:20:19.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0100000000000004e-07 2024-09-18T00:20:19.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371199+0000", 2024-09-18T00:20:19.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8899999999999999e-06 2024-09-18T00:20:19.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371211+0000", 2024-09-18T00:20:19.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1707e-05 2024-09-18T00:20:19.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371218+0000", 2024-09-18T00:20:19.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.131e-06 2024-09-18T00:20:19.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429426+0000", 2024-09-18T00:20:19.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058207771999999998 2024-09-18T00:20:19.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429467+0000", 2024-09-18T00:20:19.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1532999999999997e-05 2024-09-18T00:20:19.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429506+0000", 2024-09-18T00:20:19.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8661000000000001e-05 2024-09-18T00:20:19.062 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.062 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.062 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.062 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48504 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16359424~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:19.062 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:20:19.062 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.609435761999997, 2024-09-18T00:20:19.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058412775, 2024-09-18T00:20:19.062 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.063 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.063 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.063 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48504 2024-09-18T00:20:19.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.063 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:20:19.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:20:19.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371228+0000", 2024-09-18T00:20:19.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6899999999999996e-07 2024-09-18T00:20:19.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371231+0000", 2024-09-18T00:20:19.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6709999999999999e-06 2024-09-18T00:20:19.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371240+0000", 2024-09-18T00:20:19.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7309999999999999e-06 2024-09-18T00:20:19.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371249+0000", 2024-09-18T00:20:19.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4280000000000002e-06 2024-09-18T00:20:19.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429533+0000", 2024-09-18T00:20:19.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058283852999999997 2024-09-18T00:20:19.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429590+0000", 2024-09-18T00:20:19.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6824999999999999e-05 2024-09-18T00:20:19.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429640+0000", 2024-09-18T00:20:19.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0498000000000003e-05 2024-09-18T00:20:19.066 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.066 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.066 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.067 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48505 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16392192~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:19.067 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:20:19.067 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.609402789000001, 2024-09-18T00:20:19.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058482777999999999, 2024-09-18T00:20:19.067 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.067 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.067 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.068 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.068 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.068 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48505 2024-09-18T00:20:19.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.068 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:20:19.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:20:19.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371261+0000", 2024-09-18T00:20:19.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7800000000000001e-07 2024-09-18T00:20:19.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371264+0000", 2024-09-18T00:20:19.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-18T00:20:19.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371274+0000", 2024-09-18T00:20:19.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4690000000000003e-06 2024-09-18T00:20:19.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371281+0000", 2024-09-18T00:20:19.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4120000000000004e-06 2024-09-18T00:20:19.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429665+0000", 2024-09-18T00:20:19.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058384108999999997 2024-09-18T00:20:19.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429705+0000", 2024-09-18T00:20:19.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0299000000000002e-05 2024-09-18T00:20:19.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429743+0000", 2024-09-18T00:20:19.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7951000000000003e-05 2024-09-18T00:20:19.071 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.071 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.071 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.071 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48506 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16424960~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:19.071 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:20:19.072 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.609373577, 2024-09-18T00:20:19.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058604485999999997, 2024-09-18T00:20:19.072 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.072 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.072 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.072 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48506 2024-09-18T00:20:19.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.072 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:20:19.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371290+0000", 2024-09-18T00:20:19.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371355+0000", 2024-09-18T00:20:19.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5549000000000005e-05 2024-09-18T00:20:19.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371359+0000", 2024-09-18T00:20:19.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1049999999999997e-06 2024-09-18T00:20:19.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371380+0000", 2024-09-18T00:20:19.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0205000000000001e-05 2024-09-18T00:20:19.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371388+0000", 2024-09-18T00:20:19.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3229999999999998e-06 2024-09-18T00:20:19.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429771+0000", 2024-09-18T00:20:19.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058382878999999999 2024-09-18T00:20:19.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429842+0000", 2024-09-18T00:20:19.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1110000000000002e-05 2024-09-18T00:20:19.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429894+0000", 2024-09-18T00:20:19.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2315000000000003e-05 2024-09-18T00:20:19.075 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.076 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.076 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.076 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:48507 3.11 3:8cab04d6:::rbd_data.112a4b380b1a.0000000000000001:19 [read 16457728~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T00:20:19.076 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:20:19.076 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.609262217000001, 2024-09-18T00:20:19.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058597275999999997, 2024-09-18T00:20:19.076 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.076 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.076 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.077 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 48507 2024-09-18T00:20:19.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.077 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:20:19.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.371401+0000", 2024-09-18T00:20:19.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374394+0000", 2024-09-18T00:20:19.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029924449999999998 2024-09-18T00:20:19.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374400+0000", 2024-09-18T00:20:19.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0229999999999998e-06 2024-09-18T00:20:19.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374419+0000", 2024-09-18T00:20:19.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9488e-05 2024-09-18T00:20:19.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.374432+0000", 2024-09-18T00:20:19.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2492e-05 2024-09-18T00:20:19.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429917+0000", 2024-09-18T00:20:19.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.055485087000000002 2024-09-18T00:20:19.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429958+0000", 2024-09-18T00:20:19.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1258999999999999e-05 2024-09-18T00:20:19.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:26.429998+0000", 2024-09-18T00:20:19.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0482000000000003e-05 2024-09-18T00:20:19.080 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.080 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.080 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.080 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:58332 3.15 3:ac910418:::rbd_data.112a17f95ba0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T00:20:19.080 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:20:19.080 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 39.922758186000003, 2024-09-18T00:20:19.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.071238735999999997, 2024-09-18T00:20:19.080 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.080 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.081 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:19.081 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.081 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.081 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.081 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 58332 2024-09-18T00:20:19.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.081 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:20:19.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057905+0000", 2024-09-18T00:20:19.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057907+0000", 2024-09-18T00:20:19.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1859999999999999e-06 2024-09-18T00:20:19.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057920+0000", 2024-09-18T00:20:19.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3013000000000001e-05 2024-09-18T00:20:19.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057949+0000", 2024-09-18T00:20:19.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8745999999999999e-05 2024-09-18T00:20:19.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.057966+0000", 2024-09-18T00:20:19.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6477000000000002e-05 2024-09-18T00:20:19.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.058035+0000", 2024-09-18T00:20:19.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9560000000000005e-05 2024-09-18T00:20:19.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.058133+0000", 2024-09-18T00:20:19.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8051e-05 2024-09-18T00:20:19.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:20:19.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.061809+0000", 2024-09-18T00:20:19.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003675453 2024-09-18T00:20:19.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:19.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128667+0000", 2024-09-18T00:20:19.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.066857960999999994 2024-09-18T00:20:19.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:19.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128952+0000", 2024-09-18T00:20:19.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028583100000000001 2024-09-18T00:20:19.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:19.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129060+0000", 2024-09-18T00:20:19.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010806000000000001 2024-09-18T00:20:19.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129144+0000", 2024-09-18T00:20:19.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3398000000000003e-05 2024-09-18T00:20:19.086 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.086 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.086 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.086 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:58332 3.15 e96/32 ondisk, result = 0)", 2024-09-18T00:20:19.086 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:20:19.086 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 39.910565357000003, 2024-09-18T00:20:19.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059068112999999998, 2024-09-18T00:20:19.086 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.086 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.086 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.086 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:20:19.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070098+0000", 2024-09-18T00:20:19.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070099+0000", 2024-09-18T00:20:19.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9899999999999999e-07 2024-09-18T00:20:19.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070103+0000", 2024-09-18T00:20:19.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6880000000000001e-06 2024-09-18T00:20:19.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070115+0000", 2024-09-18T00:20:19.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2088e-05 2024-09-18T00:20:19.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.070122+0000", 2024-09-18T00:20:19.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3900000000000004e-06 2024-09-18T00:20:19.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128758+0000", 2024-09-18T00:20:19.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.058635541999999999 2024-09-18T00:20:19.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.128774+0000", 2024-09-18T00:20:19.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6858999999999999e-05 2024-09-18T00:20:19.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.129166+0000", 2024-09-18T00:20:19.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039164700000000001 2024-09-18T00:20:19.090 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.090 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.090 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.090 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.090 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.090 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:19.091 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T00:20:19.220 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:19.220 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:19.220 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-18T00:20:19.220 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:19.235 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T00:20:19.357 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:19.357 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-18T00:20:19.357 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-18T00:20:19.357 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-18T00:20:19.357 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:19.371 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T00:20:19.492 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-18T00:20:19.492 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-18T00:20:19.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-18T00:20:19.492 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-18T00:20:19.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.493 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34543 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:19.493 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:20:19.493 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.198405179999995, 2024-09-18T00:20:19.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023207593, 2024-09-18T00:20:19.493 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.493 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.493 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:19.493 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.493 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.493 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.493 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34543 2024-09-18T00:20:19.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.494 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:20:19.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290780+0000", 2024-09-18T00:20:19.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290783+0000", 2024-09-18T00:20:19.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5240000000000001e-06 2024-09-18T00:20:19.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290799+0000", 2024-09-18T00:20:19.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-18T00:20:19.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290840+0000", 2024-09-18T00:20:19.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0926000000000001e-05 2024-09-18T00:20:19.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290868+0000", 2024-09-18T00:20:19.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8163000000000001e-05 2024-09-18T00:20:19.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291294+0000", 2024-09-18T00:20:19.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00042588700000000001 2024-09-18T00:20:19.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-18T00:20:19.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:20:19.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.986e-05 2024-09-18T00:20:19.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316617+0000", 2024-09-18T00:20:19.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025303583000000001 2024-09-18T00:20:19.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:19.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316637+0000", 2024-09-18T00:20:19.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9388000000000001e-05 2024-09-18T00:20:19.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313933+0000", 2024-09-18T00:20:19.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99729681000000003 2024-09-18T00:20:19.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.313987+0000", 2024-09-18T00:20:19.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4069e-05 2024-09-18T00:20:19.498 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.498 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.499 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.499 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34544 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:19.499 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:20:19.499 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.198256825000001, 2024-09-18T00:20:19.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0232412120000001, 2024-09-18T00:20:19.499 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.499 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.499 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:19.499 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.499 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.499 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.499 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34544 2024-09-18T00:20:19.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.500 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:20:19.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290928+0000", 2024-09-18T00:20:19.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290930+0000", 2024-09-18T00:20:19.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5030000000000001e-06 2024-09-18T00:20:19.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290940+0000", 2024-09-18T00:20:19.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9000000000000001e-06 2024-09-18T00:20:19.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290964+0000", 2024-09-18T00:20:19.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4273e-05 2024-09-18T00:20:19.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.290982+0000", 2024-09-18T00:20:19.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8216e-05 2024-09-18T00:20:19.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291433+0000", 2024-09-18T00:20:19.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000451278 2024-09-18T00:20:19.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:19.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291446+0000", 2024-09-18T00:20:19.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3039e-05 2024-09-18T00:20:19.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316687+0000", 2024-09-18T00:20:19.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025240069 2024-09-18T00:20:19.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:19.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316701+0000", 2024-09-18T00:20:19.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4100000000000001e-05 2024-09-18T00:20:19.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314142+0000", 2024-09-18T00:20:19.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99744119799999997 2024-09-18T00:20:19.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314169+0000", 2024-09-18T00:20:19.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7636000000000001e-05 2024-09-18T00:20:19.504 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.504 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.504 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.505 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34545 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:19.505 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:20:19.505 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.198178303000006, 2024-09-18T00:20:19.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023257802, 2024-09-18T00:20:19.505 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.505 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.505 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:19.505 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.505 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.505 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.505 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34545 2024-09-18T00:20:19.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.505 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:20:19.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291007+0000", 2024-09-18T00:20:19.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291008+0000", 2024-09-18T00:20:19.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.237e-06 2024-09-18T00:20:19.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291015+0000", 2024-09-18T00:20:19.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2030000000000003e-06 2024-09-18T00:20:19.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291035+0000", 2024-09-18T00:20:19.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.978e-05 2024-09-18T00:20:19.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291049+0000", 2024-09-18T00:20:19.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4429e-05 2024-09-18T00:20:19.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291630+0000", 2024-09-18T00:20:19.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00058103199999999999 2024-09-18T00:20:19.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:19.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291644+0000", 2024-09-18T00:20:19.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.342e-05 2024-09-18T00:20:19.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316753+0000", 2024-09-18T00:20:19.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025108703999999999 2024-09-18T00:20:19.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:19.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316766+0000", 2024-09-18T00:20:19.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3237e-05 2024-09-18T00:20:19.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314238+0000", 2024-09-18T00:20:19.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99747232699999999 2024-09-18T00:20:19.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314265+0000", 2024-09-18T00:20:19.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6432999999999999e-05 2024-09-18T00:20:19.510 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.511 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.511 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.511 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34546 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:19.511 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:20:19.511 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.198119964, 2024-09-18T00:20:19.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.023324535, 2024-09-18T00:20:19.511 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.511 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.512 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:19.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.512 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34546 2024-09-18T00:20:19.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.512 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:20:19.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291065+0000", 2024-09-18T00:20:19.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291066+0000", 2024-09-18T00:20:19.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.569999999999999e-07 2024-09-18T00:20:19.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291073+0000", 2024-09-18T00:20:19.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.063e-06 2024-09-18T00:20:19.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291090+0000", 2024-09-18T00:20:19.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6977e-05 2024-09-18T00:20:19.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291105+0000", 2024-09-18T00:20:19.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5166e-05 2024-09-18T00:20:19.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291771+0000", 2024-09-18T00:20:19.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00066599699999999999 2024-09-18T00:20:19.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:19.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291785+0000", 2024-09-18T00:20:19.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.327e-05 2024-09-18T00:20:19.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316826+0000", 2024-09-18T00:20:19.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025041955000000001 2024-09-18T00:20:19.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:19.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316840+0000", 2024-09-18T00:20:19.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3256e-05 2024-09-18T00:20:19.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314335+0000", 2024-09-18T00:20:19.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99749496100000001 2024-09-18T00:20:19.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.314390+0000", 2024-09-18T00:20:19.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4932999999999997e-05 2024-09-18T00:20:19.517 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.517 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.517 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.517 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34547 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:19.517 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:20:19.517 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.198060665, 2024-09-18T00:20:19.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.028510467, 2024-09-18T00:20:19.518 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.518 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.518 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:19.518 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.518 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.518 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.518 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34547 2024-09-18T00:20:19.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.518 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:20:19.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291124+0000", 2024-09-18T00:20:19.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291126+0000", 2024-09-18T00:20:19.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2699999999999999e-06 2024-09-18T00:20:19.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291150+0000", 2024-09-18T00:20:19.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4669999999999999e-05 2024-09-18T00:20:19.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291171+0000", 2024-09-18T00:20:19.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0706000000000001e-05 2024-09-18T00:20:19.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291186+0000", 2024-09-18T00:20:19.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5126999999999999e-05 2024-09-18T00:20:19.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291885+0000", 2024-09-18T00:20:19.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00069850299999999995 2024-09-18T00:20:19.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:19.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291897+0000", 2024-09-18T00:20:19.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.243e-05 2024-09-18T00:20:19.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316887+0000", 2024-09-18T00:20:19.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024990155999999999 2024-09-18T00:20:19.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:19.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316900+0000", 2024-09-18T00:20:19.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.295e-05 2024-09-18T00:20:19.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319594+0000", 2024-09-18T00:20:19.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0026940719999999 2024-09-18T00:20:19.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319635+0000", 2024-09-18T00:20:19.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0583000000000001e-05 2024-09-18T00:20:19.524 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.524 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.524 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.524 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:34548 3.c 3.a410c6c (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-09-18T00:20:19.524 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:20:19.525 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.197913206999999, 2024-09-18T00:20:19.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0285287400000001, 2024-09-18T00:20:19.525 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.525 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.525 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-18T00:20:19.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.525 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 34548 2024-09-18T00:20:19.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.525 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:20:19.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291272+0000", 2024-09-18T00:20:19.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291274+0000", 2024-09-18T00:20:19.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.12e-06 2024-09-18T00:20:19.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291288+0000", 2024-09-18T00:20:19.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4367e-05 2024-09-18T00:20:19.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291314+0000", 2024-09-18T00:20:19.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5698999999999999e-05 2024-09-18T00:20:19.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291333+0000", 2024-09-18T00:20:19.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8783e-05 2024-09-18T00:20:19.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.291987+0000", 2024-09-18T00:20:19.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00065417800000000005 2024-09-18T00:20:19.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-18T00:20:19.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.292000+0000", 2024-09-18T00:20:19.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2652e-05 2024-09-18T00:20:19.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316946+0000", 2024-09-18T00:20:19.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.024946696000000001 2024-09-18T00:20:19.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-18T00:20:19.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:14.316956+0000", 2024-09-18T00:20:19.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2760000000000006e-06 2024-09-18T00:20:19.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319752+0000", 2024-09-18T00:20:19.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.002796083 2024-09-18T00:20:19.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:15.319801+0000", 2024-09-18T00:20:19.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8885999999999999e-05 2024-09-18T00:20:19.530 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.530 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.530 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.530 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41950 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 180224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:19.531 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:20:19.531 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.795260497999998, 2024-09-18T00:20:19.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053368486, 2024-09-18T00:20:19.531 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.531 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.531 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.531 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.531 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.531 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.531 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41950 2024-09-18T00:20:19.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.531 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:20:19.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693925+0000", 2024-09-18T00:20:19.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693926+0000", 2024-09-18T00:20:19.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4900000000000004e-07 2024-09-18T00:20:19.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693930+0000", 2024-09-18T00:20:19.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0130000000000004e-06 2024-09-18T00:20:19.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693940+0000", 2024-09-18T00:20:19.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0618000000000001e-05 2024-09-18T00:20:19.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693948+0000", 2024-09-18T00:20:19.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3140000000000004e-06 2024-09-18T00:20:19.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747133+0000", 2024-09-18T00:20:19.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053184689 2024-09-18T00:20:19.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747203+0000", 2024-09-18T00:20:19.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0339000000000005e-05 2024-09-18T00:20:19.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747293+0000", 2024-09-18T00:20:19.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9563999999999998e-05 2024-09-18T00:20:19.536 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.536 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.536 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.536 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41951 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 190464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:19.536 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:20:19.536 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.795225731000002, 2024-09-18T00:20:19.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053527695, 2024-09-18T00:20:19.537 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.537 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.537 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.537 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.537 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.537 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.537 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41951 2024-09-18T00:20:19.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.537 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:20:19.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693959+0000", 2024-09-18T00:20:19.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693960+0000", 2024-09-18T00:20:19.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.16e-07 2024-09-18T00:20:19.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693971+0000", 2024-09-18T00:20:19.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1204e-05 2024-09-18T00:20:19.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693981+0000", 2024-09-18T00:20:19.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-18T00:20:19.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.693989+0000", 2024-09-18T00:20:19.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0180000000000001e-06 2024-09-18T00:20:19.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747328+0000", 2024-09-18T00:20:19.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053338508 2024-09-18T00:20:19.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747400+0000", 2024-09-18T00:20:19.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2750999999999995e-05 2024-09-18T00:20:19.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747487+0000", 2024-09-18T00:20:19.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6527999999999996e-05 2024-09-18T00:20:19.541 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.541 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.541 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.542 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41952 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:19.542 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:20:19.542 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.795185324000002, 2024-09-18T00:20:19.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053728013999999998, 2024-09-18T00:20:19.542 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.542 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.542 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.542 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.542 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.542 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.543 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41952 2024-09-18T00:20:19.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.543 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:20:19.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694000+0000", 2024-09-18T00:20:19.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694001+0000", 2024-09-18T00:20:19.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3799999999999996e-07 2024-09-18T00:20:19.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694004+0000", 2024-09-18T00:20:19.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.965e-06 2024-09-18T00:20:19.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694013+0000", 2024-09-18T00:20:19.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1689999999999998e-06 2024-09-18T00:20:19.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694022+0000", 2024-09-18T00:20:19.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7560000000000005e-06 2024-09-18T00:20:19.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747531+0000", 2024-09-18T00:20:19.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053508999000000002 2024-09-18T00:20:19.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747610+0000", 2024-09-18T00:20:19.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9212e-05 2024-09-18T00:20:19.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747728+0000", 2024-09-18T00:20:19.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011797499999999999 2024-09-18T00:20:19.547 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.547 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.547 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.547 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41953 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:19.547 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:20:19.547 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.795151695000001, 2024-09-18T00:20:19.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053936783000000002, 2024-09-18T00:20:19.548 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.548 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.548 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.548 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.548 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.548 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.548 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41953 2024-09-18T00:20:19.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.548 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:20:19.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694033+0000", 2024-09-18T00:20:19.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694034+0000", 2024-09-18T00:20:19.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5199999999999996e-07 2024-09-18T00:20:19.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694037+0000", 2024-09-18T00:20:19.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.745e-06 2024-09-18T00:20:19.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694046+0000", 2024-09-18T00:20:19.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2490000000000007e-06 2024-09-18T00:20:19.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694054+0000", 2024-09-18T00:20:19.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1179999999999992e-06 2024-09-18T00:20:19.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747766+0000", 2024-09-18T00:20:19.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053711362999999998 2024-09-18T00:20:19.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747856+0000", 2024-09-18T00:20:19.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0779000000000006e-05 2024-09-18T00:20:19.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.747970+0000", 2024-09-18T00:20:19.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011377700000000001 2024-09-18T00:20:19.555 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.555 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.555 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.556 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41954 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:19.556 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:20:19.556 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.795121541999997, 2024-09-18T00:20:19.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054104898999999998, 2024-09-18T00:20:19.556 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.557 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.557 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.557 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.557 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.557 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.557 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41954 2024-09-18T00:20:19.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.557 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:20:19.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:20:19.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694064+0000", 2024-09-18T00:20:19.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4000000000000002e-07 2024-09-18T00:20:19.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694067+0000", 2024-09-18T00:20:19.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7329999999999999e-06 2024-09-18T00:20:19.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694076+0000", 2024-09-18T00:20:19.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2019999999999993e-06 2024-09-18T00:20:19.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694083+0000", 2024-09-18T00:20:19.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0029999999999997e-06 2024-09-18T00:20:19.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748003+0000", 2024-09-18T00:20:19.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053920463000000002 2024-09-18T00:20:19.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748077+0000", 2024-09-18T00:20:19.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4041000000000005e-05 2024-09-18T00:20:19.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748168+0000", 2024-09-18T00:20:19.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0916999999999997e-05 2024-09-18T00:20:19.561 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.562 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.562 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.562 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41955 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 231424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:19.562 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:20:19.562 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.795092703999998, 2024-09-18T00:20:19.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054272659000000001, 2024-09-18T00:20:19.562 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.562 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.562 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.562 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.563 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.563 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.563 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41955 2024-09-18T00:20:19.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.563 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:20:19.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694092+0000", 2024-09-18T00:20:19.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694093+0000", 2024-09-18T00:20:19.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0699999999999997e-07 2024-09-18T00:20:19.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694109+0000", 2024-09-18T00:20:19.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6061000000000001e-05 2024-09-18T00:20:19.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694121+0000", 2024-09-18T00:20:19.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.216e-05 2024-09-18T00:20:19.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694129+0000", 2024-09-18T00:20:19.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0029999999999994e-06 2024-09-18T00:20:19.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748202+0000", 2024-09-18T00:20:19.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054072571 2024-09-18T00:20:19.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748274+0000", 2024-09-18T00:20:19.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2117000000000001e-05 2024-09-18T00:20:19.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748365+0000", 2024-09-18T00:20:19.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1139999999999998e-05 2024-09-18T00:20:19.567 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.567 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.567 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.567 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41956 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:19.567 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:20:19.567 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.795044496000003, 2024-09-18T00:20:19.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054420496999999998, 2024-09-18T00:20:19.567 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.567 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.567 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.568 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.568 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.568 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.568 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41956 2024-09-18T00:20:19.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.568 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:20:19.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:20:19.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694141+0000", 2024-09-18T00:20:19.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8699999999999995e-07 2024-09-18T00:20:19.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694144+0000", 2024-09-18T00:20:19.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3510000000000002e-06 2024-09-18T00:20:19.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694154+0000", 2024-09-18T00:20:19.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0769999999999997e-06 2024-09-18T00:20:19.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694163+0000", 2024-09-18T00:20:19.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3300000000000005e-06 2024-09-18T00:20:19.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748397+0000", 2024-09-18T00:20:19.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054234497999999999 2024-09-18T00:20:19.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748471+0000", 2024-09-18T00:20:19.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3441000000000004e-05 2024-09-18T00:20:19.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748561+0000", 2024-09-18T00:20:19.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0212999999999998e-05 2024-09-18T00:20:19.572 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.572 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.572 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.572 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:41957 3.5 3:a0b7aa4d:::rbd_data.112a3496573c.0000000000000023:17 [read 251904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:19.572 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:20:19.572 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.795013244000003, 2024-09-18T00:20:19.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054590310000000003, 2024-09-18T00:20:19.572 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.573 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.573 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.573 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.573 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.573 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.573 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 41957 2024-09-18T00:20:19.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.573 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:20:19.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694172+0000", 2024-09-18T00:20:19.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694173+0000", 2024-09-18T00:20:19.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2500000000000005e-07 2024-09-18T00:20:19.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694175+0000", 2024-09-18T00:20:19.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.655e-06 2024-09-18T00:20:19.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694184+0000", 2024-09-18T00:20:19.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7630000000000006e-06 2024-09-18T00:20:19.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.694191+0000", 2024-09-18T00:20:19.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8870000000000003e-06 2024-09-18T00:20:19.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748593+0000", 2024-09-18T00:20:19.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054402525 2024-09-18T00:20:19.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748669+0000", 2024-09-18T00:20:19.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5532999999999997e-05 2024-09-18T00:20:19.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.748762+0000", 2024-09-18T00:20:19.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3221999999999996e-05 2024-09-18T00:20:19.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.577 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.578 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42392 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:19.578 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:20:19.578 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.781983805000003, 2024-09-18T00:20:19.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053378162, 2024-09-18T00:20:19.578 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.578 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.578 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.578 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.578 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.578 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.579 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42392 2024-09-18T00:20:19.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.579 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:20:19.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707201+0000", 2024-09-18T00:20:19.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707430+0000", 2024-09-18T00:20:19.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022838599999999999 2024-09-18T00:20:19.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707434+0000", 2024-09-18T00:20:19.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5220000000000001e-06 2024-09-18T00:20:19.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707467+0000", 2024-09-18T00:20:19.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2570000000000002e-05 2024-09-18T00:20:19.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707475+0000", 2024-09-18T00:20:19.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0730000000000005e-06 2024-09-18T00:20:19.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760446+0000", 2024-09-18T00:20:19.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052971635000000003 2024-09-18T00:20:19.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760512+0000", 2024-09-18T00:20:19.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5679999999999995e-05 2024-09-18T00:20:19.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760579+0000", 2024-09-18T00:20:19.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7296000000000001e-05 2024-09-18T00:20:19.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.583 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.583 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.583 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42393 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 358400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:19.583 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:20:19.583 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.781694854000001, 2024-09-18T00:20:19.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053269618999999997, 2024-09-18T00:20:19.583 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.583 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.583 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.583 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.583 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.584 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.584 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42393 2024-09-18T00:20:19.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.584 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:20:19.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707490+0000", 2024-09-18T00:20:19.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707491+0000", 2024-09-18T00:20:19.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1399999999999997e-07 2024-09-18T00:20:19.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707503+0000", 2024-09-18T00:20:19.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1912e-05 2024-09-18T00:20:19.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707512+0000", 2024-09-18T00:20:19.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1030000000000008e-06 2024-09-18T00:20:19.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707520+0000", 2024-09-18T00:20:19.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1249999999999993e-06 2024-09-18T00:20:19.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760605+0000", 2024-09-18T00:20:19.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053085495000000003 2024-09-18T00:20:19.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760680+0000", 2024-09-18T00:20:19.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5024000000000005e-05 2024-09-18T00:20:19.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760760+0000", 2024-09-18T00:20:19.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9345999999999996e-05 2024-09-18T00:20:19.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.588 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.589 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:42394 3.5 3:a7f24f29:::rbd_data.112a3496573c.000000000000001f:17 [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-18T00:20:19.589 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:20:19.589 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.781654865999997, 2024-09-18T00:20:19.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053403144999999999, 2024-09-18T00:20:19.589 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.589 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.589 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.589 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 42394 2024-09-18T00:20:19.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.589 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:20:19.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.707530+0000", 2024-09-18T00:20:19.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711550+0000", 2024-09-18T00:20:19.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040200189999999997 2024-09-18T00:20:19.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711556+0000", 2024-09-18T00:20:19.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2339999999999999e-06 2024-09-18T00:20:19.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711569+0000", 2024-09-18T00:20:19.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2337999999999999e-05 2024-09-18T00:20:19.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.711577+0000", 2024-09-18T00:20:19.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5660000000000007e-06 2024-09-18T00:20:19.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760790+0000", 2024-09-18T00:20:19.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049212196999999999 2024-09-18T00:20:19.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760857+0000", 2024-09-18T00:20:19.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7756000000000007e-05 2024-09-18T00:20:19.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:20.760933+0000", 2024-09-18T00:20:19.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6035e-05 2024-09-18T00:20:19.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.593 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.594 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:55489 3.5 3:a5f1e468:::rbd_data.112a859a01c9.000000000000000c:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-18T00:20:19.594 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:20:19.594 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 47.857800740000002, 2024-09-18T00:20:19.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.05748731, 2024-09-18T00:20:19.594 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.594 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.594 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:19.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.594 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 55489 2024-09-18T00:20:19.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.595 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:20:19.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631384+0000", 2024-09-18T00:20:19.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631388+0000", 2024-09-18T00:20:19.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6169999999999998e-06 2024-09-18T00:20:19.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631405+0000", 2024-09-18T00:20:19.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6643e-05 2024-09-18T00:20:19.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631444+0000", 2024-09-18T00:20:19.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9226e-05 2024-09-18T00:20:19.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631465+0000", 2024-09-18T00:20:19.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0826999999999999e-05 2024-09-18T00:20:19.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631578+0000", 2024-09-18T00:20:19.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011291899999999999 2024-09-18T00:20:19.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.631651+0000", 2024-09-18T00:20:19.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3264999999999997e-05 2024-09-18T00:20:19.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-18T00:20:19.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.634073+0000", 2024-09-18T00:20:19.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0024221350000000002 2024-09-18T00:20:19.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:19.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688732+0000", 2024-09-18T00:20:19.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.054658610000000003 2024-09-18T00:20:19.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:19.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688794+0000", 2024-09-18T00:20:19.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2255999999999995e-05 2024-09-18T00:20:19.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:19.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688834+0000", 2024-09-18T00:20:19.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9916999999999997e-05 2024-09-18T00:20:19.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:31.688872+0000", 2024-09-18T00:20:19.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7895000000000002e-05 2024-09-18T00:20:19.600 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.600 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.600 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.600 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4394.0:58361 3.1d 3:bac90a3d:::rbd_data.112a17f95ba0.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-18T00:20:19.600 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:20:19.600 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 40.285455245999998, 2024-09-18T00:20:19.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.101552963, 2024-09-18T00:20:19.600 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.601 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.601 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T00:20:19.601 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-18T00:20:19.601 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4394", 2024-09-18T00:20:19.601 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.97:0/1340991356", 2024-09-18T00:20:19.601 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 58361 2024-09-18T00:20:19.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.601 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:20:19.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203730+0000", 2024-09-18T00:20:19.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203733+0000", 2024-09-18T00:20:19.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.01e-06 2024-09-18T00:20:19.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203745+0000", 2024-09-18T00:20:19.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2513e-05 2024-09-18T00:20:19.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203777+0000", 2024-09-18T00:20:19.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1216000000000001e-05 2024-09-18T00:20:19.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203792+0000", 2024-09-18T00:20:19.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5586e-05 2024-09-18T00:20:19.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.203911+0000", 2024-09-18T00:20:19.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119234 2024-09-18T00:20:19.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.204056+0000", 2024-09-18T00:20:19.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000144943 2024-09-18T00:20:19.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-18T00:20:19.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.210219+0000", 2024-09-18T00:20:19.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061630649999999997 2024-09-18T00:20:19.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-18T00:20:19.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305071+0000", 2024-09-18T00:20:19.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.094852120999999998 2024-09-18T00:20:19.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-18T00:20:19.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305106+0000", 2024-09-18T00:20:19.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4848999999999998e-05 2024-09-18T00:20:19.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-18T00:20:19.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305208+0000", 2024-09-18T00:20:19.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101518 2024-09-18T00:20:19.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305283+0000", 2024-09-18T00:20:19.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4907999999999995e-05 2024-09-18T00:20:19.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.607 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.607 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop_reply(client.4394.0:58361 3.1d e96/79 ondisk, result = 0)", 2024-09-18T00:20:19.607 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:20:19.607 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 40.257972205000002, 2024-09-18T00:20:19.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.073876681, 2024-09-18T00:20:19.608 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-18T00:20:19.608 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-18T00:20:19.608 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-18T00:20:19.608 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-18T00:20:19.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-18T00:20:19.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:20:19.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-18T00:20:19.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231213+0000", 2024-09-18T00:20:19.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-18T00:20:19.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-18T00:20:19.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231215+0000", 2024-09-18T00:20:19.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9570000000000001e-06 2024-09-18T00:20:19.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-18T00:20:19.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231223+0000", 2024-09-18T00:20:19.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0900000000000005e-06 2024-09-18T00:20:19.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-18T00:20:19.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231252+0000", 2024-09-18T00:20:19.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9552000000000001e-05 2024-09-18T00:20:19.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-18T00:20:19.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.231267+0000", 2024-09-18T00:20:19.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4263e-05 2024-09-18T00:20:19.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-18T00:20:19.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.304871+0000", 2024-09-18T00:20:19.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.073604257000000006 2024-09-18T00:20:19.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-18T00:20:19.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.304889+0000", 2024-09-18T00:20:19.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.808e-05 2024-09-18T00:20:19.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-18T00:20:19.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-18T00:20:19.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-18T00:20:19.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-18T00:19:39.305090+0000", 2024-09-18T00:20:19.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020048200000000001 2024-09-18T00:20:19.611 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.611 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.611 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.611 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-18T00:20:19.612 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-18T00:20:19.612 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-18T00:20:19.612 INFO:teuthology.orchestra.run.smithi097.stderr:noscrub is unset 2024-09-18T00:20:19.620 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T00:20:20.422 INFO:teuthology.orchestra.run.smithi097.stderr:nodeep-scrub is unset 2024-09-18T00:20:22.643 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-18T00:20:22.643 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-18T00:20:22.644 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-18T00:20:22.644 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-18T00:20:22.646 INFO:tasks.ceph.osd.1:Started 2024-09-18T00:20:22.646 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:20:22.764 INFO:teuthology.orchestra.run.smithi097.stderr:no valid command found; 10 closest matches: 2024-09-18T00:20:22.764 INFO:teuthology.orchestra.run.smithi097.stderr:0 2024-09-18T00:20:22.764 INFO:teuthology.orchestra.run.smithi097.stderr:1 2024-09-18T00:20:22.764 INFO:teuthology.orchestra.run.smithi097.stderr:2 2024-09-18T00:20:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:abort 2024-09-18T00:20:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:assert 2024-09-18T00:20:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:config diff 2024-09-18T00:20:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:config diff get 2024-09-18T00:20:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:config get 2024-09-18T00:20:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:config help [] 2024-09-18T00:20:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:config set ... 2024-09-18T00:20:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:admin_socket: invalid command 2024-09-18T00:20:22.766 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T00:20:22.767 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-18T00:20:23.743 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:20:23.739+0000 7f362142b7c0 -1 Falling back to public interface 2024-09-18T00:20:27.689 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:20:27.683+0000 7f362142b7c0 -1 osd.1 126 log_to_monitors true 2024-09-18T00:20:27.768 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:20:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:no valid command found; 10 closest matches: 2024-09-18T00:20:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:0 2024-09-18T00:20:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:1 2024-09-18T00:20:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:2 2024-09-18T00:20:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:abort 2024-09-18T00:20:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:assert 2024-09-18T00:20:27.890 INFO:teuthology.orchestra.run.smithi097.stderr:bluefs debug_inject_read_zeros 2024-09-18T00:20:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:bluefs files list 2024-09-18T00:20:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:bluefs stats 2024-09-18T00:20:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:bluestore allocator dump block 2024-09-18T00:20:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:bluestore allocator fragmentation block 2024-09-18T00:20:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:admin_socket: invalid command 2024-09-18T00:20:27.893 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T00:20:27.894 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-18T00:20:28.646 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-09-18T00:20:28.643+0000 7f3615209640 -1 osd.1 126 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T00:20:32.894 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T00:20:33.027 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:33.027 INFO:teuthology.orchestra.run.smithi097.stdout: "ops": [], 2024-09-18T00:20:33.027 INFO:teuthology.orchestra.run.smithi097.stdout: "num_ops": 0 2024-09-18T00:20:33.027 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:33.041 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-18T00:20:33.041 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-09-18T00:20:33.041 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 6 2024-09-18T00:20:33.353 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-09-18T00:20:33.347+0000 7f63a6107640 -1 mon.a@0(leader).osd e140 definitely_dead 0 2024-09-18T00:20:33.678 INFO:teuthology.orchestra.run.smithi097.stderr:marked in osd.6. 2024-09-18T00:20:33.701 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-09-18T00:20:33.701 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-18T00:20:33.833 INFO:teuthology.orchestra.run.smithi097.stdout:mon.a: {} 2024-09-18T00:20:33.833 INFO:teuthology.orchestra.run.smithi097.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T00:20:33.846 INFO:teuthology.orchestra.run.smithi097.stdout:mon.b: {} 2024-09-18T00:20:33.846 INFO:teuthology.orchestra.run.smithi097.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T00:20:33.856 INFO:teuthology.orchestra.run.smithi097.stdout:mon.c: {} 2024-09-18T00:20:33.856 INFO:teuthology.orchestra.run.smithi097.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T00:20:33.873 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T00:20:34.687 INFO:teuthology.orchestra.run.smithi097.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T00:20:34.711 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-18T00:20:35.681 INFO:teuthology.orchestra.run.smithi097.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-18T00:20:35.710 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T00:20:36.694 INFO:teuthology.orchestra.run.smithi097.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T00:20:36.723 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-18T00:20:37.718 INFO:teuthology.orchestra.run.smithi097.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-18T00:20:37.746 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T00:20:38.723 INFO:teuthology.orchestra.run.smithi097.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T00:20:38.747 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-18T00:20:39.725 INFO:teuthology.orchestra.run.smithi097.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-18T00:20:39.750 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T00:20:40.750 INFO:teuthology.orchestra.run.smithi097.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T00:20:40.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-18T00:20:41.748 INFO:teuthology.orchestra.run.smithi097.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-18T00:20:41.772 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T00:20:42.766 INFO:teuthology.orchestra.run.smithi097.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T00:20:42.792 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-18T00:20:43.758 INFO:teuthology.orchestra.run.smithi097.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-18T00:20:43.783 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T00:20:44.780 INFO:teuthology.orchestra.run.smithi097.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T00:20:44.803 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-18T00:20:45.431 INFO:teuthology.orchestra.run.smithi097.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-18T00:20:45.458 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T00:20:46.434 INFO:teuthology.orchestra.run.smithi097.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T00:20:46.459 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-18T00:20:47.443 INFO:teuthology.orchestra.run.smithi097.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-18T00:20:47.469 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T00:20:48.447 INFO:teuthology.orchestra.run.smithi097.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T00:20:48.477 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-18T00:20:49.459 INFO:teuthology.orchestra.run.smithi097.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-18T00:20:49.483 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-18T00:20:49.483 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-18T00:20:49.483 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-18T00:20:49.484 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-18T00:20:49.484 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T00:20:49.484 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T00:20:49.783 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:20:49.783 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":157,"fsid":"b993f56a-c4ef-4602-bb41-f32ae406b45a","created":"2024-09-18T00:17:35.658788+0000","modified":"2024-09-18T00:20:49.455539+0000","last_up_change":"2024-09-18T00:20:29.617045+0000","last_in_change":"2024-09-18T00:20:33.661993+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T00:17:50.375367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T00:17:53.896718+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-18T00:18:03.715378+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":30,"snap_epoch":135,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bfda7950-3d83-435d-8313-2f14345096ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6801","nonce":40468291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6803","nonce":40468291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6807","nonce":40468291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6805","nonce":40468291}]},"public_addr":"172.21.15.97:6801/40468291","cluster_addr":"172.21.15.97:6803/40468291","heartbeat_back_addr":"172.21.15.97:6807/40468291","heartbeat_front_addr":"172.21.15.97:6805/40468291","state":["exists","up"]},{"osd":1,"uuid":"35949061-aa30-4aff-b39d-90aebeffb115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":106,"last_clean_end":126,"up_from":139,"up_thru":143,"down_at":127,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1355723817},{"type":"v1","addr":"172.21.15.97:6825","nonce":1355723817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1355723817},{"type":"v1","addr":"172.21.15.97:6827","nonce":1355723817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1355723817},{"type":"v1","addr":"172.21.15.97:6831","nonce":1355723817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1355723817},{"type":"v1","addr":"172.21.15.97:6829","nonce":1355723817}]},"public_addr":"172.21.15.97:6825/1355723817","cluster_addr":"172.21.15.97:6827/1355723817","heartbeat_back_addr":"172.21.15.97:6831/1355723817","heartbeat_front_addr":"172.21.15.97:6829/1355723817","state":["exists","up"]},{"osd":2,"uuid":"31b3e1c1-7648-44fe-9931-758ffe9a53ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6817","nonce":2425060925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6819","nonce":2425060925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6823","nonce":2425060925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6821","nonce":2425060925}]},"public_addr":"172.21.15.97:6817/2425060925","cluster_addr":"172.21.15.97:6819/2425060925","heartbeat_back_addr":"172.21.15.97:6823/2425060925","heartbeat_front_addr":"172.21.15.97:6821/2425060925","state":["exists","up"]},{"osd":3,"uuid":"d7daebbc-0e33-42ab-97ee-a6c1b449ae8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6809","nonce":3512666164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6811","nonce":3512666164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6815","nonce":3512666164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6813","nonce":3512666164}]},"public_addr":"172.21.15.97:6809/3512666164","cluster_addr":"172.21.15.97:6811/3512666164","heartbeat_back_addr":"172.21.15.97:6815/3512666164","heartbeat_front_addr":"172.21.15.97:6813/3512666164","state":["exists","up"]},{"osd":4,"uuid":"d55ca69b-5000-45ff-acc4-00f7c4763cad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6804","nonce":3191744356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6808","nonce":3191744356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6815","nonce":3191744356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6812","nonce":3191744356}]},"public_addr":"172.21.15.114:6804/3191744356","cluster_addr":"172.21.15.114:6808/3191744356","heartbeat_back_addr":"172.21.15.114:6815/3191744356","heartbeat_front_addr":"172.21.15.114:6812/3191744356","state":["exists","up"]},{"osd":5,"uuid":"196a329b-bd93-4cc2-85bb-64ed81d3918b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":55,"up_from":76,"up_thru":155,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":407304039},{"type":"v1","addr":"172.21.15.114:6801","nonce":407304039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6803","nonce":407304039},{"type":"v1","addr":"172.21.15.114:6805","nonce":407304039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6811","nonce":407304039},{"type":"v1","addr":"172.21.15.114:6813","nonce":407304039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6807","nonce":407304039},{"type":"v1","addr":"172.21.15.114:6809","nonce":407304039}]},"public_addr":"172.21.15.114:6801/407304039","cluster_addr":"172.21.15.114:6805/407304039","heartbeat_back_addr":"172.21.15.114:6813/407304039","heartbeat_front_addr":"172.21.15.114:6809/407304039","state":["exists","up"]},{"osd":6,"uuid":"dc9712f0-8858-467a-8397-d5eb97187165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6825","nonce":1148348470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6827","nonce":1148348470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6831","nonce":1148348470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6829","nonce":1148348470}]},"public_addr":"172.21.15.114:6825/1148348470","cluster_addr":"172.21.15.114:6827/1148348470","heartbeat_back_addr":"172.21.15.114:6831/1148348470","heartbeat_front_addr":"172.21.15.114:6829/1148348470","state":["exists","up"]},{"osd":7,"uuid":"b6d774ee-fe6d-462c-afa4-7ffd5d7343a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6817","nonce":2133497352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6819","nonce":2133497352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6823","nonce":2133497352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6821","nonce":2133497352}]},"public_addr":"172.21.15.114:6817/2133497352","cluster_addr":"172.21.15.114:6819/2133497352","heartbeat_back_addr":"172.21.15.114:6823/2133497352","heartbeat_front_addr":"172.21.15.114:6821/2133497352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.237241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.504107+0000","dead_epoch":127},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.690795+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.197056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.459497+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.805108+0000","dead_epoch":56},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.484317+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.741853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.a","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T00:20:49.802 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T00:20:49.802 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T00:20:50.101 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:20:50.101 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":157,"fsid":"b993f56a-c4ef-4602-bb41-f32ae406b45a","created":"2024-09-18T00:17:35.658788+0000","modified":"2024-09-18T00:20:49.455539+0000","last_up_change":"2024-09-18T00:20:29.617045+0000","last_in_change":"2024-09-18T00:20:33.661993+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T00:17:50.375367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T00:17:53.896718+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-18T00:18:03.715378+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":30,"snap_epoch":135,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bfda7950-3d83-435d-8313-2f14345096ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6801","nonce":40468291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6803","nonce":40468291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6807","nonce":40468291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6805","nonce":40468291}]},"public_addr":"172.21.15.97:6801/40468291","cluster_addr":"172.21.15.97:6803/40468291","heartbeat_back_addr":"172.21.15.97:6807/40468291","heartbeat_front_addr":"172.21.15.97:6805/40468291","state":["exists","up"]},{"osd":1,"uuid":"35949061-aa30-4aff-b39d-90aebeffb115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":106,"last_clean_end":126,"up_from":139,"up_thru":143,"down_at":127,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1355723817},{"type":"v1","addr":"172.21.15.97:6825","nonce":1355723817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1355723817},{"type":"v1","addr":"172.21.15.97:6827","nonce":1355723817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1355723817},{"type":"v1","addr":"172.21.15.97:6831","nonce":1355723817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1355723817},{"type":"v1","addr":"172.21.15.97:6829","nonce":1355723817}]},"public_addr":"172.21.15.97:6825/1355723817","cluster_addr":"172.21.15.97:6827/1355723817","heartbeat_back_addr":"172.21.15.97:6831/1355723817","heartbeat_front_addr":"172.21.15.97:6829/1355723817","state":["exists","up"]},{"osd":2,"uuid":"31b3e1c1-7648-44fe-9931-758ffe9a53ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6817","nonce":2425060925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6819","nonce":2425060925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6823","nonce":2425060925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6821","nonce":2425060925}]},"public_addr":"172.21.15.97:6817/2425060925","cluster_addr":"172.21.15.97:6819/2425060925","heartbeat_back_addr":"172.21.15.97:6823/2425060925","heartbeat_front_addr":"172.21.15.97:6821/2425060925","state":["exists","up"]},{"osd":3,"uuid":"d7daebbc-0e33-42ab-97ee-a6c1b449ae8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6809","nonce":3512666164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6811","nonce":3512666164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6815","nonce":3512666164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6813","nonce":3512666164}]},"public_addr":"172.21.15.97:6809/3512666164","cluster_addr":"172.21.15.97:6811/3512666164","heartbeat_back_addr":"172.21.15.97:6815/3512666164","heartbeat_front_addr":"172.21.15.97:6813/3512666164","state":["exists","up"]},{"osd":4,"uuid":"d55ca69b-5000-45ff-acc4-00f7c4763cad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6804","nonce":3191744356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6808","nonce":3191744356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6815","nonce":3191744356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6812","nonce":3191744356}]},"public_addr":"172.21.15.114:6804/3191744356","cluster_addr":"172.21.15.114:6808/3191744356","heartbeat_back_addr":"172.21.15.114:6815/3191744356","heartbeat_front_addr":"172.21.15.114:6812/3191744356","state":["exists","up"]},{"osd":5,"uuid":"196a329b-bd93-4cc2-85bb-64ed81d3918b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":55,"up_from":76,"up_thru":155,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":407304039},{"type":"v1","addr":"172.21.15.114:6801","nonce":407304039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6803","nonce":407304039},{"type":"v1","addr":"172.21.15.114:6805","nonce":407304039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6811","nonce":407304039},{"type":"v1","addr":"172.21.15.114:6813","nonce":407304039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6807","nonce":407304039},{"type":"v1","addr":"172.21.15.114:6809","nonce":407304039}]},"public_addr":"172.21.15.114:6801/407304039","cluster_addr":"172.21.15.114:6805/407304039","heartbeat_back_addr":"172.21.15.114:6813/407304039","heartbeat_front_addr":"172.21.15.114:6809/407304039","state":["exists","up"]},{"osd":6,"uuid":"dc9712f0-8858-467a-8397-d5eb97187165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6825","nonce":1148348470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6827","nonce":1148348470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6831","nonce":1148348470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6829","nonce":1148348470}]},"public_addr":"172.21.15.114:6825/1148348470","cluster_addr":"172.21.15.114:6827/1148348470","heartbeat_back_addr":"172.21.15.114:6831/1148348470","heartbeat_front_addr":"172.21.15.114:6829/1148348470","state":["exists","up"]},{"osd":7,"uuid":"b6d774ee-fe6d-462c-afa4-7ffd5d7343a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6817","nonce":2133497352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6819","nonce":2133497352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6823","nonce":2133497352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6821","nonce":2133497352}]},"public_addr":"172.21.15.114:6817/2133497352","cluster_addr":"172.21.15.114:6819/2133497352","heartbeat_back_addr":"172.21.15.114:6823/2133497352","heartbeat_front_addr":"172.21.15.114:6821/2133497352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.237241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.504107+0000","dead_epoch":127},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.690795+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.197056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.459497+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.805108+0000","dead_epoch":56},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.484317+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.741853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.a","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T00:20:50.120 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T00:20:50.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T00:20:50.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T00:20:50.122 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T00:20:50.122 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T00:20:50.122 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T00:20:50.123 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-18T00:20:50.123 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-18T00:20:50.370 INFO:teuthology.orchestra.run.smithi097.stdout:597000454150 2024-09-18T00:20:50.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T00:20:50.385 INFO:teuthology.orchestra.run.smithi097.stdout:326417514517 2024-09-18T00:20:50.386 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T00:20:50.389 INFO:teuthology.orchestra.run.smithi097.stdout:64424509479 2024-09-18T00:20:50.389 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T00:20:50.390 INFO:teuthology.orchestra.run.smithi097.stdout:64424509480 2024-09-18T00:20:50.391 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T00:20:50.391 INFO:teuthology.orchestra.run.smithi097.stdout:60129542185 2024-09-18T00:20:50.392 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T00:20:50.395 INFO:teuthology.orchestra.run.smithi097.stdout:64424509482 2024-09-18T00:20:50.396 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T00:20:50.399 INFO:teuthology.orchestra.run.smithi097.stdout:60129542184 2024-09-18T00:20:50.400 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T00:20:50.403 INFO:teuthology.orchestra.run.smithi097.stdout:64424509479 2024-09-18T00:20:50.403 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T00:20:50.893 INFO:teuthology.orchestra.run.smithi097.stdout:326417514517 2024-09-18T00:20:50.900 INFO:teuthology.orchestra.run.smithi097.stdout:60129542184 2024-09-18T00:20:50.901 INFO:teuthology.orchestra.run.smithi097.stdout:597000454150 2024-09-18T00:20:50.924 INFO:tasks.ceph.ceph_manager.ceph:need seq 326417514517 got 326417514517 for osd.5 2024-09-18T00:20:50.924 DEBUG:teuthology.parallel:result is None 2024-09-18T00:20:50.925 INFO:teuthology.orchestra.run.smithi097.stdout:64424509479 2024-09-18T00:20:50.934 INFO:tasks.ceph.ceph_manager.ceph:need seq 597000454150 got 597000454150 for osd.1 2024-09-18T00:20:50.934 DEBUG:teuthology.parallel:result is None 2024-09-18T00:20:50.934 INFO:teuthology.orchestra.run.smithi097.stdout:64424509478 2024-09-18T00:20:50.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542185 got 60129542184 for osd.0 2024-09-18T00:20:50.951 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509480 got 64424509479 for osd.4 2024-09-18T00:20:50.954 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509478 for osd.6 2024-09-18T00:20:50.960 INFO:teuthology.orchestra.run.smithi097.stdout:60129542183 2024-09-18T00:20:50.965 INFO:teuthology.orchestra.run.smithi097.stdout:64424509477 2024-09-18T00:20:50.977 INFO:teuthology.orchestra.run.smithi097.stdout:64424509481 2024-09-18T00:20:50.978 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542184 got 60129542183 for osd.3 2024-09-18T00:20:50.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509477 for osd.7 2024-09-18T00:20:50.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509482 got 64424509481 for osd.2 2024-09-18T00:20:51.939 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T00:20:51.952 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T00:20:51.955 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T00:20:51.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T00:20:51.989 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T00:20:51.997 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T00:20:52.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-18T00:20:52.295+0000 7fae8d5dd640 0 --2- 172.21.15.97:0/1523008167 >> v2:172.21.15.97:3300/0 conn(0x7fae880721a0 0x7fae88142900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-18T00:20:52.390 INFO:teuthology.orchestra.run.smithi097.stdout:60129542183 2024-09-18T00:20:52.398 INFO:teuthology.orchestra.run.smithi097.stdout:64424509477 2024-09-18T00:20:52.416 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542184 got 60129542183 for osd.3 2024-09-18T00:20:52.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509477 for osd.7 2024-09-18T00:20:52.449 INFO:teuthology.orchestra.run.smithi097.stdout:64424509479 2024-09-18T00:20:52.471 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509480 got 64424509479 for osd.4 2024-09-18T00:20:52.471 INFO:teuthology.orchestra.run.smithi097.stdout:64424509481 2024-09-18T00:20:52.474 INFO:teuthology.orchestra.run.smithi097.stdout:64424509478 2024-09-18T00:20:52.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509482 got 64424509481 for osd.2 2024-09-18T00:20:52.500 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509478 for osd.6 2024-09-18T00:20:52.530 INFO:teuthology.orchestra.run.smithi097.stdout:60129542185 2024-09-18T00:20:52.547 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542185 got 60129542185 for osd.0 2024-09-18T00:20:52.547 DEBUG:teuthology.parallel:result is None 2024-09-18T00:20:53.417 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T00:20:53.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T00:20:53.471 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T00:20:53.499 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T00:20:53.501 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T00:20:53.768 INFO:teuthology.orchestra.run.smithi097.stdout:60129542184 2024-09-18T00:20:53.786 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542184 got 60129542184 for osd.3 2024-09-18T00:20:53.786 DEBUG:teuthology.parallel:result is None 2024-09-18T00:20:53.818 INFO:teuthology.orchestra.run.smithi097.stdout:64424509481 2024-09-18T00:20:53.832 INFO:teuthology.orchestra.run.smithi097.stdout:64424509482 2024-09-18T00:20:53.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509480 got 64424509481 for osd.4 2024-09-18T00:20:53.835 DEBUG:teuthology.parallel:result is None 2024-09-18T00:20:53.850 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509482 got 64424509482 for osd.2 2024-09-18T00:20:53.850 DEBUG:teuthology.parallel:result is None 2024-09-18T00:20:53.891 INFO:teuthology.orchestra.run.smithi097.stdout:64424509479 2024-09-18T00:20:53.914 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509479 for osd.7 2024-09-18T00:20:53.914 DEBUG:teuthology.parallel:result is None 2024-09-18T00:20:53.929 INFO:teuthology.orchestra.run.smithi097.stdout:64424509480 2024-09-18T00:20:53.946 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509480 for osd.6 2024-09-18T00:20:53.946 DEBUG:teuthology.parallel:result is None 2024-09-18T00:20:53.946 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T00:20:53.946 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:20:54.278 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:20:54.278 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-18T00:20:54.294 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-18T00:20:52.347712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":65,"num_read":96618,"num_read_kb":273533,"num_write":19451,"num_write_kb":184146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1839,"num_bytes_recovered":27009154,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16316,"ondisk_log_size":16316,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":365644,"kb_used_data":14728,"kb_used_omap":35,"kb_used_meta":350684,"kb_avail":754609076,"statfs":{"total":773094113280,"available":772719693824,"internally_reserved":0,"allocated":15081472,"data_stored":8296592,"data_compressed":13209,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":36807,"internal_metadata":359100473},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.913498"},"pg_stats":[{"pgid":"3.1f","version":"135'778","reported_seq":3008,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451528+0000","last_change":"2024-09-18T00:20:17.573423+0000","last_active":"2024-09-18T00:20:48.451528+0000","last_peered":"2024-09-18T00:20:48.451528+0000","last_clean":"2024-09-18T00:20:48.451528+0000","last_became_active":"2024-09-18T00:19:18.327350+0000","last_became_peered":"2024-09-18T00:19:18.327350+0000","last_unstale":"2024-09-18T00:20:48.451528+0000","last_undegraded":"2024-09-18T00:20:48.451528+0000","last_fullsized":"2024-09-18T00:20:48.451528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":778,"log_dups_size":0,"ondisk_log_size":778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:46:14.185856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024250489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4587,"num_read_kb":13915,"num_write":1017,"num_write_kb":7086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":2252743,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"133'465","reported_seq":1557,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.457751+0000","last_change":"2024-09-18T00:20:37.723738+0000","last_active":"2024-09-18T00:20:48.457751+0000","last_peered":"2024-09-18T00:20:48.457751+0000","last_clean":"2024-09-18T00:20:48.457751+0000","last_became_active":"2024-09-18T00:20:37.722312+0000","last_became_peered":"2024-09-18T00:20:37.722312+0000","last_unstale":"2024-09-18T00:20:48.457751+0000","last_undegraded":"2024-09-18T00:20:48.457751+0000","last_fullsized":"2024-09-18T00:20:48.457751+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:14:23.427889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057225740000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1405,"num_read_kb":1924,"num_write":538,"num_write_kb":2847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":46741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"135'781","reported_seq":1530,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.470232+0000","last_change":"2024-09-18T00:20:48.470232+0000","last_active":"2024-09-18T00:20:48.470232+0000","last_peered":"2024-09-18T00:20:48.470232+0000","last_clean":"2024-09-18T00:20:48.470232+0000","last_became_active":"2024-09-18T00:20:48.465769+0000","last_became_peered":"2024-09-18T00:20:48.465769+0000","last_unstale":"2024-09-18T00:20:48.470232+0000","last_undegraded":"2024-09-18T00:20:48.470232+0000","last_fullsized":"2024-09-18T00:20:48.470232+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:11:07.738987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032750750000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3226,"num_read_kb":4864,"num_write":1184,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":510608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"135'453","reported_seq":1951,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.467870+0000","last_change":"2024-09-18T00:20:48.467870+0000","last_active":"2024-09-18T00:20:48.467870+0000","last_peered":"2024-09-18T00:20:48.467870+0000","last_clean":"2024-09-18T00:20:48.467870+0000","last_became_active":"2024-09-18T00:20:48.463565+0000","last_became_peered":"2024-09-18T00:20:48.463565+0000","last_unstale":"2024-09-18T00:20:48.467870+0000","last_undegraded":"2024-09-18T00:20:48.467870+0000","last_fullsized":"2024-09-18T00:20:48.467870+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:07:09.913555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002116522,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3432,"num_read_kb":9919,"num_write":470,"num_write_kb":3833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":72221,"num_keys_recovered":54,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"133'289","reported_seq":1665,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451394+0000","last_change":"2024-09-18T00:20:17.572140+0000","last_active":"2024-09-18T00:20:48.451394+0000","last_peered":"2024-09-18T00:20:48.451394+0000","last_clean":"2024-09-18T00:20:48.451394+0000","last_became_active":"2024-09-18T00:18:05.415810+0000","last_became_peered":"2024-09-18T00:18:05.415810+0000","last_unstale":"2024-09-18T00:20:48.451394+0000","last_undegraded":"2024-09-18T00:20:48.451394+0000","last_fullsized":"2024-09-18T00:20:48.451394+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:35:11.075422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015232959999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1498,"num_read_kb":2662,"num_write":281,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"140'226","reported_seq":3328,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459066+0000","last_change":"2024-09-18T00:20:30.686023+0000","last_active":"2024-09-18T00:20:48.459066+0000","last_peered":"2024-09-18T00:20:48.459066+0000","last_clean":"2024-09-18T00:20:48.459066+0000","last_became_active":"2024-09-18T00:20:30.637311+0000","last_became_peered":"2024-09-18T00:20:30.637311+0000","last_unstale":"2024-09-18T00:20:48.459066+0000","last_undegraded":"2024-09-18T00:20:48.459066+0000","last_fullsized":"2024-09-18T00:20:48.459066+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:56:01.105381+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012894545,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":2320,"num_read_kb":4551,"num_write":211,"num_write_kb":2083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":645956,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"140'384","reported_seq":1512,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462374+0000","last_change":"2024-09-18T00:20:30.693807+0000","last_active":"2024-09-18T00:20:49.462374+0000","last_peered":"2024-09-18T00:20:49.462374+0000","last_clean":"2024-09-18T00:20:49.462374+0000","last_became_active":"2024-09-18T00:20:30.640499+0000","last_became_peered":"2024-09-18T00:20:30.640499+0000","last_unstale":"2024-09-18T00:20:49.462374+0000","last_undegraded":"2024-09-18T00:20:49.462374+0000","last_fullsized":"2024-09-18T00:20:49.462374+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:26.838200+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018217595999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1131,"num_read_kb":4419,"num_write":389,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":1283165,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"133'327","reported_seq":3716,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.457199+0000","last_change":"2024-09-18T00:20:17.694311+0000","last_active":"2024-09-18T00:20:48.457199+0000","last_peered":"2024-09-18T00:20:48.457199+0000","last_clean":"2024-09-18T00:20:48.457199+0000","last_became_active":"2024-09-18T00:18:05.406821+0000","last_became_peered":"2024-09-18T00:18:05.406821+0000","last_unstale":"2024-09-18T00:20:48.457199+0000","last_undegraded":"2024-09-18T00:20:48.457199+0000","last_fullsized":"2024-09-18T00:20:48.457199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:13:08.591812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022731449999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3612,"num_read_kb":21086,"num_write":298,"num_write_kb":8166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'372","reported_seq":1395,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451657+0000","last_change":"2024-09-18T00:20:17.573988+0000","last_active":"2024-09-18T00:20:48.451657+0000","last_peered":"2024-09-18T00:20:48.451657+0000","last_clean":"2024-09-18T00:20:48.451657+0000","last_became_active":"2024-09-18T00:19:18.328043+0000","last_became_peered":"2024-09-18T00:19:18.328043+0000","last_unstale":"2024-09-18T00:20:48.451657+0000","last_undegraded":"2024-09-18T00:20:48.451657+0000","last_fullsized":"2024-09-18T00:20:48.451657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:11:44.153392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025255270000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1318,"num_read_kb":6224,"num_write":322,"num_write_kb":4815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":491875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"135'690","reported_seq":2486,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.471288+0000","last_change":"2024-09-18T00:20:48.471288+0000","last_active":"2024-09-18T00:20:48.471288+0000","last_peered":"2024-09-18T00:20:48.471288+0000","last_clean":"2024-09-18T00:20:48.471288+0000","last_became_active":"2024-09-18T00:20:48.467364+0000","last_became_peered":"2024-09-18T00:20:48.467364+0000","last_unstale":"2024-09-18T00:20:48.471288+0000","last_undegraded":"2024-09-18T00:20:48.471288+0000","last_fullsized":"2024-09-18T00:20:48.471288+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:25.555616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021376099999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4502,"num_read_kb":5989,"num_write":925,"num_write_kb":5689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1385456,"num_keys_recovered":29,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"133'489","reported_seq":1056,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463552+0000","last_change":"2024-09-18T00:20:17.577703+0000","last_active":"2024-09-18T00:20:49.463552+0000","last_peered":"2024-09-18T00:20:49.463552+0000","last_clean":"2024-09-18T00:20:49.463552+0000","last_became_active":"2024-09-18T00:18:16.438164+0000","last_became_peered":"2024-09-18T00:18:16.438164+0000","last_unstale":"2024-09-18T00:20:49.463552+0000","last_undegraded":"2024-09-18T00:20:49.463552+0000","last_fullsized":"2024-09-18T00:20:49.463552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:18:47.201515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014879730000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":1078,"num_read_kb":1346,"num_write":552,"num_write_kb":3993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"134'530","reported_seq":1981,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459503+0000","last_change":"2024-09-18T00:20:17.576876+0000","last_active":"2024-09-18T00:20:48.459503+0000","last_peered":"2024-09-18T00:20:48.459503+0000","last_clean":"2024-09-18T00:20:48.459503+0000","last_became_active":"2024-09-18T00:18:16.441758+0000","last_became_peered":"2024-09-18T00:18:16.441758+0000","last_unstale":"2024-09-18T00:20:48.459503+0000","last_undegraded":"2024-09-18T00:20:48.459503+0000","last_fullsized":"2024-09-18T00:20:48.459503+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:18:47.366485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050089940000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2844,"num_read_kb":5653,"num_write":555,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":103027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":38,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451888+0000","last_change":"2024-09-18T00:20:34.754460+0000","last_active":"2024-09-18T00:20:48.451888+0000","last_peered":"2024-09-18T00:20:48.451888+0000","last_clean":"2024-09-18T00:20:48.451888+0000","last_became_active":"2024-09-18T00:20:34.753675+0000","last_became_peered":"2024-09-18T00:20:34.753675+0000","last_unstale":"2024-09-18T00:20:48.451888+0000","last_undegraded":"2024-09-18T00:20:48.451888+0000","last_fullsized":"2024-09-18T00:20:48.451888+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:37.203669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089554000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"135'348","reported_seq":2423,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.452208+0000","last_change":"2024-09-18T00:20:30.727046+0000","last_active":"2024-09-18T00:20:48.452208+0000","last_peered":"2024-09-18T00:20:48.452208+0000","last_clean":"2024-09-18T00:20:48.452208+0000","last_became_active":"2024-09-18T00:20:30.648069+0000","last_became_peered":"2024-09-18T00:20:30.648069+0000","last_unstale":"2024-09-18T00:20:48.452208+0000","last_undegraded":"2024-09-18T00:20:48.452208+0000","last_fullsized":"2024-09-18T00:20:48.452208+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:36.621517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052221869999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2016,"num_read_kb":5374,"num_write":329,"num_write_kb":5799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":1635567,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":59,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462482+0000","last_change":"2024-09-18T00:20:34.694081+0000","last_active":"2024-09-18T00:20:49.462482+0000","last_peered":"2024-09-18T00:20:49.462482+0000","last_clean":"2024-09-18T00:20:49.462482+0000","last_became_active":"2024-09-18T00:20:34.693550+0000","last_became_peered":"2024-09-18T00:20:34.693550+0000","last_unstale":"2024-09-18T00:20:49.462482+0000","last_undegraded":"2024-09-18T00:20:49.462482+0000","last_fullsized":"2024-09-18T00:20:49.462482+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:34:59.142586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'665","reported_seq":3136,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.457589+0000","last_change":"2024-09-18T00:20:17.695571+0000","last_active":"2024-09-18T00:20:48.457589+0000","last_peered":"2024-09-18T00:20:48.457589+0000","last_clean":"2024-09-18T00:20:48.457589+0000","last_became_active":"2024-09-18T00:18:16.440536+0000","last_became_peered":"2024-09-18T00:18:16.440536+0000","last_unstale":"2024-09-18T00:20:48.457589+0000","last_undegraded":"2024-09-18T00:20:48.457589+0000","last_fullsized":"2024-09-18T00:20:48.457589+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:35.729973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034870019999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":3858,"num_read_kb":13564,"num_write":915,"num_write_kb":9611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":73512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":257,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463801+0000","last_change":"2024-09-18T00:18:16.439964+0000","last_active":"2024-09-18T00:20:49.463801+0000","last_peered":"2024-09-18T00:20:49.463801+0000","last_clean":"2024-09-18T00:20:49.463801+0000","last_became_active":"2024-09-18T00:18:16.437150+0000","last_became_peered":"2024-09-18T00:18:16.437150+0000","last_unstale":"2024-09-18T00:20:49.463801+0000","last_undegraded":"2024-09-18T00:20:49.463801+0000","last_fullsized":"2024-09-18T00:20:49.463801+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:25.772294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"140'491","reported_seq":2308,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462407+0000","last_change":"2024-09-18T00:20:30.710100+0000","last_active":"2024-09-18T00:20:49.462407+0000","last_peered":"2024-09-18T00:20:49.462407+0000","last_clean":"2024-09-18T00:20:49.462407+0000","last_became_active":"2024-09-18T00:20:30.641845+0000","last_became_peered":"2024-09-18T00:20:30.641845+0000","last_unstale":"2024-09-18T00:20:49.462407+0000","last_undegraded":"2024-09-18T00:20:49.462407+0000","last_fullsized":"2024-09-18T00:20:49.462407+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:00:46.444023+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.013533892,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2426,"num_read_kb":4923,"num_write":423,"num_write_kb":4982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":1553378,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462313+0000","last_change":"2024-09-18T00:20:30.651121+0000","last_active":"2024-09-18T00:20:49.462313+0000","last_peered":"2024-09-18T00:20:49.462313+0000","last_clean":"2024-09-18T00:20:49.462313+0000","last_became_active":"2024-09-18T00:20:30.650373+0000","last_became_peered":"2024-09-18T00:20:30.650373+0000","last_unstale":"2024-09-18T00:20:49.462313+0000","last_undegraded":"2024-09-18T00:20:49.462313+0000","last_fullsized":"2024-09-18T00:20:49.462313+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:36:36.287145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"133'363","reported_seq":2712,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.470308+0000","last_change":"2024-09-18T00:20:48.470308+0000","last_active":"2024-09-18T00:20:48.470308+0000","last_peered":"2024-09-18T00:20:48.470308+0000","last_clean":"2024-09-18T00:20:48.470308+0000","last_became_active":"2024-09-18T00:20:48.462691+0000","last_became_peered":"2024-09-18T00:20:48.462691+0000","last_unstale":"2024-09-18T00:20:48.470308+0000","last_undegraded":"2024-09-18T00:20:48.470308+0000","last_fullsized":"2024-09-18T00:20:48.470308+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:08.083732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001889571,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2213,"num_read_kb":12284,"num_write":366,"num_write_kb":5310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":675014,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"135'484","reported_seq":2135,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459435+0000","last_change":"2024-09-18T00:20:17.573853+0000","last_active":"2024-09-18T00:20:48.459435+0000","last_peered":"2024-09-18T00:20:48.459435+0000","last_clean":"2024-09-18T00:20:48.459435+0000","last_became_active":"2024-09-18T00:18:05.407889+0000","last_became_peered":"2024-09-18T00:18:05.407889+0000","last_unstale":"2024-09-18T00:20:48.459435+0000","last_undegraded":"2024-09-18T00:20:48.459435+0000","last_fullsized":"2024-09-18T00:20:48.459435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:40.520294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002502663,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":1766,"num_read_kb":5407,"num_write":446,"num_write_kb":5379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":201,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.461100+0000","last_change":"2024-09-18T00:20:48.461100+0000","last_active":"2024-09-18T00:20:48.461100+0000","last_peered":"2024-09-18T00:20:48.461100+0000","last_clean":"2024-09-18T00:20:48.461100+0000","last_became_active":"2024-09-18T00:20:48.460277+0000","last_became_peered":"2024-09-18T00:20:48.460277+0000","last_unstale":"2024-09-18T00:20:48.461100+0000","last_undegraded":"2024-09-18T00:20:48.461100+0000","last_fullsized":"2024-09-18T00:20:48.461100+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:01:02.391844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088189300000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"134'369","reported_seq":1075,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462205+0000","last_change":"2024-09-18T00:20:30.726339+0000","last_active":"2024-09-18T00:20:49.462205+0000","last_peered":"2024-09-18T00:20:49.462205+0000","last_clean":"2024-09-18T00:20:49.462205+0000","last_became_active":"2024-09-18T00:20:30.647777+0000","last_became_peered":"2024-09-18T00:20:30.647777+0000","last_unstale":"2024-09-18T00:20:49.462205+0000","last_undegraded":"2024-09-18T00:20:49.462205+0000","last_fullsized":"2024-09-18T00:20:49.462205+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:21:04.035067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011927055000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":740,"num_read_kb":1879,"num_write":446,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":1002237,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":289,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.461675+0000","last_change":"2024-09-18T00:17:57.477633+0000","last_active":"2024-09-18T00:20:49.461675+0000","last_peered":"2024-09-18T00:20:49.461675+0000","last_clean":"2024-09-18T00:20:49.461675+0000","last_became_active":"2024-09-18T00:17:55.631277+0000","last_became_peered":"2024-09-18T00:17:55.631277+0000","last_unstale":"2024-09-18T00:20:49.461675+0000","last_undegraded":"2024-09-18T00:20:49.461675+0000","last_fullsized":"2024-09-18T00:20:49.461675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:55.421684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001141204,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"135'502","reported_seq":2618,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451329+0000","last_change":"2024-09-18T00:20:17.571764+0000","last_active":"2024-09-18T00:20:48.451329+0000","last_peered":"2024-09-18T00:20:48.451329+0000","last_clean":"2024-09-18T00:20:48.451329+0000","last_became_active":"2024-09-18T00:18:05.410952+0000","last_became_peered":"2024-09-18T00:18:05.410952+0000","last_unstale":"2024-09-18T00:20:48.451329+0000","last_undegraded":"2024-09-18T00:20:48.451329+0000","last_fullsized":"2024-09-18T00:20:48.451329+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:15.435600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014272110000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3308,"num_read_kb":10125,"num_write":594,"num_write_kb":8641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":276,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463375+0000","last_change":"2024-09-18T00:20:30.657193+0000","last_active":"2024-09-18T00:20:49.463375+0000","last_peered":"2024-09-18T00:20:49.463375+0000","last_clean":"2024-09-18T00:20:49.463375+0000","last_became_active":"2024-09-18T00:20:30.656400+0000","last_became_peered":"2024-09-18T00:20:30.656400+0000","last_unstale":"2024-09-18T00:20:49.463375+0000","last_undegraded":"2024-09-18T00:20:49.463375+0000","last_fullsized":"2024-09-18T00:20:49.463375+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:50:27.874106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"134'493","reported_seq":2491,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.457219+0000","last_change":"2024-09-18T00:20:17.697271+0000","last_active":"2024-09-18T00:20:48.457219+0000","last_peered":"2024-09-18T00:20:48.457219+0000","last_clean":"2024-09-18T00:20:48.457219+0000","last_became_active":"2024-09-18T00:19:18.330155+0000","last_became_peered":"2024-09-18T00:19:18.330155+0000","last_unstale":"2024-09-18T00:20:48.457219+0000","last_undegraded":"2024-09-18T00:20:48.457219+0000","last_fullsized":"2024-09-18T00:20:48.457219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:01:28.392225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005129074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2497,"num_read_kb":7710,"num_write":474,"num_write_kb":3192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":493176,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":120,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459774+0000","last_change":"2024-09-18T00:20:48.459774+0000","last_active":"2024-09-18T00:20:48.459774+0000","last_peered":"2024-09-18T00:20:48.459774+0000","last_clean":"2024-09-18T00:20:48.459774+0000","last_became_active":"2024-09-18T00:20:48.457808+0000","last_became_peered":"2024-09-18T00:20:48.457808+0000","last_unstale":"2024-09-18T00:20:48.459774+0000","last_undegraded":"2024-09-18T00:20:48.459774+0000","last_fullsized":"2024-09-18T00:20:48.459774+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:43:17.325507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":438,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463383+0000","last_change":"2024-09-18T00:18:19.466612+0000","last_active":"2024-09-18T00:20:49.463383+0000","last_peered":"2024-09-18T00:20:49.463383+0000","last_clean":"2024-09-18T00:20:49.463383+0000","last_became_active":"2024-09-18T00:18:19.465557+0000","last_became_peered":"2024-09-18T00:18:19.465557+0000","last_unstale":"2024-09-18T00:20:49.463383+0000","last_undegraded":"2024-09-18T00:20:49.463383+0000","last_fullsized":"2024-09-18T00:20:49.463383+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_clean_scrub_stamp":"2024-09-18T00:17:50.414591+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:08:40.389972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"140'661","reported_seq":2375,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.457239+0000","last_change":"2024-09-18T00:20:30.742660+0000","last_active":"2024-09-18T00:20:48.457239+0000","last_peered":"2024-09-18T00:20:48.457239+0000","last_clean":"2024-09-18T00:20:48.457239+0000","last_became_active":"2024-09-18T00:20:30.656699+0000","last_became_peered":"2024-09-18T00:20:30.656699+0000","last_unstale":"2024-09-18T00:20:48.457239+0000","last_undegraded":"2024-09-18T00:20:48.457239+0000","last_fullsized":"2024-09-18T00:20:48.457239+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:37:48.645340+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0070126900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3348,"num_read_kb":6619,"num_write":872,"num_write_kb":5519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":1197799,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"134'383","reported_seq":2112,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459075+0000","last_change":"2024-09-18T00:20:17.574821+0000","last_active":"2024-09-18T00:20:48.459075+0000","last_peered":"2024-09-18T00:20:48.459075+0000","last_clean":"2024-09-18T00:20:48.459075+0000","last_became_active":"2024-09-18T00:18:05.401097+0000","last_became_peered":"2024-09-18T00:18:05.401097+0000","last_unstale":"2024-09-18T00:20:48.459075+0000","last_undegraded":"2024-09-18T00:20:48.459075+0000","last_fullsized":"2024-09-18T00:20:48.459075+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:50:58.016930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003875547,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3069,"num_read_kb":9114,"num_write":378,"num_write_kb":6107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"140'412","reported_seq":2388,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451796+0000","last_change":"2024-09-18T00:20:37.726930+0000","last_active":"2024-09-18T00:20:48.451796+0000","last_peered":"2024-09-18T00:20:48.451796+0000","last_clean":"2024-09-18T00:20:48.451796+0000","last_became_active":"2024-09-18T00:20:37.721802+0000","last_became_peered":"2024-09-18T00:20:37.721802+0000","last_unstale":"2024-09-18T00:20:48.451796+0000","last_undegraded":"2024-09-18T00:20:48.451796+0000","last_fullsized":"2024-09-18T00:20:48.451796+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:37.117595+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.007118643,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":1971,"num_read_kb":7346,"num_write":375,"num_write_kb":8437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":8153977,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"134'425","reported_seq":1162,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.457472+0000","last_change":"2024-09-18T00:20:17.696586+0000","last_active":"2024-09-18T00:20:48.457472+0000","last_peered":"2024-09-18T00:20:48.457472+0000","last_clean":"2024-09-18T00:20:48.457472+0000","last_became_active":"2024-09-18T00:18:05.406977+0000","last_became_peered":"2024-09-18T00:18:05.406977+0000","last_unstale":"2024-09-18T00:20:48.457472+0000","last_undegraded":"2024-09-18T00:20:48.457472+0000","last_fullsized":"2024-09-18T00:20:48.457472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:06:59.846008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044405699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1237,"num_read_kb":3121,"num_write":630,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"135'1438","reported_seq":3731,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.467838+0000","last_change":"2024-09-18T00:20:48.467838+0000","last_active":"2024-09-18T00:20:48.467838+0000","last_peered":"2024-09-18T00:20:48.467838+0000","last_clean":"2024-09-18T00:20:48.467838+0000","last_became_active":"2024-09-18T00:20:48.462359+0000","last_became_peered":"2024-09-18T00:20:48.462359+0000","last_unstale":"2024-09-18T00:20:48.467838+0000","last_undegraded":"2024-09-18T00:20:48.467838+0000","last_fullsized":"2024-09-18T00:20:48.467838+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":1438,"log_dups_size":0,"ondisk_log_size":1438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:28:28.471113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032308089999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9631,"num_read_kb":9532,"num_write":2186,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":392734,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"134'780","reported_seq":3314,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463834+0000","last_change":"2024-09-18T00:20:17.579849+0000","last_active":"2024-09-18T00:20:49.463834+0000","last_peered":"2024-09-18T00:20:49.463834+0000","last_clean":"2024-09-18T00:20:49.463834+0000","last_became_active":"2024-09-18T00:19:18.333730+0000","last_became_peered":"2024-09-18T00:19:18.333730+0000","last_unstale":"2024-09-18T00:20:49.463834+0000","last_undegraded":"2024-09-18T00:20:49.463834+0000","last_fullsized":"2024-09-18T00:20:49.463834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:12:14.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029127850000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5845,"num_read_kb":7620,"num_write":1029,"num_write_kb":6233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":697208,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"134'352","reported_seq":1424,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463902+0000","last_change":"2024-09-18T00:20:17.578670+0000","last_active":"2024-09-18T00:20:49.463902+0000","last_peered":"2024-09-18T00:20:49.463902+0000","last_clean":"2024-09-18T00:20:49.463902+0000","last_became_active":"2024-09-18T00:18:16.440223+0000","last_became_peered":"2024-09-18T00:18:16.440223+0000","last_unstale":"2024-09-18T00:20:49.463902+0000","last_undegraded":"2024-09-18T00:20:49.463902+0000","last_fullsized":"2024-09-18T00:20:49.463902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:17.972019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002121001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1129,"num_read_kb":6709,"num_write":391,"num_write_kb":4357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":7258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"133'397","reported_seq":3507,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.464104+0000","last_change":"2024-09-18T00:20:17.579351+0000","last_active":"2024-09-18T00:20:49.464104+0000","last_peered":"2024-09-18T00:20:49.464104+0000","last_clean":"2024-09-18T00:20:49.464104+0000","last_became_active":"2024-09-18T00:18:16.441020+0000","last_became_peered":"2024-09-18T00:18:16.441020+0000","last_unstale":"2024-09-18T00:20:49.464104+0000","last_undegraded":"2024-09-18T00:20:49.464104+0000","last_fullsized":"2024-09-18T00:20:49.464104+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:01:45.246626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00266241,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4197,"num_read_kb":11008,"num_write":397,"num_write_kb":9044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"134'378","reported_seq":4809,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451504+0000","last_change":"2024-09-18T00:20:37.726301+0000","last_active":"2024-09-18T00:20:48.451504+0000","last_peered":"2024-09-18T00:20:48.451504+0000","last_clean":"2024-09-18T00:20:48.451504+0000","last_became_active":"2024-09-18T00:20:37.723504+0000","last_became_peered":"2024-09-18T00:20:37.723504+0000","last_unstale":"2024-09-18T00:20:48.451504+0000","last_undegraded":"2024-09-18T00:20:48.451504+0000","last_fullsized":"2024-09-18T00:20:48.451504+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:16:42.705342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001519245,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5107,"num_read_kb":32748,"num_write":405,"num_write_kb":13893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":3037482,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"133'368","reported_seq":4657,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463399+0000","last_change":"2024-09-18T00:20:17.577141+0000","last_active":"2024-09-18T00:20:49.463399+0000","last_peered":"2024-09-18T00:20:49.463399+0000","last_clean":"2024-09-18T00:20:49.463399+0000","last_became_active":"2024-09-18T00:18:16.438286+0000","last_became_peered":"2024-09-18T00:18:16.438286+0000","last_unstale":"2024-09-18T00:20:49.463399+0000","last_undegraded":"2024-09-18T00:20:49.463399+0000","last_fullsized":"2024-09-18T00:20:49.463399+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:09:46.156797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092097700000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4916,"num_read_kb":14028,"num_write":359,"num_write_kb":4057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":70145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"134'355","reported_seq":1976,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451372+0000","last_change":"2024-09-18T00:20:17.572303+0000","last_active":"2024-09-18T00:20:48.451372+0000","last_peered":"2024-09-18T00:20:48.451372+0000","last_clean":"2024-09-18T00:20:48.451372+0000","last_became_active":"2024-09-18T00:18:16.438556+0000","last_became_peered":"2024-09-18T00:18:16.438556+0000","last_unstale":"2024-09-18T00:20:48.451372+0000","last_undegraded":"2024-09-18T00:20:48.451372+0000","last_fullsized":"2024-09-18T00:20:48.451372+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:36.881982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019750979999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1630,"num_read_kb":9063,"num_write":386,"num_write_kb":8736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":47120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"133'748","reported_seq":3136,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463471+0000","last_change":"2024-09-18T00:20:17.579483+0000","last_active":"2024-09-18T00:20:49.463471+0000","last_peered":"2024-09-18T00:20:49.463471+0000","last_clean":"2024-09-18T00:20:49.463471+0000","last_became_active":"2024-09-18T00:18:16.440910+0000","last_became_peered":"2024-09-18T00:18:16.440910+0000","last_unstale":"2024-09-18T00:20:49.463471+0000","last_undegraded":"2024-09-18T00:20:49.463471+0000","last_fullsized":"2024-09-18T00:20:49.463471+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:00:51.845256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028046910000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4645,"num_read_kb":12707,"num_write":1147,"num_write_kb":3134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":65,"num_read":96502,"num_read_kb":273433,"num_write":19290,"num_write_kb":181728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1834,"num_bytes_recovered":25828399,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19623,"internal_metadata":0},"log_size":16196,"ondisk_log_size":16196,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20247,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55260,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316580,"statfs":{"total":96636764160,"available":96580177920,"internally_reserved":0,"allocated":1908736,"data_stored":1478380,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37172,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334668,"statfs":{"total":96636764160,"available":96598700032,"internally_reserved":0,"allocated":1867776,"data_stored":888012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":76,"seq":326417514517,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38208,"kb_used_data":1828,"kb_used_omap":9,"kb_used_meta":36342,"kb_avail":94333632,"statfs":{"total":96636764160,"available":96597639168,"internally_reserved":0,"allocated":1871872,"data_stored":888031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9346,"internal_metadata":37215102},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55228,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1871872,"data_stored":888031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41344,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330496,"statfs":{"total":96636764160,"available":96594427904,"internally_reserved":0,"allocated":1880064,"data_stored":893854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":139,"seq":597000454150,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":1844,"kb_used_omap":17,"kb_used_meta":30190,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":1888256,"data_stored":893892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17900,"internal_metadata":30915092},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542184,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73280,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298560,"statfs":{"total":96636764160,"available":96561725440,"internally_reserved":0,"allocated":1867776,"data_stored":888012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1925120,"data_stored":1478380,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13445,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6178,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:20:54.297 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:20:54.575 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:20:54.575 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-18T00:20:54.591 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-18T00:20:54.348193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":65,"num_read":96618,"num_read_kb":273533,"num_write":19451,"num_write_kb":184146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1839,"num_bytes_recovered":27009154,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16316,"ondisk_log_size":16316,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":365648,"kb_used_data":14752,"kb_used_omap":35,"kb_used_meta":350684,"kb_avail":754609072,"statfs":{"total":773094113280,"available":772719689728,"internally_reserved":0,"allocated":15106048,"data_stored":8308276,"data_compressed":13209,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":36807,"internal_metadata":359100473},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.911748"},"pg_stats":[{"pgid":"3.1f","version":"135'778","reported_seq":3008,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451528+0000","last_change":"2024-09-18T00:20:17.573423+0000","last_active":"2024-09-18T00:20:48.451528+0000","last_peered":"2024-09-18T00:20:48.451528+0000","last_clean":"2024-09-18T00:20:48.451528+0000","last_became_active":"2024-09-18T00:19:18.327350+0000","last_became_peered":"2024-09-18T00:19:18.327350+0000","last_unstale":"2024-09-18T00:20:48.451528+0000","last_undegraded":"2024-09-18T00:20:48.451528+0000","last_fullsized":"2024-09-18T00:20:48.451528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":778,"log_dups_size":0,"ondisk_log_size":778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:46:14.185856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024250489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4587,"num_read_kb":13915,"num_write":1017,"num_write_kb":7086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":2252743,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"133'465","reported_seq":1559,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095625+0000","last_change":"2024-09-18T00:20:37.723738+0000","last_active":"2024-09-18T00:20:51.095625+0000","last_peered":"2024-09-18T00:20:51.095625+0000","last_clean":"2024-09-18T00:20:51.095625+0000","last_became_active":"2024-09-18T00:20:37.722312+0000","last_became_peered":"2024-09-18T00:20:37.722312+0000","last_unstale":"2024-09-18T00:20:51.095625+0000","last_undegraded":"2024-09-18T00:20:51.095625+0000","last_fullsized":"2024-09-18T00:20:51.095625+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:14:23.427889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057225740000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1405,"num_read_kb":1924,"num_write":538,"num_write_kb":2847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":46741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"135'781","reported_seq":1532,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097665+0000","last_change":"2024-09-18T00:20:48.470232+0000","last_active":"2024-09-18T00:20:51.097665+0000","last_peered":"2024-09-18T00:20:51.097665+0000","last_clean":"2024-09-18T00:20:51.097665+0000","last_became_active":"2024-09-18T00:20:48.465769+0000","last_became_peered":"2024-09-18T00:20:48.465769+0000","last_unstale":"2024-09-18T00:20:51.097665+0000","last_undegraded":"2024-09-18T00:20:51.097665+0000","last_fullsized":"2024-09-18T00:20:51.097665+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:11:07.738987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032750750000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3226,"num_read_kb":4864,"num_write":1184,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":510608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"135'453","reported_seq":1953,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097692+0000","last_change":"2024-09-18T00:20:48.467870+0000","last_active":"2024-09-18T00:20:51.097692+0000","last_peered":"2024-09-18T00:20:51.097692+0000","last_clean":"2024-09-18T00:20:51.097692+0000","last_became_active":"2024-09-18T00:20:48.463565+0000","last_became_peered":"2024-09-18T00:20:48.463565+0000","last_unstale":"2024-09-18T00:20:51.097692+0000","last_undegraded":"2024-09-18T00:20:51.097692+0000","last_fullsized":"2024-09-18T00:20:51.097692+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:07:09.913555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002116522,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3432,"num_read_kb":9919,"num_write":470,"num_write_kb":3833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":72221,"num_keys_recovered":54,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"133'289","reported_seq":1665,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451394+0000","last_change":"2024-09-18T00:20:17.572140+0000","last_active":"2024-09-18T00:20:48.451394+0000","last_peered":"2024-09-18T00:20:48.451394+0000","last_clean":"2024-09-18T00:20:48.451394+0000","last_became_active":"2024-09-18T00:18:05.415810+0000","last_became_peered":"2024-09-18T00:18:05.415810+0000","last_unstale":"2024-09-18T00:20:48.451394+0000","last_undegraded":"2024-09-18T00:20:48.451394+0000","last_fullsized":"2024-09-18T00:20:48.451394+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:35:11.075422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015232959999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1498,"num_read_kb":2662,"num_write":281,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"140'226","reported_seq":3328,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459066+0000","last_change":"2024-09-18T00:20:30.686023+0000","last_active":"2024-09-18T00:20:48.459066+0000","last_peered":"2024-09-18T00:20:48.459066+0000","last_clean":"2024-09-18T00:20:48.459066+0000","last_became_active":"2024-09-18T00:20:30.637311+0000","last_became_peered":"2024-09-18T00:20:30.637311+0000","last_unstale":"2024-09-18T00:20:48.459066+0000","last_undegraded":"2024-09-18T00:20:48.459066+0000","last_fullsized":"2024-09-18T00:20:48.459066+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:56:01.105381+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012894545,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":2320,"num_read_kb":4551,"num_write":211,"num_write_kb":2083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":645956,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"140'384","reported_seq":1512,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462374+0000","last_change":"2024-09-18T00:20:30.693807+0000","last_active":"2024-09-18T00:20:49.462374+0000","last_peered":"2024-09-18T00:20:49.462374+0000","last_clean":"2024-09-18T00:20:49.462374+0000","last_became_active":"2024-09-18T00:20:30.640499+0000","last_became_peered":"2024-09-18T00:20:30.640499+0000","last_unstale":"2024-09-18T00:20:49.462374+0000","last_undegraded":"2024-09-18T00:20:49.462374+0000","last_fullsized":"2024-09-18T00:20:49.462374+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:26.838200+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018217595999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1131,"num_read_kb":4419,"num_write":389,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":1283165,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"133'327","reported_seq":3718,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095462+0000","last_change":"2024-09-18T00:20:17.694311+0000","last_active":"2024-09-18T00:20:51.095462+0000","last_peered":"2024-09-18T00:20:51.095462+0000","last_clean":"2024-09-18T00:20:51.095462+0000","last_became_active":"2024-09-18T00:18:05.406821+0000","last_became_peered":"2024-09-18T00:18:05.406821+0000","last_unstale":"2024-09-18T00:20:51.095462+0000","last_undegraded":"2024-09-18T00:20:51.095462+0000","last_fullsized":"2024-09-18T00:20:51.095462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:13:08.591812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022731449999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3612,"num_read_kb":21086,"num_write":298,"num_write_kb":8166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'372","reported_seq":1395,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451657+0000","last_change":"2024-09-18T00:20:17.573988+0000","last_active":"2024-09-18T00:20:48.451657+0000","last_peered":"2024-09-18T00:20:48.451657+0000","last_clean":"2024-09-18T00:20:48.451657+0000","last_became_active":"2024-09-18T00:19:18.328043+0000","last_became_peered":"2024-09-18T00:19:18.328043+0000","last_unstale":"2024-09-18T00:20:48.451657+0000","last_undegraded":"2024-09-18T00:20:48.451657+0000","last_fullsized":"2024-09-18T00:20:48.451657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:11:44.153392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025255270000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1318,"num_read_kb":6224,"num_write":322,"num_write_kb":4815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":491875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"135'690","reported_seq":2488,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097894+0000","last_change":"2024-09-18T00:20:48.471288+0000","last_active":"2024-09-18T00:20:51.097894+0000","last_peered":"2024-09-18T00:20:51.097894+0000","last_clean":"2024-09-18T00:20:51.097894+0000","last_became_active":"2024-09-18T00:20:48.467364+0000","last_became_peered":"2024-09-18T00:20:48.467364+0000","last_unstale":"2024-09-18T00:20:51.097894+0000","last_undegraded":"2024-09-18T00:20:51.097894+0000","last_fullsized":"2024-09-18T00:20:51.097894+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:25.555616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021376099999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4502,"num_read_kb":5989,"num_write":925,"num_write_kb":5689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1385456,"num_keys_recovered":29,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"133'489","reported_seq":1056,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463552+0000","last_change":"2024-09-18T00:20:17.577703+0000","last_active":"2024-09-18T00:20:49.463552+0000","last_peered":"2024-09-18T00:20:49.463552+0000","last_clean":"2024-09-18T00:20:49.463552+0000","last_became_active":"2024-09-18T00:18:16.438164+0000","last_became_peered":"2024-09-18T00:18:16.438164+0000","last_unstale":"2024-09-18T00:20:49.463552+0000","last_undegraded":"2024-09-18T00:20:49.463552+0000","last_fullsized":"2024-09-18T00:20:49.463552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:18:47.201515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014879730000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":1078,"num_read_kb":1346,"num_write":552,"num_write_kb":3993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"134'530","reported_seq":1981,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459503+0000","last_change":"2024-09-18T00:20:17.576876+0000","last_active":"2024-09-18T00:20:48.459503+0000","last_peered":"2024-09-18T00:20:48.459503+0000","last_clean":"2024-09-18T00:20:48.459503+0000","last_became_active":"2024-09-18T00:18:16.441758+0000","last_became_peered":"2024-09-18T00:18:16.441758+0000","last_unstale":"2024-09-18T00:20:48.459503+0000","last_undegraded":"2024-09-18T00:20:48.459503+0000","last_fullsized":"2024-09-18T00:20:48.459503+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:18:47.366485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050089940000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2844,"num_read_kb":5653,"num_write":555,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":103027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":38,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451888+0000","last_change":"2024-09-18T00:20:34.754460+0000","last_active":"2024-09-18T00:20:48.451888+0000","last_peered":"2024-09-18T00:20:48.451888+0000","last_clean":"2024-09-18T00:20:48.451888+0000","last_became_active":"2024-09-18T00:20:34.753675+0000","last_became_peered":"2024-09-18T00:20:34.753675+0000","last_unstale":"2024-09-18T00:20:48.451888+0000","last_undegraded":"2024-09-18T00:20:48.451888+0000","last_fullsized":"2024-09-18T00:20:48.451888+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:37.203669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089554000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"135'348","reported_seq":2423,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.452208+0000","last_change":"2024-09-18T00:20:30.727046+0000","last_active":"2024-09-18T00:20:48.452208+0000","last_peered":"2024-09-18T00:20:48.452208+0000","last_clean":"2024-09-18T00:20:48.452208+0000","last_became_active":"2024-09-18T00:20:30.648069+0000","last_became_peered":"2024-09-18T00:20:30.648069+0000","last_unstale":"2024-09-18T00:20:48.452208+0000","last_undegraded":"2024-09-18T00:20:48.452208+0000","last_fullsized":"2024-09-18T00:20:48.452208+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:36.621517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052221869999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2016,"num_read_kb":5374,"num_write":329,"num_write_kb":5799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":1635567,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":59,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462482+0000","last_change":"2024-09-18T00:20:34.694081+0000","last_active":"2024-09-18T00:20:49.462482+0000","last_peered":"2024-09-18T00:20:49.462482+0000","last_clean":"2024-09-18T00:20:49.462482+0000","last_became_active":"2024-09-18T00:20:34.693550+0000","last_became_peered":"2024-09-18T00:20:34.693550+0000","last_unstale":"2024-09-18T00:20:49.462482+0000","last_undegraded":"2024-09-18T00:20:49.462482+0000","last_fullsized":"2024-09-18T00:20:49.462482+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:34:59.142586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'665","reported_seq":3138,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095867+0000","last_change":"2024-09-18T00:20:17.695571+0000","last_active":"2024-09-18T00:20:51.095867+0000","last_peered":"2024-09-18T00:20:51.095867+0000","last_clean":"2024-09-18T00:20:51.095867+0000","last_became_active":"2024-09-18T00:18:16.440536+0000","last_became_peered":"2024-09-18T00:18:16.440536+0000","last_unstale":"2024-09-18T00:20:51.095867+0000","last_undegraded":"2024-09-18T00:20:51.095867+0000","last_fullsized":"2024-09-18T00:20:51.095867+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:35.729973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034870019999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":3858,"num_read_kb":13564,"num_write":915,"num_write_kb":9611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":73512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":257,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463801+0000","last_change":"2024-09-18T00:18:16.439964+0000","last_active":"2024-09-18T00:20:49.463801+0000","last_peered":"2024-09-18T00:20:49.463801+0000","last_clean":"2024-09-18T00:20:49.463801+0000","last_became_active":"2024-09-18T00:18:16.437150+0000","last_became_peered":"2024-09-18T00:18:16.437150+0000","last_unstale":"2024-09-18T00:20:49.463801+0000","last_undegraded":"2024-09-18T00:20:49.463801+0000","last_fullsized":"2024-09-18T00:20:49.463801+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:25.772294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"140'491","reported_seq":2308,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462407+0000","last_change":"2024-09-18T00:20:30.710100+0000","last_active":"2024-09-18T00:20:49.462407+0000","last_peered":"2024-09-18T00:20:49.462407+0000","last_clean":"2024-09-18T00:20:49.462407+0000","last_became_active":"2024-09-18T00:20:30.641845+0000","last_became_peered":"2024-09-18T00:20:30.641845+0000","last_unstale":"2024-09-18T00:20:49.462407+0000","last_undegraded":"2024-09-18T00:20:49.462407+0000","last_fullsized":"2024-09-18T00:20:49.462407+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:00:46.444023+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.013533892,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2426,"num_read_kb":4923,"num_write":423,"num_write_kb":4982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":1553378,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462313+0000","last_change":"2024-09-18T00:20:30.651121+0000","last_active":"2024-09-18T00:20:49.462313+0000","last_peered":"2024-09-18T00:20:49.462313+0000","last_clean":"2024-09-18T00:20:49.462313+0000","last_became_active":"2024-09-18T00:20:30.650373+0000","last_became_peered":"2024-09-18T00:20:30.650373+0000","last_unstale":"2024-09-18T00:20:49.462313+0000","last_undegraded":"2024-09-18T00:20:49.462313+0000","last_fullsized":"2024-09-18T00:20:49.462313+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:36:36.287145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"133'363","reported_seq":2714,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097799+0000","last_change":"2024-09-18T00:20:48.470308+0000","last_active":"2024-09-18T00:20:51.097799+0000","last_peered":"2024-09-18T00:20:51.097799+0000","last_clean":"2024-09-18T00:20:51.097799+0000","last_became_active":"2024-09-18T00:20:48.462691+0000","last_became_peered":"2024-09-18T00:20:48.462691+0000","last_unstale":"2024-09-18T00:20:51.097799+0000","last_undegraded":"2024-09-18T00:20:51.097799+0000","last_fullsized":"2024-09-18T00:20:51.097799+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:08.083732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001889571,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2213,"num_read_kb":12284,"num_write":366,"num_write_kb":5310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":675014,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"135'484","reported_seq":2135,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459435+0000","last_change":"2024-09-18T00:20:17.573853+0000","last_active":"2024-09-18T00:20:48.459435+0000","last_peered":"2024-09-18T00:20:48.459435+0000","last_clean":"2024-09-18T00:20:48.459435+0000","last_became_active":"2024-09-18T00:18:05.407889+0000","last_became_peered":"2024-09-18T00:18:05.407889+0000","last_unstale":"2024-09-18T00:20:48.459435+0000","last_undegraded":"2024-09-18T00:20:48.459435+0000","last_fullsized":"2024-09-18T00:20:48.459435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:40.520294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002502663,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":1766,"num_read_kb":5407,"num_write":446,"num_write_kb":5379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":203,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.098054+0000","last_change":"2024-09-18T00:20:48.461100+0000","last_active":"2024-09-18T00:20:51.098054+0000","last_peered":"2024-09-18T00:20:51.098054+0000","last_clean":"2024-09-18T00:20:51.098054+0000","last_became_active":"2024-09-18T00:20:48.460277+0000","last_became_peered":"2024-09-18T00:20:48.460277+0000","last_unstale":"2024-09-18T00:20:51.098054+0000","last_undegraded":"2024-09-18T00:20:51.098054+0000","last_fullsized":"2024-09-18T00:20:51.098054+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:01:02.391844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088189300000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"134'369","reported_seq":1075,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462205+0000","last_change":"2024-09-18T00:20:30.726339+0000","last_active":"2024-09-18T00:20:49.462205+0000","last_peered":"2024-09-18T00:20:49.462205+0000","last_clean":"2024-09-18T00:20:49.462205+0000","last_became_active":"2024-09-18T00:20:30.647777+0000","last_became_peered":"2024-09-18T00:20:30.647777+0000","last_unstale":"2024-09-18T00:20:49.462205+0000","last_undegraded":"2024-09-18T00:20:49.462205+0000","last_fullsized":"2024-09-18T00:20:49.462205+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:21:04.035067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011927055000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":740,"num_read_kb":1879,"num_write":446,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":1002237,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":289,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.461675+0000","last_change":"2024-09-18T00:17:57.477633+0000","last_active":"2024-09-18T00:20:49.461675+0000","last_peered":"2024-09-18T00:20:49.461675+0000","last_clean":"2024-09-18T00:20:49.461675+0000","last_became_active":"2024-09-18T00:17:55.631277+0000","last_became_peered":"2024-09-18T00:17:55.631277+0000","last_unstale":"2024-09-18T00:20:49.461675+0000","last_undegraded":"2024-09-18T00:20:49.461675+0000","last_fullsized":"2024-09-18T00:20:49.461675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:55.421684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001141204,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"135'502","reported_seq":2618,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451329+0000","last_change":"2024-09-18T00:20:17.571764+0000","last_active":"2024-09-18T00:20:48.451329+0000","last_peered":"2024-09-18T00:20:48.451329+0000","last_clean":"2024-09-18T00:20:48.451329+0000","last_became_active":"2024-09-18T00:18:05.410952+0000","last_became_peered":"2024-09-18T00:18:05.410952+0000","last_unstale":"2024-09-18T00:20:48.451329+0000","last_undegraded":"2024-09-18T00:20:48.451329+0000","last_fullsized":"2024-09-18T00:20:48.451329+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:15.435600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014272110000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3308,"num_read_kb":10125,"num_write":594,"num_write_kb":8641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":276,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463375+0000","last_change":"2024-09-18T00:20:30.657193+0000","last_active":"2024-09-18T00:20:49.463375+0000","last_peered":"2024-09-18T00:20:49.463375+0000","last_clean":"2024-09-18T00:20:49.463375+0000","last_became_active":"2024-09-18T00:20:30.656400+0000","last_became_peered":"2024-09-18T00:20:30.656400+0000","last_unstale":"2024-09-18T00:20:49.463375+0000","last_undegraded":"2024-09-18T00:20:49.463375+0000","last_fullsized":"2024-09-18T00:20:49.463375+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:50:27.874106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"134'493","reported_seq":2493,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095513+0000","last_change":"2024-09-18T00:20:17.697271+0000","last_active":"2024-09-18T00:20:51.095513+0000","last_peered":"2024-09-18T00:20:51.095513+0000","last_clean":"2024-09-18T00:20:51.095513+0000","last_became_active":"2024-09-18T00:19:18.330155+0000","last_became_peered":"2024-09-18T00:19:18.330155+0000","last_unstale":"2024-09-18T00:20:51.095513+0000","last_undegraded":"2024-09-18T00:20:51.095513+0000","last_fullsized":"2024-09-18T00:20:51.095513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:01:28.392225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005129074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2497,"num_read_kb":7710,"num_write":474,"num_write_kb":3192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":493176,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":122,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097622+0000","last_change":"2024-09-18T00:20:48.459774+0000","last_active":"2024-09-18T00:20:51.097622+0000","last_peered":"2024-09-18T00:20:51.097622+0000","last_clean":"2024-09-18T00:20:51.097622+0000","last_became_active":"2024-09-18T00:20:48.457808+0000","last_became_peered":"2024-09-18T00:20:48.457808+0000","last_unstale":"2024-09-18T00:20:51.097622+0000","last_undegraded":"2024-09-18T00:20:51.097622+0000","last_fullsized":"2024-09-18T00:20:51.097622+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:43:17.325507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":438,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463383+0000","last_change":"2024-09-18T00:18:19.466612+0000","last_active":"2024-09-18T00:20:49.463383+0000","last_peered":"2024-09-18T00:20:49.463383+0000","last_clean":"2024-09-18T00:20:49.463383+0000","last_became_active":"2024-09-18T00:18:19.465557+0000","last_became_peered":"2024-09-18T00:18:19.465557+0000","last_unstale":"2024-09-18T00:20:49.463383+0000","last_undegraded":"2024-09-18T00:20:49.463383+0000","last_fullsized":"2024-09-18T00:20:49.463383+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_clean_scrub_stamp":"2024-09-18T00:17:50.414591+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:08:40.389972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"140'661","reported_seq":2377,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095512+0000","last_change":"2024-09-18T00:20:30.742660+0000","last_active":"2024-09-18T00:20:51.095512+0000","last_peered":"2024-09-18T00:20:51.095512+0000","last_clean":"2024-09-18T00:20:51.095512+0000","last_became_active":"2024-09-18T00:20:30.656699+0000","last_became_peered":"2024-09-18T00:20:30.656699+0000","last_unstale":"2024-09-18T00:20:51.095512+0000","last_undegraded":"2024-09-18T00:20:51.095512+0000","last_fullsized":"2024-09-18T00:20:51.095512+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:37:48.645340+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0070126900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3348,"num_read_kb":6619,"num_write":872,"num_write_kb":5519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":1197799,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"134'383","reported_seq":2112,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459075+0000","last_change":"2024-09-18T00:20:17.574821+0000","last_active":"2024-09-18T00:20:48.459075+0000","last_peered":"2024-09-18T00:20:48.459075+0000","last_clean":"2024-09-18T00:20:48.459075+0000","last_became_active":"2024-09-18T00:18:05.401097+0000","last_became_peered":"2024-09-18T00:18:05.401097+0000","last_unstale":"2024-09-18T00:20:48.459075+0000","last_undegraded":"2024-09-18T00:20:48.459075+0000","last_fullsized":"2024-09-18T00:20:48.459075+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:50:58.016930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003875547,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3069,"num_read_kb":9114,"num_write":378,"num_write_kb":6107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"140'412","reported_seq":2388,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451796+0000","last_change":"2024-09-18T00:20:37.726930+0000","last_active":"2024-09-18T00:20:48.451796+0000","last_peered":"2024-09-18T00:20:48.451796+0000","last_clean":"2024-09-18T00:20:48.451796+0000","last_became_active":"2024-09-18T00:20:37.721802+0000","last_became_peered":"2024-09-18T00:20:37.721802+0000","last_unstale":"2024-09-18T00:20:48.451796+0000","last_undegraded":"2024-09-18T00:20:48.451796+0000","last_fullsized":"2024-09-18T00:20:48.451796+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:37.117595+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.007118643,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":1971,"num_read_kb":7346,"num_write":375,"num_write_kb":8437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":8153977,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"134'425","reported_seq":1164,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095561+0000","last_change":"2024-09-18T00:20:17.696586+0000","last_active":"2024-09-18T00:20:51.095561+0000","last_peered":"2024-09-18T00:20:51.095561+0000","last_clean":"2024-09-18T00:20:51.095561+0000","last_became_active":"2024-09-18T00:18:05.406977+0000","last_became_peered":"2024-09-18T00:18:05.406977+0000","last_unstale":"2024-09-18T00:20:51.095561+0000","last_undegraded":"2024-09-18T00:20:51.095561+0000","last_fullsized":"2024-09-18T00:20:51.095561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:06:59.846008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044405699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1237,"num_read_kb":3121,"num_write":630,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"135'1438","reported_seq":3733,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097706+0000","last_change":"2024-09-18T00:20:48.467838+0000","last_active":"2024-09-18T00:20:51.097706+0000","last_peered":"2024-09-18T00:20:51.097706+0000","last_clean":"2024-09-18T00:20:51.097706+0000","last_became_active":"2024-09-18T00:20:48.462359+0000","last_became_peered":"2024-09-18T00:20:48.462359+0000","last_unstale":"2024-09-18T00:20:51.097706+0000","last_undegraded":"2024-09-18T00:20:51.097706+0000","last_fullsized":"2024-09-18T00:20:51.097706+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":1438,"log_dups_size":0,"ondisk_log_size":1438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:28:28.471113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032308089999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9631,"num_read_kb":9532,"num_write":2186,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":392734,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"134'780","reported_seq":3314,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463834+0000","last_change":"2024-09-18T00:20:17.579849+0000","last_active":"2024-09-18T00:20:49.463834+0000","last_peered":"2024-09-18T00:20:49.463834+0000","last_clean":"2024-09-18T00:20:49.463834+0000","last_became_active":"2024-09-18T00:19:18.333730+0000","last_became_peered":"2024-09-18T00:19:18.333730+0000","last_unstale":"2024-09-18T00:20:49.463834+0000","last_undegraded":"2024-09-18T00:20:49.463834+0000","last_fullsized":"2024-09-18T00:20:49.463834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:12:14.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029127850000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5845,"num_read_kb":7620,"num_write":1029,"num_write_kb":6233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":697208,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"134'352","reported_seq":1424,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463902+0000","last_change":"2024-09-18T00:20:17.578670+0000","last_active":"2024-09-18T00:20:49.463902+0000","last_peered":"2024-09-18T00:20:49.463902+0000","last_clean":"2024-09-18T00:20:49.463902+0000","last_became_active":"2024-09-18T00:18:16.440223+0000","last_became_peered":"2024-09-18T00:18:16.440223+0000","last_unstale":"2024-09-18T00:20:49.463902+0000","last_undegraded":"2024-09-18T00:20:49.463902+0000","last_fullsized":"2024-09-18T00:20:49.463902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:17.972019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002121001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1129,"num_read_kb":6709,"num_write":391,"num_write_kb":4357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":7258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"133'397","reported_seq":3507,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.464104+0000","last_change":"2024-09-18T00:20:17.579351+0000","last_active":"2024-09-18T00:20:49.464104+0000","last_peered":"2024-09-18T00:20:49.464104+0000","last_clean":"2024-09-18T00:20:49.464104+0000","last_became_active":"2024-09-18T00:18:16.441020+0000","last_became_peered":"2024-09-18T00:18:16.441020+0000","last_unstale":"2024-09-18T00:20:49.464104+0000","last_undegraded":"2024-09-18T00:20:49.464104+0000","last_fullsized":"2024-09-18T00:20:49.464104+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:01:45.246626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00266241,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4197,"num_read_kb":11008,"num_write":397,"num_write_kb":9044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"134'378","reported_seq":4809,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451504+0000","last_change":"2024-09-18T00:20:37.726301+0000","last_active":"2024-09-18T00:20:48.451504+0000","last_peered":"2024-09-18T00:20:48.451504+0000","last_clean":"2024-09-18T00:20:48.451504+0000","last_became_active":"2024-09-18T00:20:37.723504+0000","last_became_peered":"2024-09-18T00:20:37.723504+0000","last_unstale":"2024-09-18T00:20:48.451504+0000","last_undegraded":"2024-09-18T00:20:48.451504+0000","last_fullsized":"2024-09-18T00:20:48.451504+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:16:42.705342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001519245,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5107,"num_read_kb":32748,"num_write":405,"num_write_kb":13893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":3037482,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"133'368","reported_seq":4657,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463399+0000","last_change":"2024-09-18T00:20:17.577141+0000","last_active":"2024-09-18T00:20:49.463399+0000","last_peered":"2024-09-18T00:20:49.463399+0000","last_clean":"2024-09-18T00:20:49.463399+0000","last_became_active":"2024-09-18T00:18:16.438286+0000","last_became_peered":"2024-09-18T00:18:16.438286+0000","last_unstale":"2024-09-18T00:20:49.463399+0000","last_undegraded":"2024-09-18T00:20:49.463399+0000","last_fullsized":"2024-09-18T00:20:49.463399+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:09:46.156797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092097700000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4916,"num_read_kb":14028,"num_write":359,"num_write_kb":4057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":70145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"134'355","reported_seq":1976,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451372+0000","last_change":"2024-09-18T00:20:17.572303+0000","last_active":"2024-09-18T00:20:48.451372+0000","last_peered":"2024-09-18T00:20:48.451372+0000","last_clean":"2024-09-18T00:20:48.451372+0000","last_became_active":"2024-09-18T00:18:16.438556+0000","last_became_peered":"2024-09-18T00:18:16.438556+0000","last_unstale":"2024-09-18T00:20:48.451372+0000","last_undegraded":"2024-09-18T00:20:48.451372+0000","last_fullsized":"2024-09-18T00:20:48.451372+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:36.881982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019750979999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1630,"num_read_kb":9063,"num_write":386,"num_write_kb":8736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":47120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"133'748","reported_seq":3136,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463471+0000","last_change":"2024-09-18T00:20:17.579483+0000","last_active":"2024-09-18T00:20:49.463471+0000","last_peered":"2024-09-18T00:20:49.463471+0000","last_clean":"2024-09-18T00:20:49.463471+0000","last_became_active":"2024-09-18T00:18:16.440910+0000","last_became_peered":"2024-09-18T00:18:16.440910+0000","last_unstale":"2024-09-18T00:20:49.463471+0000","last_undegraded":"2024-09-18T00:20:49.463471+0000","last_fullsized":"2024-09-18T00:20:49.463471+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:00:51.845256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028046910000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4645,"num_read_kb":12707,"num_write":1147,"num_write_kb":3134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":65,"num_read":96502,"num_read_kb":273433,"num_write":19290,"num_write_kb":181728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1834,"num_bytes_recovered":25828399,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19623,"internal_metadata":0},"log_size":16196,"ondisk_log_size":16196,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20247,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55260,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316580,"statfs":{"total":96636764160,"available":96580177920,"internally_reserved":0,"allocated":1908736,"data_stored":1478380,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37172,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334668,"statfs":{"total":96636764160,"available":96598700032,"internally_reserved":0,"allocated":1867776,"data_stored":888012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":76,"seq":326417514518,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38212,"kb_used_data":1840,"kb_used_omap":9,"kb_used_meta":36342,"kb_avail":94333628,"statfs":{"total":96636764160,"available":96597635072,"internally_reserved":0,"allocated":1884160,"data_stored":893873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9346,"internal_metadata":37215102},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55228,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1871872,"data_stored":888031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41344,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330496,"statfs":{"total":96636764160,"available":96594427904,"internally_reserved":0,"allocated":1880064,"data_stored":893854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":139,"seq":597000454151,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":1844,"kb_used_omap":17,"kb_used_meta":30190,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":1888256,"data_stored":893892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17900,"internal_metadata":30915092},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542185,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73280,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298560,"statfs":{"total":96636764160,"available":96561725440,"internally_reserved":0,"allocated":1880064,"data_stored":893854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1925120,"data_stored":1478380,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13445,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6178,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:20:54.593 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T00:20:54.593 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-18T00:20:54.893 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-18T00:20:54.902 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T00:20:54.902 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:20:55.177 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:20:55.178 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-18T00:20:55.194 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-18T00:20:54.348193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":65,"num_read":96618,"num_read_kb":273533,"num_write":19451,"num_write_kb":184146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1839,"num_bytes_recovered":27009154,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16316,"ondisk_log_size":16316,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":365648,"kb_used_data":14752,"kb_used_omap":35,"kb_used_meta":350684,"kb_avail":754609072,"statfs":{"total":773094113280,"available":772719689728,"internally_reserved":0,"allocated":15106048,"data_stored":8308276,"data_compressed":13209,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":36807,"internal_metadata":359100473},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.911748"},"pg_stats":[{"pgid":"3.1f","version":"135'778","reported_seq":3008,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451528+0000","last_change":"2024-09-18T00:20:17.573423+0000","last_active":"2024-09-18T00:20:48.451528+0000","last_peered":"2024-09-18T00:20:48.451528+0000","last_clean":"2024-09-18T00:20:48.451528+0000","last_became_active":"2024-09-18T00:19:18.327350+0000","last_became_peered":"2024-09-18T00:19:18.327350+0000","last_unstale":"2024-09-18T00:20:48.451528+0000","last_undegraded":"2024-09-18T00:20:48.451528+0000","last_fullsized":"2024-09-18T00:20:48.451528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":778,"log_dups_size":0,"ondisk_log_size":778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:46:14.185856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024250489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4587,"num_read_kb":13915,"num_write":1017,"num_write_kb":7086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":2252743,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"133'465","reported_seq":1559,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095625+0000","last_change":"2024-09-18T00:20:37.723738+0000","last_active":"2024-09-18T00:20:51.095625+0000","last_peered":"2024-09-18T00:20:51.095625+0000","last_clean":"2024-09-18T00:20:51.095625+0000","last_became_active":"2024-09-18T00:20:37.722312+0000","last_became_peered":"2024-09-18T00:20:37.722312+0000","last_unstale":"2024-09-18T00:20:51.095625+0000","last_undegraded":"2024-09-18T00:20:51.095625+0000","last_fullsized":"2024-09-18T00:20:51.095625+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:14:23.427889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057225740000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1405,"num_read_kb":1924,"num_write":538,"num_write_kb":2847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":46741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"135'781","reported_seq":1532,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097665+0000","last_change":"2024-09-18T00:20:48.470232+0000","last_active":"2024-09-18T00:20:51.097665+0000","last_peered":"2024-09-18T00:20:51.097665+0000","last_clean":"2024-09-18T00:20:51.097665+0000","last_became_active":"2024-09-18T00:20:48.465769+0000","last_became_peered":"2024-09-18T00:20:48.465769+0000","last_unstale":"2024-09-18T00:20:51.097665+0000","last_undegraded":"2024-09-18T00:20:51.097665+0000","last_fullsized":"2024-09-18T00:20:51.097665+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:11:07.738987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032750750000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3226,"num_read_kb":4864,"num_write":1184,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":510608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"135'453","reported_seq":1953,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097692+0000","last_change":"2024-09-18T00:20:48.467870+0000","last_active":"2024-09-18T00:20:51.097692+0000","last_peered":"2024-09-18T00:20:51.097692+0000","last_clean":"2024-09-18T00:20:51.097692+0000","last_became_active":"2024-09-18T00:20:48.463565+0000","last_became_peered":"2024-09-18T00:20:48.463565+0000","last_unstale":"2024-09-18T00:20:51.097692+0000","last_undegraded":"2024-09-18T00:20:51.097692+0000","last_fullsized":"2024-09-18T00:20:51.097692+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:07:09.913555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002116522,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3432,"num_read_kb":9919,"num_write":470,"num_write_kb":3833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":72221,"num_keys_recovered":54,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"133'289","reported_seq":1665,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451394+0000","last_change":"2024-09-18T00:20:17.572140+0000","last_active":"2024-09-18T00:20:48.451394+0000","last_peered":"2024-09-18T00:20:48.451394+0000","last_clean":"2024-09-18T00:20:48.451394+0000","last_became_active":"2024-09-18T00:18:05.415810+0000","last_became_peered":"2024-09-18T00:18:05.415810+0000","last_unstale":"2024-09-18T00:20:48.451394+0000","last_undegraded":"2024-09-18T00:20:48.451394+0000","last_fullsized":"2024-09-18T00:20:48.451394+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:35:11.075422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015232959999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1498,"num_read_kb":2662,"num_write":281,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"140'226","reported_seq":3328,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459066+0000","last_change":"2024-09-18T00:20:30.686023+0000","last_active":"2024-09-18T00:20:48.459066+0000","last_peered":"2024-09-18T00:20:48.459066+0000","last_clean":"2024-09-18T00:20:48.459066+0000","last_became_active":"2024-09-18T00:20:30.637311+0000","last_became_peered":"2024-09-18T00:20:30.637311+0000","last_unstale":"2024-09-18T00:20:48.459066+0000","last_undegraded":"2024-09-18T00:20:48.459066+0000","last_fullsized":"2024-09-18T00:20:48.459066+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:56:01.105381+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012894545,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":2320,"num_read_kb":4551,"num_write":211,"num_write_kb":2083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":645956,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"140'384","reported_seq":1512,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462374+0000","last_change":"2024-09-18T00:20:30.693807+0000","last_active":"2024-09-18T00:20:49.462374+0000","last_peered":"2024-09-18T00:20:49.462374+0000","last_clean":"2024-09-18T00:20:49.462374+0000","last_became_active":"2024-09-18T00:20:30.640499+0000","last_became_peered":"2024-09-18T00:20:30.640499+0000","last_unstale":"2024-09-18T00:20:49.462374+0000","last_undegraded":"2024-09-18T00:20:49.462374+0000","last_fullsized":"2024-09-18T00:20:49.462374+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:26.838200+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018217595999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1131,"num_read_kb":4419,"num_write":389,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":1283165,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"133'327","reported_seq":3718,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095462+0000","last_change":"2024-09-18T00:20:17.694311+0000","last_active":"2024-09-18T00:20:51.095462+0000","last_peered":"2024-09-18T00:20:51.095462+0000","last_clean":"2024-09-18T00:20:51.095462+0000","last_became_active":"2024-09-18T00:18:05.406821+0000","last_became_peered":"2024-09-18T00:18:05.406821+0000","last_unstale":"2024-09-18T00:20:51.095462+0000","last_undegraded":"2024-09-18T00:20:51.095462+0000","last_fullsized":"2024-09-18T00:20:51.095462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:13:08.591812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022731449999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3612,"num_read_kb":21086,"num_write":298,"num_write_kb":8166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'372","reported_seq":1395,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451657+0000","last_change":"2024-09-18T00:20:17.573988+0000","last_active":"2024-09-18T00:20:48.451657+0000","last_peered":"2024-09-18T00:20:48.451657+0000","last_clean":"2024-09-18T00:20:48.451657+0000","last_became_active":"2024-09-18T00:19:18.328043+0000","last_became_peered":"2024-09-18T00:19:18.328043+0000","last_unstale":"2024-09-18T00:20:48.451657+0000","last_undegraded":"2024-09-18T00:20:48.451657+0000","last_fullsized":"2024-09-18T00:20:48.451657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:11:44.153392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025255270000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1318,"num_read_kb":6224,"num_write":322,"num_write_kb":4815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":491875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"135'690","reported_seq":2488,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097894+0000","last_change":"2024-09-18T00:20:48.471288+0000","last_active":"2024-09-18T00:20:51.097894+0000","last_peered":"2024-09-18T00:20:51.097894+0000","last_clean":"2024-09-18T00:20:51.097894+0000","last_became_active":"2024-09-18T00:20:48.467364+0000","last_became_peered":"2024-09-18T00:20:48.467364+0000","last_unstale":"2024-09-18T00:20:51.097894+0000","last_undegraded":"2024-09-18T00:20:51.097894+0000","last_fullsized":"2024-09-18T00:20:51.097894+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:25.555616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021376099999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4502,"num_read_kb":5989,"num_write":925,"num_write_kb":5689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1385456,"num_keys_recovered":29,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"133'489","reported_seq":1056,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463552+0000","last_change":"2024-09-18T00:20:17.577703+0000","last_active":"2024-09-18T00:20:49.463552+0000","last_peered":"2024-09-18T00:20:49.463552+0000","last_clean":"2024-09-18T00:20:49.463552+0000","last_became_active":"2024-09-18T00:18:16.438164+0000","last_became_peered":"2024-09-18T00:18:16.438164+0000","last_unstale":"2024-09-18T00:20:49.463552+0000","last_undegraded":"2024-09-18T00:20:49.463552+0000","last_fullsized":"2024-09-18T00:20:49.463552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:18:47.201515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014879730000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":1078,"num_read_kb":1346,"num_write":552,"num_write_kb":3993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"134'530","reported_seq":1981,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459503+0000","last_change":"2024-09-18T00:20:17.576876+0000","last_active":"2024-09-18T00:20:48.459503+0000","last_peered":"2024-09-18T00:20:48.459503+0000","last_clean":"2024-09-18T00:20:48.459503+0000","last_became_active":"2024-09-18T00:18:16.441758+0000","last_became_peered":"2024-09-18T00:18:16.441758+0000","last_unstale":"2024-09-18T00:20:48.459503+0000","last_undegraded":"2024-09-18T00:20:48.459503+0000","last_fullsized":"2024-09-18T00:20:48.459503+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:18:47.366485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050089940000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2844,"num_read_kb":5653,"num_write":555,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":103027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":38,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451888+0000","last_change":"2024-09-18T00:20:34.754460+0000","last_active":"2024-09-18T00:20:48.451888+0000","last_peered":"2024-09-18T00:20:48.451888+0000","last_clean":"2024-09-18T00:20:48.451888+0000","last_became_active":"2024-09-18T00:20:34.753675+0000","last_became_peered":"2024-09-18T00:20:34.753675+0000","last_unstale":"2024-09-18T00:20:48.451888+0000","last_undegraded":"2024-09-18T00:20:48.451888+0000","last_fullsized":"2024-09-18T00:20:48.451888+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:37.203669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089554000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"135'348","reported_seq":2423,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.452208+0000","last_change":"2024-09-18T00:20:30.727046+0000","last_active":"2024-09-18T00:20:48.452208+0000","last_peered":"2024-09-18T00:20:48.452208+0000","last_clean":"2024-09-18T00:20:48.452208+0000","last_became_active":"2024-09-18T00:20:30.648069+0000","last_became_peered":"2024-09-18T00:20:30.648069+0000","last_unstale":"2024-09-18T00:20:48.452208+0000","last_undegraded":"2024-09-18T00:20:48.452208+0000","last_fullsized":"2024-09-18T00:20:48.452208+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:36.621517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052221869999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2016,"num_read_kb":5374,"num_write":329,"num_write_kb":5799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":1635567,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":59,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462482+0000","last_change":"2024-09-18T00:20:34.694081+0000","last_active":"2024-09-18T00:20:49.462482+0000","last_peered":"2024-09-18T00:20:49.462482+0000","last_clean":"2024-09-18T00:20:49.462482+0000","last_became_active":"2024-09-18T00:20:34.693550+0000","last_became_peered":"2024-09-18T00:20:34.693550+0000","last_unstale":"2024-09-18T00:20:49.462482+0000","last_undegraded":"2024-09-18T00:20:49.462482+0000","last_fullsized":"2024-09-18T00:20:49.462482+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:34:59.142586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'665","reported_seq":3138,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095867+0000","last_change":"2024-09-18T00:20:17.695571+0000","last_active":"2024-09-18T00:20:51.095867+0000","last_peered":"2024-09-18T00:20:51.095867+0000","last_clean":"2024-09-18T00:20:51.095867+0000","last_became_active":"2024-09-18T00:18:16.440536+0000","last_became_peered":"2024-09-18T00:18:16.440536+0000","last_unstale":"2024-09-18T00:20:51.095867+0000","last_undegraded":"2024-09-18T00:20:51.095867+0000","last_fullsized":"2024-09-18T00:20:51.095867+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:35.729973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034870019999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":3858,"num_read_kb":13564,"num_write":915,"num_write_kb":9611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":73512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":257,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463801+0000","last_change":"2024-09-18T00:18:16.439964+0000","last_active":"2024-09-18T00:20:49.463801+0000","last_peered":"2024-09-18T00:20:49.463801+0000","last_clean":"2024-09-18T00:20:49.463801+0000","last_became_active":"2024-09-18T00:18:16.437150+0000","last_became_peered":"2024-09-18T00:18:16.437150+0000","last_unstale":"2024-09-18T00:20:49.463801+0000","last_undegraded":"2024-09-18T00:20:49.463801+0000","last_fullsized":"2024-09-18T00:20:49.463801+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:25.772294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"140'491","reported_seq":2308,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462407+0000","last_change":"2024-09-18T00:20:30.710100+0000","last_active":"2024-09-18T00:20:49.462407+0000","last_peered":"2024-09-18T00:20:49.462407+0000","last_clean":"2024-09-18T00:20:49.462407+0000","last_became_active":"2024-09-18T00:20:30.641845+0000","last_became_peered":"2024-09-18T00:20:30.641845+0000","last_unstale":"2024-09-18T00:20:49.462407+0000","last_undegraded":"2024-09-18T00:20:49.462407+0000","last_fullsized":"2024-09-18T00:20:49.462407+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:00:46.444023+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.013533892,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2426,"num_read_kb":4923,"num_write":423,"num_write_kb":4982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":1553378,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462313+0000","last_change":"2024-09-18T00:20:30.651121+0000","last_active":"2024-09-18T00:20:49.462313+0000","last_peered":"2024-09-18T00:20:49.462313+0000","last_clean":"2024-09-18T00:20:49.462313+0000","last_became_active":"2024-09-18T00:20:30.650373+0000","last_became_peered":"2024-09-18T00:20:30.650373+0000","last_unstale":"2024-09-18T00:20:49.462313+0000","last_undegraded":"2024-09-18T00:20:49.462313+0000","last_fullsized":"2024-09-18T00:20:49.462313+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:36:36.287145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"133'363","reported_seq":2714,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097799+0000","last_change":"2024-09-18T00:20:48.470308+0000","last_active":"2024-09-18T00:20:51.097799+0000","last_peered":"2024-09-18T00:20:51.097799+0000","last_clean":"2024-09-18T00:20:51.097799+0000","last_became_active":"2024-09-18T00:20:48.462691+0000","last_became_peered":"2024-09-18T00:20:48.462691+0000","last_unstale":"2024-09-18T00:20:51.097799+0000","last_undegraded":"2024-09-18T00:20:51.097799+0000","last_fullsized":"2024-09-18T00:20:51.097799+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:08.083732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001889571,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2213,"num_read_kb":12284,"num_write":366,"num_write_kb":5310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":675014,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"135'484","reported_seq":2135,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459435+0000","last_change":"2024-09-18T00:20:17.573853+0000","last_active":"2024-09-18T00:20:48.459435+0000","last_peered":"2024-09-18T00:20:48.459435+0000","last_clean":"2024-09-18T00:20:48.459435+0000","last_became_active":"2024-09-18T00:18:05.407889+0000","last_became_peered":"2024-09-18T00:18:05.407889+0000","last_unstale":"2024-09-18T00:20:48.459435+0000","last_undegraded":"2024-09-18T00:20:48.459435+0000","last_fullsized":"2024-09-18T00:20:48.459435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:40.520294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002502663,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":1766,"num_read_kb":5407,"num_write":446,"num_write_kb":5379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":203,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.098054+0000","last_change":"2024-09-18T00:20:48.461100+0000","last_active":"2024-09-18T00:20:51.098054+0000","last_peered":"2024-09-18T00:20:51.098054+0000","last_clean":"2024-09-18T00:20:51.098054+0000","last_became_active":"2024-09-18T00:20:48.460277+0000","last_became_peered":"2024-09-18T00:20:48.460277+0000","last_unstale":"2024-09-18T00:20:51.098054+0000","last_undegraded":"2024-09-18T00:20:51.098054+0000","last_fullsized":"2024-09-18T00:20:51.098054+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:01:02.391844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088189300000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"134'369","reported_seq":1075,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462205+0000","last_change":"2024-09-18T00:20:30.726339+0000","last_active":"2024-09-18T00:20:49.462205+0000","last_peered":"2024-09-18T00:20:49.462205+0000","last_clean":"2024-09-18T00:20:49.462205+0000","last_became_active":"2024-09-18T00:20:30.647777+0000","last_became_peered":"2024-09-18T00:20:30.647777+0000","last_unstale":"2024-09-18T00:20:49.462205+0000","last_undegraded":"2024-09-18T00:20:49.462205+0000","last_fullsized":"2024-09-18T00:20:49.462205+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:21:04.035067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011927055000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":740,"num_read_kb":1879,"num_write":446,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":1002237,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":289,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.461675+0000","last_change":"2024-09-18T00:17:57.477633+0000","last_active":"2024-09-18T00:20:49.461675+0000","last_peered":"2024-09-18T00:20:49.461675+0000","last_clean":"2024-09-18T00:20:49.461675+0000","last_became_active":"2024-09-18T00:17:55.631277+0000","last_became_peered":"2024-09-18T00:17:55.631277+0000","last_unstale":"2024-09-18T00:20:49.461675+0000","last_undegraded":"2024-09-18T00:20:49.461675+0000","last_fullsized":"2024-09-18T00:20:49.461675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:55.421684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001141204,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"135'502","reported_seq":2618,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451329+0000","last_change":"2024-09-18T00:20:17.571764+0000","last_active":"2024-09-18T00:20:48.451329+0000","last_peered":"2024-09-18T00:20:48.451329+0000","last_clean":"2024-09-18T00:20:48.451329+0000","last_became_active":"2024-09-18T00:18:05.410952+0000","last_became_peered":"2024-09-18T00:18:05.410952+0000","last_unstale":"2024-09-18T00:20:48.451329+0000","last_undegraded":"2024-09-18T00:20:48.451329+0000","last_fullsized":"2024-09-18T00:20:48.451329+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:15.435600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014272110000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3308,"num_read_kb":10125,"num_write":594,"num_write_kb":8641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":276,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463375+0000","last_change":"2024-09-18T00:20:30.657193+0000","last_active":"2024-09-18T00:20:49.463375+0000","last_peered":"2024-09-18T00:20:49.463375+0000","last_clean":"2024-09-18T00:20:49.463375+0000","last_became_active":"2024-09-18T00:20:30.656400+0000","last_became_peered":"2024-09-18T00:20:30.656400+0000","last_unstale":"2024-09-18T00:20:49.463375+0000","last_undegraded":"2024-09-18T00:20:49.463375+0000","last_fullsized":"2024-09-18T00:20:49.463375+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:50:27.874106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"134'493","reported_seq":2493,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095513+0000","last_change":"2024-09-18T00:20:17.697271+0000","last_active":"2024-09-18T00:20:51.095513+0000","last_peered":"2024-09-18T00:20:51.095513+0000","last_clean":"2024-09-18T00:20:51.095513+0000","last_became_active":"2024-09-18T00:19:18.330155+0000","last_became_peered":"2024-09-18T00:19:18.330155+0000","last_unstale":"2024-09-18T00:20:51.095513+0000","last_undegraded":"2024-09-18T00:20:51.095513+0000","last_fullsized":"2024-09-18T00:20:51.095513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:01:28.392225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005129074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2497,"num_read_kb":7710,"num_write":474,"num_write_kb":3192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":493176,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":122,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097622+0000","last_change":"2024-09-18T00:20:48.459774+0000","last_active":"2024-09-18T00:20:51.097622+0000","last_peered":"2024-09-18T00:20:51.097622+0000","last_clean":"2024-09-18T00:20:51.097622+0000","last_became_active":"2024-09-18T00:20:48.457808+0000","last_became_peered":"2024-09-18T00:20:48.457808+0000","last_unstale":"2024-09-18T00:20:51.097622+0000","last_undegraded":"2024-09-18T00:20:51.097622+0000","last_fullsized":"2024-09-18T00:20:51.097622+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:43:17.325507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":438,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463383+0000","last_change":"2024-09-18T00:18:19.466612+0000","last_active":"2024-09-18T00:20:49.463383+0000","last_peered":"2024-09-18T00:20:49.463383+0000","last_clean":"2024-09-18T00:20:49.463383+0000","last_became_active":"2024-09-18T00:18:19.465557+0000","last_became_peered":"2024-09-18T00:18:19.465557+0000","last_unstale":"2024-09-18T00:20:49.463383+0000","last_undegraded":"2024-09-18T00:20:49.463383+0000","last_fullsized":"2024-09-18T00:20:49.463383+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_clean_scrub_stamp":"2024-09-18T00:17:50.414591+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:08:40.389972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"140'661","reported_seq":2377,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095512+0000","last_change":"2024-09-18T00:20:30.742660+0000","last_active":"2024-09-18T00:20:51.095512+0000","last_peered":"2024-09-18T00:20:51.095512+0000","last_clean":"2024-09-18T00:20:51.095512+0000","last_became_active":"2024-09-18T00:20:30.656699+0000","last_became_peered":"2024-09-18T00:20:30.656699+0000","last_unstale":"2024-09-18T00:20:51.095512+0000","last_undegraded":"2024-09-18T00:20:51.095512+0000","last_fullsized":"2024-09-18T00:20:51.095512+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:37:48.645340+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0070126900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3348,"num_read_kb":6619,"num_write":872,"num_write_kb":5519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":1197799,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"134'383","reported_seq":2112,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459075+0000","last_change":"2024-09-18T00:20:17.574821+0000","last_active":"2024-09-18T00:20:48.459075+0000","last_peered":"2024-09-18T00:20:48.459075+0000","last_clean":"2024-09-18T00:20:48.459075+0000","last_became_active":"2024-09-18T00:18:05.401097+0000","last_became_peered":"2024-09-18T00:18:05.401097+0000","last_unstale":"2024-09-18T00:20:48.459075+0000","last_undegraded":"2024-09-18T00:20:48.459075+0000","last_fullsized":"2024-09-18T00:20:48.459075+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:50:58.016930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003875547,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3069,"num_read_kb":9114,"num_write":378,"num_write_kb":6107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"140'412","reported_seq":2388,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451796+0000","last_change":"2024-09-18T00:20:37.726930+0000","last_active":"2024-09-18T00:20:48.451796+0000","last_peered":"2024-09-18T00:20:48.451796+0000","last_clean":"2024-09-18T00:20:48.451796+0000","last_became_active":"2024-09-18T00:20:37.721802+0000","last_became_peered":"2024-09-18T00:20:37.721802+0000","last_unstale":"2024-09-18T00:20:48.451796+0000","last_undegraded":"2024-09-18T00:20:48.451796+0000","last_fullsized":"2024-09-18T00:20:48.451796+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:37.117595+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.007118643,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":1971,"num_read_kb":7346,"num_write":375,"num_write_kb":8437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":8153977,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"134'425","reported_seq":1164,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095561+0000","last_change":"2024-09-18T00:20:17.696586+0000","last_active":"2024-09-18T00:20:51.095561+0000","last_peered":"2024-09-18T00:20:51.095561+0000","last_clean":"2024-09-18T00:20:51.095561+0000","last_became_active":"2024-09-18T00:18:05.406977+0000","last_became_peered":"2024-09-18T00:18:05.406977+0000","last_unstale":"2024-09-18T00:20:51.095561+0000","last_undegraded":"2024-09-18T00:20:51.095561+0000","last_fullsized":"2024-09-18T00:20:51.095561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:06:59.846008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044405699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1237,"num_read_kb":3121,"num_write":630,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"135'1438","reported_seq":3733,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097706+0000","last_change":"2024-09-18T00:20:48.467838+0000","last_active":"2024-09-18T00:20:51.097706+0000","last_peered":"2024-09-18T00:20:51.097706+0000","last_clean":"2024-09-18T00:20:51.097706+0000","last_became_active":"2024-09-18T00:20:48.462359+0000","last_became_peered":"2024-09-18T00:20:48.462359+0000","last_unstale":"2024-09-18T00:20:51.097706+0000","last_undegraded":"2024-09-18T00:20:51.097706+0000","last_fullsized":"2024-09-18T00:20:51.097706+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":1438,"log_dups_size":0,"ondisk_log_size":1438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:28:28.471113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032308089999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9631,"num_read_kb":9532,"num_write":2186,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":392734,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"134'780","reported_seq":3314,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463834+0000","last_change":"2024-09-18T00:20:17.579849+0000","last_active":"2024-09-18T00:20:49.463834+0000","last_peered":"2024-09-18T00:20:49.463834+0000","last_clean":"2024-09-18T00:20:49.463834+0000","last_became_active":"2024-09-18T00:19:18.333730+0000","last_became_peered":"2024-09-18T00:19:18.333730+0000","last_unstale":"2024-09-18T00:20:49.463834+0000","last_undegraded":"2024-09-18T00:20:49.463834+0000","last_fullsized":"2024-09-18T00:20:49.463834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:12:14.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029127850000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5845,"num_read_kb":7620,"num_write":1029,"num_write_kb":6233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":697208,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"134'352","reported_seq":1424,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463902+0000","last_change":"2024-09-18T00:20:17.578670+0000","last_active":"2024-09-18T00:20:49.463902+0000","last_peered":"2024-09-18T00:20:49.463902+0000","last_clean":"2024-09-18T00:20:49.463902+0000","last_became_active":"2024-09-18T00:18:16.440223+0000","last_became_peered":"2024-09-18T00:18:16.440223+0000","last_unstale":"2024-09-18T00:20:49.463902+0000","last_undegraded":"2024-09-18T00:20:49.463902+0000","last_fullsized":"2024-09-18T00:20:49.463902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:17.972019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002121001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1129,"num_read_kb":6709,"num_write":391,"num_write_kb":4357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":7258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"133'397","reported_seq":3507,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.464104+0000","last_change":"2024-09-18T00:20:17.579351+0000","last_active":"2024-09-18T00:20:49.464104+0000","last_peered":"2024-09-18T00:20:49.464104+0000","last_clean":"2024-09-18T00:20:49.464104+0000","last_became_active":"2024-09-18T00:18:16.441020+0000","last_became_peered":"2024-09-18T00:18:16.441020+0000","last_unstale":"2024-09-18T00:20:49.464104+0000","last_undegraded":"2024-09-18T00:20:49.464104+0000","last_fullsized":"2024-09-18T00:20:49.464104+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:01:45.246626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00266241,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4197,"num_read_kb":11008,"num_write":397,"num_write_kb":9044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"134'378","reported_seq":4809,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451504+0000","last_change":"2024-09-18T00:20:37.726301+0000","last_active":"2024-09-18T00:20:48.451504+0000","last_peered":"2024-09-18T00:20:48.451504+0000","last_clean":"2024-09-18T00:20:48.451504+0000","last_became_active":"2024-09-18T00:20:37.723504+0000","last_became_peered":"2024-09-18T00:20:37.723504+0000","last_unstale":"2024-09-18T00:20:48.451504+0000","last_undegraded":"2024-09-18T00:20:48.451504+0000","last_fullsized":"2024-09-18T00:20:48.451504+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:16:42.705342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001519245,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5107,"num_read_kb":32748,"num_write":405,"num_write_kb":13893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":3037482,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"133'368","reported_seq":4657,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463399+0000","last_change":"2024-09-18T00:20:17.577141+0000","last_active":"2024-09-18T00:20:49.463399+0000","last_peered":"2024-09-18T00:20:49.463399+0000","last_clean":"2024-09-18T00:20:49.463399+0000","last_became_active":"2024-09-18T00:18:16.438286+0000","last_became_peered":"2024-09-18T00:18:16.438286+0000","last_unstale":"2024-09-18T00:20:49.463399+0000","last_undegraded":"2024-09-18T00:20:49.463399+0000","last_fullsized":"2024-09-18T00:20:49.463399+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:09:46.156797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092097700000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4916,"num_read_kb":14028,"num_write":359,"num_write_kb":4057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":70145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"134'355","reported_seq":1976,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451372+0000","last_change":"2024-09-18T00:20:17.572303+0000","last_active":"2024-09-18T00:20:48.451372+0000","last_peered":"2024-09-18T00:20:48.451372+0000","last_clean":"2024-09-18T00:20:48.451372+0000","last_became_active":"2024-09-18T00:18:16.438556+0000","last_became_peered":"2024-09-18T00:18:16.438556+0000","last_unstale":"2024-09-18T00:20:48.451372+0000","last_undegraded":"2024-09-18T00:20:48.451372+0000","last_fullsized":"2024-09-18T00:20:48.451372+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:36.881982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019750979999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1630,"num_read_kb":9063,"num_write":386,"num_write_kb":8736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":47120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"133'748","reported_seq":3136,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463471+0000","last_change":"2024-09-18T00:20:17.579483+0000","last_active":"2024-09-18T00:20:49.463471+0000","last_peered":"2024-09-18T00:20:49.463471+0000","last_clean":"2024-09-18T00:20:49.463471+0000","last_became_active":"2024-09-18T00:18:16.440910+0000","last_became_peered":"2024-09-18T00:18:16.440910+0000","last_unstale":"2024-09-18T00:20:49.463471+0000","last_undegraded":"2024-09-18T00:20:49.463471+0000","last_fullsized":"2024-09-18T00:20:49.463471+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:00:51.845256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028046910000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4645,"num_read_kb":12707,"num_write":1147,"num_write_kb":3134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":65,"num_read":96502,"num_read_kb":273433,"num_write":19290,"num_write_kb":181728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1834,"num_bytes_recovered":25828399,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19623,"internal_metadata":0},"log_size":16196,"ondisk_log_size":16196,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20247,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55260,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316580,"statfs":{"total":96636764160,"available":96580177920,"internally_reserved":0,"allocated":1908736,"data_stored":1478380,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37172,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334668,"statfs":{"total":96636764160,"available":96598700032,"internally_reserved":0,"allocated":1867776,"data_stored":888012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":76,"seq":326417514518,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38212,"kb_used_data":1840,"kb_used_omap":9,"kb_used_meta":36342,"kb_avail":94333628,"statfs":{"total":96636764160,"available":96597635072,"internally_reserved":0,"allocated":1884160,"data_stored":893873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9346,"internal_metadata":37215102},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55228,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1871872,"data_stored":888031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41344,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330496,"statfs":{"total":96636764160,"available":96594427904,"internally_reserved":0,"allocated":1880064,"data_stored":893854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":139,"seq":597000454151,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":1844,"kb_used_omap":17,"kb_used_meta":30190,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":1888256,"data_stored":893892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17900,"internal_metadata":30915092},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542185,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73280,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298560,"statfs":{"total":96636764160,"available":96561725440,"internally_reserved":0,"allocated":1880064,"data_stored":893854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1925120,"data_stored":1478380,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13445,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6178,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:20:55.196 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:20:55.474 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:20:55.474 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-18T00:20:55.490 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-18T00:20:54.348193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":65,"num_read":96618,"num_read_kb":273533,"num_write":19451,"num_write_kb":184146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1839,"num_bytes_recovered":27009154,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16316,"ondisk_log_size":16316,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":365648,"kb_used_data":14752,"kb_used_omap":35,"kb_used_meta":350684,"kb_avail":754609072,"statfs":{"total":773094113280,"available":772719689728,"internally_reserved":0,"allocated":15106048,"data_stored":8308276,"data_compressed":13209,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":36807,"internal_metadata":359100473},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.911748"},"pg_stats":[{"pgid":"3.1f","version":"135'778","reported_seq":3008,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451528+0000","last_change":"2024-09-18T00:20:17.573423+0000","last_active":"2024-09-18T00:20:48.451528+0000","last_peered":"2024-09-18T00:20:48.451528+0000","last_clean":"2024-09-18T00:20:48.451528+0000","last_became_active":"2024-09-18T00:19:18.327350+0000","last_became_peered":"2024-09-18T00:19:18.327350+0000","last_unstale":"2024-09-18T00:20:48.451528+0000","last_undegraded":"2024-09-18T00:20:48.451528+0000","last_fullsized":"2024-09-18T00:20:48.451528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":778,"log_dups_size":0,"ondisk_log_size":778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:46:14.185856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024250489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4587,"num_read_kb":13915,"num_write":1017,"num_write_kb":7086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":2252743,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"133'465","reported_seq":1559,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095625+0000","last_change":"2024-09-18T00:20:37.723738+0000","last_active":"2024-09-18T00:20:51.095625+0000","last_peered":"2024-09-18T00:20:51.095625+0000","last_clean":"2024-09-18T00:20:51.095625+0000","last_became_active":"2024-09-18T00:20:37.722312+0000","last_became_peered":"2024-09-18T00:20:37.722312+0000","last_unstale":"2024-09-18T00:20:51.095625+0000","last_undegraded":"2024-09-18T00:20:51.095625+0000","last_fullsized":"2024-09-18T00:20:51.095625+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:14:23.427889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057225740000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1405,"num_read_kb":1924,"num_write":538,"num_write_kb":2847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":46741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"135'781","reported_seq":1532,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097665+0000","last_change":"2024-09-18T00:20:48.470232+0000","last_active":"2024-09-18T00:20:51.097665+0000","last_peered":"2024-09-18T00:20:51.097665+0000","last_clean":"2024-09-18T00:20:51.097665+0000","last_became_active":"2024-09-18T00:20:48.465769+0000","last_became_peered":"2024-09-18T00:20:48.465769+0000","last_unstale":"2024-09-18T00:20:51.097665+0000","last_undegraded":"2024-09-18T00:20:51.097665+0000","last_fullsized":"2024-09-18T00:20:51.097665+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:11:07.738987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032750750000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3226,"num_read_kb":4864,"num_write":1184,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":510608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"135'453","reported_seq":1953,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097692+0000","last_change":"2024-09-18T00:20:48.467870+0000","last_active":"2024-09-18T00:20:51.097692+0000","last_peered":"2024-09-18T00:20:51.097692+0000","last_clean":"2024-09-18T00:20:51.097692+0000","last_became_active":"2024-09-18T00:20:48.463565+0000","last_became_peered":"2024-09-18T00:20:48.463565+0000","last_unstale":"2024-09-18T00:20:51.097692+0000","last_undegraded":"2024-09-18T00:20:51.097692+0000","last_fullsized":"2024-09-18T00:20:51.097692+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:07:09.913555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002116522,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3432,"num_read_kb":9919,"num_write":470,"num_write_kb":3833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":72221,"num_keys_recovered":54,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"133'289","reported_seq":1665,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451394+0000","last_change":"2024-09-18T00:20:17.572140+0000","last_active":"2024-09-18T00:20:48.451394+0000","last_peered":"2024-09-18T00:20:48.451394+0000","last_clean":"2024-09-18T00:20:48.451394+0000","last_became_active":"2024-09-18T00:18:05.415810+0000","last_became_peered":"2024-09-18T00:18:05.415810+0000","last_unstale":"2024-09-18T00:20:48.451394+0000","last_undegraded":"2024-09-18T00:20:48.451394+0000","last_fullsized":"2024-09-18T00:20:48.451394+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:35:11.075422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015232959999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1498,"num_read_kb":2662,"num_write":281,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"140'226","reported_seq":3328,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459066+0000","last_change":"2024-09-18T00:20:30.686023+0000","last_active":"2024-09-18T00:20:48.459066+0000","last_peered":"2024-09-18T00:20:48.459066+0000","last_clean":"2024-09-18T00:20:48.459066+0000","last_became_active":"2024-09-18T00:20:30.637311+0000","last_became_peered":"2024-09-18T00:20:30.637311+0000","last_unstale":"2024-09-18T00:20:48.459066+0000","last_undegraded":"2024-09-18T00:20:48.459066+0000","last_fullsized":"2024-09-18T00:20:48.459066+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:56:01.105381+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012894545,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":2320,"num_read_kb":4551,"num_write":211,"num_write_kb":2083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":645956,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"140'384","reported_seq":1512,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462374+0000","last_change":"2024-09-18T00:20:30.693807+0000","last_active":"2024-09-18T00:20:49.462374+0000","last_peered":"2024-09-18T00:20:49.462374+0000","last_clean":"2024-09-18T00:20:49.462374+0000","last_became_active":"2024-09-18T00:20:30.640499+0000","last_became_peered":"2024-09-18T00:20:30.640499+0000","last_unstale":"2024-09-18T00:20:49.462374+0000","last_undegraded":"2024-09-18T00:20:49.462374+0000","last_fullsized":"2024-09-18T00:20:49.462374+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:26.838200+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018217595999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1131,"num_read_kb":4419,"num_write":389,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":1283165,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"133'327","reported_seq":3718,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095462+0000","last_change":"2024-09-18T00:20:17.694311+0000","last_active":"2024-09-18T00:20:51.095462+0000","last_peered":"2024-09-18T00:20:51.095462+0000","last_clean":"2024-09-18T00:20:51.095462+0000","last_became_active":"2024-09-18T00:18:05.406821+0000","last_became_peered":"2024-09-18T00:18:05.406821+0000","last_unstale":"2024-09-18T00:20:51.095462+0000","last_undegraded":"2024-09-18T00:20:51.095462+0000","last_fullsized":"2024-09-18T00:20:51.095462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:13:08.591812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022731449999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3612,"num_read_kb":21086,"num_write":298,"num_write_kb":8166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'372","reported_seq":1395,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451657+0000","last_change":"2024-09-18T00:20:17.573988+0000","last_active":"2024-09-18T00:20:48.451657+0000","last_peered":"2024-09-18T00:20:48.451657+0000","last_clean":"2024-09-18T00:20:48.451657+0000","last_became_active":"2024-09-18T00:19:18.328043+0000","last_became_peered":"2024-09-18T00:19:18.328043+0000","last_unstale":"2024-09-18T00:20:48.451657+0000","last_undegraded":"2024-09-18T00:20:48.451657+0000","last_fullsized":"2024-09-18T00:20:48.451657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:11:44.153392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025255270000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1318,"num_read_kb":6224,"num_write":322,"num_write_kb":4815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":491875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"135'690","reported_seq":2488,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097894+0000","last_change":"2024-09-18T00:20:48.471288+0000","last_active":"2024-09-18T00:20:51.097894+0000","last_peered":"2024-09-18T00:20:51.097894+0000","last_clean":"2024-09-18T00:20:51.097894+0000","last_became_active":"2024-09-18T00:20:48.467364+0000","last_became_peered":"2024-09-18T00:20:48.467364+0000","last_unstale":"2024-09-18T00:20:51.097894+0000","last_undegraded":"2024-09-18T00:20:51.097894+0000","last_fullsized":"2024-09-18T00:20:51.097894+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:25.555616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021376099999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4502,"num_read_kb":5989,"num_write":925,"num_write_kb":5689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1385456,"num_keys_recovered":29,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"133'489","reported_seq":1056,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463552+0000","last_change":"2024-09-18T00:20:17.577703+0000","last_active":"2024-09-18T00:20:49.463552+0000","last_peered":"2024-09-18T00:20:49.463552+0000","last_clean":"2024-09-18T00:20:49.463552+0000","last_became_active":"2024-09-18T00:18:16.438164+0000","last_became_peered":"2024-09-18T00:18:16.438164+0000","last_unstale":"2024-09-18T00:20:49.463552+0000","last_undegraded":"2024-09-18T00:20:49.463552+0000","last_fullsized":"2024-09-18T00:20:49.463552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:18:47.201515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014879730000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":1078,"num_read_kb":1346,"num_write":552,"num_write_kb":3993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"134'530","reported_seq":1981,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459503+0000","last_change":"2024-09-18T00:20:17.576876+0000","last_active":"2024-09-18T00:20:48.459503+0000","last_peered":"2024-09-18T00:20:48.459503+0000","last_clean":"2024-09-18T00:20:48.459503+0000","last_became_active":"2024-09-18T00:18:16.441758+0000","last_became_peered":"2024-09-18T00:18:16.441758+0000","last_unstale":"2024-09-18T00:20:48.459503+0000","last_undegraded":"2024-09-18T00:20:48.459503+0000","last_fullsized":"2024-09-18T00:20:48.459503+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:18:47.366485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050089940000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2844,"num_read_kb":5653,"num_write":555,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":103027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":38,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451888+0000","last_change":"2024-09-18T00:20:34.754460+0000","last_active":"2024-09-18T00:20:48.451888+0000","last_peered":"2024-09-18T00:20:48.451888+0000","last_clean":"2024-09-18T00:20:48.451888+0000","last_became_active":"2024-09-18T00:20:34.753675+0000","last_became_peered":"2024-09-18T00:20:34.753675+0000","last_unstale":"2024-09-18T00:20:48.451888+0000","last_undegraded":"2024-09-18T00:20:48.451888+0000","last_fullsized":"2024-09-18T00:20:48.451888+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:37.203669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089554000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"135'348","reported_seq":2423,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.452208+0000","last_change":"2024-09-18T00:20:30.727046+0000","last_active":"2024-09-18T00:20:48.452208+0000","last_peered":"2024-09-18T00:20:48.452208+0000","last_clean":"2024-09-18T00:20:48.452208+0000","last_became_active":"2024-09-18T00:20:30.648069+0000","last_became_peered":"2024-09-18T00:20:30.648069+0000","last_unstale":"2024-09-18T00:20:48.452208+0000","last_undegraded":"2024-09-18T00:20:48.452208+0000","last_fullsized":"2024-09-18T00:20:48.452208+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:36.621517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052221869999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2016,"num_read_kb":5374,"num_write":329,"num_write_kb":5799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":1635567,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":59,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462482+0000","last_change":"2024-09-18T00:20:34.694081+0000","last_active":"2024-09-18T00:20:49.462482+0000","last_peered":"2024-09-18T00:20:49.462482+0000","last_clean":"2024-09-18T00:20:49.462482+0000","last_became_active":"2024-09-18T00:20:34.693550+0000","last_became_peered":"2024-09-18T00:20:34.693550+0000","last_unstale":"2024-09-18T00:20:49.462482+0000","last_undegraded":"2024-09-18T00:20:49.462482+0000","last_fullsized":"2024-09-18T00:20:49.462482+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:34:59.142586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'665","reported_seq":3138,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095867+0000","last_change":"2024-09-18T00:20:17.695571+0000","last_active":"2024-09-18T00:20:51.095867+0000","last_peered":"2024-09-18T00:20:51.095867+0000","last_clean":"2024-09-18T00:20:51.095867+0000","last_became_active":"2024-09-18T00:18:16.440536+0000","last_became_peered":"2024-09-18T00:18:16.440536+0000","last_unstale":"2024-09-18T00:20:51.095867+0000","last_undegraded":"2024-09-18T00:20:51.095867+0000","last_fullsized":"2024-09-18T00:20:51.095867+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:35.729973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034870019999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":3858,"num_read_kb":13564,"num_write":915,"num_write_kb":9611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":73512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":257,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463801+0000","last_change":"2024-09-18T00:18:16.439964+0000","last_active":"2024-09-18T00:20:49.463801+0000","last_peered":"2024-09-18T00:20:49.463801+0000","last_clean":"2024-09-18T00:20:49.463801+0000","last_became_active":"2024-09-18T00:18:16.437150+0000","last_became_peered":"2024-09-18T00:18:16.437150+0000","last_unstale":"2024-09-18T00:20:49.463801+0000","last_undegraded":"2024-09-18T00:20:49.463801+0000","last_fullsized":"2024-09-18T00:20:49.463801+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:25.772294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"140'491","reported_seq":2308,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462407+0000","last_change":"2024-09-18T00:20:30.710100+0000","last_active":"2024-09-18T00:20:49.462407+0000","last_peered":"2024-09-18T00:20:49.462407+0000","last_clean":"2024-09-18T00:20:49.462407+0000","last_became_active":"2024-09-18T00:20:30.641845+0000","last_became_peered":"2024-09-18T00:20:30.641845+0000","last_unstale":"2024-09-18T00:20:49.462407+0000","last_undegraded":"2024-09-18T00:20:49.462407+0000","last_fullsized":"2024-09-18T00:20:49.462407+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:00:46.444023+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.013533892,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2426,"num_read_kb":4923,"num_write":423,"num_write_kb":4982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":1553378,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462313+0000","last_change":"2024-09-18T00:20:30.651121+0000","last_active":"2024-09-18T00:20:49.462313+0000","last_peered":"2024-09-18T00:20:49.462313+0000","last_clean":"2024-09-18T00:20:49.462313+0000","last_became_active":"2024-09-18T00:20:30.650373+0000","last_became_peered":"2024-09-18T00:20:30.650373+0000","last_unstale":"2024-09-18T00:20:49.462313+0000","last_undegraded":"2024-09-18T00:20:49.462313+0000","last_fullsized":"2024-09-18T00:20:49.462313+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:36:36.287145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"133'363","reported_seq":2714,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097799+0000","last_change":"2024-09-18T00:20:48.470308+0000","last_active":"2024-09-18T00:20:51.097799+0000","last_peered":"2024-09-18T00:20:51.097799+0000","last_clean":"2024-09-18T00:20:51.097799+0000","last_became_active":"2024-09-18T00:20:48.462691+0000","last_became_peered":"2024-09-18T00:20:48.462691+0000","last_unstale":"2024-09-18T00:20:51.097799+0000","last_undegraded":"2024-09-18T00:20:51.097799+0000","last_fullsized":"2024-09-18T00:20:51.097799+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:08.083732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001889571,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2213,"num_read_kb":12284,"num_write":366,"num_write_kb":5310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":675014,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"135'484","reported_seq":2135,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459435+0000","last_change":"2024-09-18T00:20:17.573853+0000","last_active":"2024-09-18T00:20:48.459435+0000","last_peered":"2024-09-18T00:20:48.459435+0000","last_clean":"2024-09-18T00:20:48.459435+0000","last_became_active":"2024-09-18T00:18:05.407889+0000","last_became_peered":"2024-09-18T00:18:05.407889+0000","last_unstale":"2024-09-18T00:20:48.459435+0000","last_undegraded":"2024-09-18T00:20:48.459435+0000","last_fullsized":"2024-09-18T00:20:48.459435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:40.520294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002502663,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":1766,"num_read_kb":5407,"num_write":446,"num_write_kb":5379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":203,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.098054+0000","last_change":"2024-09-18T00:20:48.461100+0000","last_active":"2024-09-18T00:20:51.098054+0000","last_peered":"2024-09-18T00:20:51.098054+0000","last_clean":"2024-09-18T00:20:51.098054+0000","last_became_active":"2024-09-18T00:20:48.460277+0000","last_became_peered":"2024-09-18T00:20:48.460277+0000","last_unstale":"2024-09-18T00:20:51.098054+0000","last_undegraded":"2024-09-18T00:20:51.098054+0000","last_fullsized":"2024-09-18T00:20:51.098054+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:01:02.391844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088189300000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"134'369","reported_seq":1075,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462205+0000","last_change":"2024-09-18T00:20:30.726339+0000","last_active":"2024-09-18T00:20:49.462205+0000","last_peered":"2024-09-18T00:20:49.462205+0000","last_clean":"2024-09-18T00:20:49.462205+0000","last_became_active":"2024-09-18T00:20:30.647777+0000","last_became_peered":"2024-09-18T00:20:30.647777+0000","last_unstale":"2024-09-18T00:20:49.462205+0000","last_undegraded":"2024-09-18T00:20:49.462205+0000","last_fullsized":"2024-09-18T00:20:49.462205+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:21:04.035067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011927055000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":740,"num_read_kb":1879,"num_write":446,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":1002237,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":289,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.461675+0000","last_change":"2024-09-18T00:17:57.477633+0000","last_active":"2024-09-18T00:20:49.461675+0000","last_peered":"2024-09-18T00:20:49.461675+0000","last_clean":"2024-09-18T00:20:49.461675+0000","last_became_active":"2024-09-18T00:17:55.631277+0000","last_became_peered":"2024-09-18T00:17:55.631277+0000","last_unstale":"2024-09-18T00:20:49.461675+0000","last_undegraded":"2024-09-18T00:20:49.461675+0000","last_fullsized":"2024-09-18T00:20:49.461675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:55.421684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001141204,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"135'502","reported_seq":2618,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451329+0000","last_change":"2024-09-18T00:20:17.571764+0000","last_active":"2024-09-18T00:20:48.451329+0000","last_peered":"2024-09-18T00:20:48.451329+0000","last_clean":"2024-09-18T00:20:48.451329+0000","last_became_active":"2024-09-18T00:18:05.410952+0000","last_became_peered":"2024-09-18T00:18:05.410952+0000","last_unstale":"2024-09-18T00:20:48.451329+0000","last_undegraded":"2024-09-18T00:20:48.451329+0000","last_fullsized":"2024-09-18T00:20:48.451329+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:15.435600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014272110000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3308,"num_read_kb":10125,"num_write":594,"num_write_kb":8641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":276,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463375+0000","last_change":"2024-09-18T00:20:30.657193+0000","last_active":"2024-09-18T00:20:49.463375+0000","last_peered":"2024-09-18T00:20:49.463375+0000","last_clean":"2024-09-18T00:20:49.463375+0000","last_became_active":"2024-09-18T00:20:30.656400+0000","last_became_peered":"2024-09-18T00:20:30.656400+0000","last_unstale":"2024-09-18T00:20:49.463375+0000","last_undegraded":"2024-09-18T00:20:49.463375+0000","last_fullsized":"2024-09-18T00:20:49.463375+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:50:27.874106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"134'493","reported_seq":2493,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095513+0000","last_change":"2024-09-18T00:20:17.697271+0000","last_active":"2024-09-18T00:20:51.095513+0000","last_peered":"2024-09-18T00:20:51.095513+0000","last_clean":"2024-09-18T00:20:51.095513+0000","last_became_active":"2024-09-18T00:19:18.330155+0000","last_became_peered":"2024-09-18T00:19:18.330155+0000","last_unstale":"2024-09-18T00:20:51.095513+0000","last_undegraded":"2024-09-18T00:20:51.095513+0000","last_fullsized":"2024-09-18T00:20:51.095513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:01:28.392225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005129074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2497,"num_read_kb":7710,"num_write":474,"num_write_kb":3192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":493176,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":122,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097622+0000","last_change":"2024-09-18T00:20:48.459774+0000","last_active":"2024-09-18T00:20:51.097622+0000","last_peered":"2024-09-18T00:20:51.097622+0000","last_clean":"2024-09-18T00:20:51.097622+0000","last_became_active":"2024-09-18T00:20:48.457808+0000","last_became_peered":"2024-09-18T00:20:48.457808+0000","last_unstale":"2024-09-18T00:20:51.097622+0000","last_undegraded":"2024-09-18T00:20:51.097622+0000","last_fullsized":"2024-09-18T00:20:51.097622+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:43:17.325507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":438,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463383+0000","last_change":"2024-09-18T00:18:19.466612+0000","last_active":"2024-09-18T00:20:49.463383+0000","last_peered":"2024-09-18T00:20:49.463383+0000","last_clean":"2024-09-18T00:20:49.463383+0000","last_became_active":"2024-09-18T00:18:19.465557+0000","last_became_peered":"2024-09-18T00:18:19.465557+0000","last_unstale":"2024-09-18T00:20:49.463383+0000","last_undegraded":"2024-09-18T00:20:49.463383+0000","last_fullsized":"2024-09-18T00:20:49.463383+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_clean_scrub_stamp":"2024-09-18T00:17:50.414591+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:08:40.389972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"140'661","reported_seq":2377,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095512+0000","last_change":"2024-09-18T00:20:30.742660+0000","last_active":"2024-09-18T00:20:51.095512+0000","last_peered":"2024-09-18T00:20:51.095512+0000","last_clean":"2024-09-18T00:20:51.095512+0000","last_became_active":"2024-09-18T00:20:30.656699+0000","last_became_peered":"2024-09-18T00:20:30.656699+0000","last_unstale":"2024-09-18T00:20:51.095512+0000","last_undegraded":"2024-09-18T00:20:51.095512+0000","last_fullsized":"2024-09-18T00:20:51.095512+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:37:48.645340+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0070126900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3348,"num_read_kb":6619,"num_write":872,"num_write_kb":5519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":1197799,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"134'383","reported_seq":2112,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459075+0000","last_change":"2024-09-18T00:20:17.574821+0000","last_active":"2024-09-18T00:20:48.459075+0000","last_peered":"2024-09-18T00:20:48.459075+0000","last_clean":"2024-09-18T00:20:48.459075+0000","last_became_active":"2024-09-18T00:18:05.401097+0000","last_became_peered":"2024-09-18T00:18:05.401097+0000","last_unstale":"2024-09-18T00:20:48.459075+0000","last_undegraded":"2024-09-18T00:20:48.459075+0000","last_fullsized":"2024-09-18T00:20:48.459075+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:50:58.016930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003875547,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3069,"num_read_kb":9114,"num_write":378,"num_write_kb":6107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"140'412","reported_seq":2388,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451796+0000","last_change":"2024-09-18T00:20:37.726930+0000","last_active":"2024-09-18T00:20:48.451796+0000","last_peered":"2024-09-18T00:20:48.451796+0000","last_clean":"2024-09-18T00:20:48.451796+0000","last_became_active":"2024-09-18T00:20:37.721802+0000","last_became_peered":"2024-09-18T00:20:37.721802+0000","last_unstale":"2024-09-18T00:20:48.451796+0000","last_undegraded":"2024-09-18T00:20:48.451796+0000","last_fullsized":"2024-09-18T00:20:48.451796+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:37.117595+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.007118643,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":1971,"num_read_kb":7346,"num_write":375,"num_write_kb":8437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":8153977,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"134'425","reported_seq":1164,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095561+0000","last_change":"2024-09-18T00:20:17.696586+0000","last_active":"2024-09-18T00:20:51.095561+0000","last_peered":"2024-09-18T00:20:51.095561+0000","last_clean":"2024-09-18T00:20:51.095561+0000","last_became_active":"2024-09-18T00:18:05.406977+0000","last_became_peered":"2024-09-18T00:18:05.406977+0000","last_unstale":"2024-09-18T00:20:51.095561+0000","last_undegraded":"2024-09-18T00:20:51.095561+0000","last_fullsized":"2024-09-18T00:20:51.095561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:06:59.846008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044405699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1237,"num_read_kb":3121,"num_write":630,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"135'1438","reported_seq":3733,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097706+0000","last_change":"2024-09-18T00:20:48.467838+0000","last_active":"2024-09-18T00:20:51.097706+0000","last_peered":"2024-09-18T00:20:51.097706+0000","last_clean":"2024-09-18T00:20:51.097706+0000","last_became_active":"2024-09-18T00:20:48.462359+0000","last_became_peered":"2024-09-18T00:20:48.462359+0000","last_unstale":"2024-09-18T00:20:51.097706+0000","last_undegraded":"2024-09-18T00:20:51.097706+0000","last_fullsized":"2024-09-18T00:20:51.097706+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":1438,"log_dups_size":0,"ondisk_log_size":1438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:28:28.471113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032308089999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9631,"num_read_kb":9532,"num_write":2186,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":392734,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"134'780","reported_seq":3314,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463834+0000","last_change":"2024-09-18T00:20:17.579849+0000","last_active":"2024-09-18T00:20:49.463834+0000","last_peered":"2024-09-18T00:20:49.463834+0000","last_clean":"2024-09-18T00:20:49.463834+0000","last_became_active":"2024-09-18T00:19:18.333730+0000","last_became_peered":"2024-09-18T00:19:18.333730+0000","last_unstale":"2024-09-18T00:20:49.463834+0000","last_undegraded":"2024-09-18T00:20:49.463834+0000","last_fullsized":"2024-09-18T00:20:49.463834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:12:14.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029127850000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5845,"num_read_kb":7620,"num_write":1029,"num_write_kb":6233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":697208,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"134'352","reported_seq":1424,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463902+0000","last_change":"2024-09-18T00:20:17.578670+0000","last_active":"2024-09-18T00:20:49.463902+0000","last_peered":"2024-09-18T00:20:49.463902+0000","last_clean":"2024-09-18T00:20:49.463902+0000","last_became_active":"2024-09-18T00:18:16.440223+0000","last_became_peered":"2024-09-18T00:18:16.440223+0000","last_unstale":"2024-09-18T00:20:49.463902+0000","last_undegraded":"2024-09-18T00:20:49.463902+0000","last_fullsized":"2024-09-18T00:20:49.463902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:17.972019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002121001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1129,"num_read_kb":6709,"num_write":391,"num_write_kb":4357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":7258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"133'397","reported_seq":3507,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.464104+0000","last_change":"2024-09-18T00:20:17.579351+0000","last_active":"2024-09-18T00:20:49.464104+0000","last_peered":"2024-09-18T00:20:49.464104+0000","last_clean":"2024-09-18T00:20:49.464104+0000","last_became_active":"2024-09-18T00:18:16.441020+0000","last_became_peered":"2024-09-18T00:18:16.441020+0000","last_unstale":"2024-09-18T00:20:49.464104+0000","last_undegraded":"2024-09-18T00:20:49.464104+0000","last_fullsized":"2024-09-18T00:20:49.464104+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:01:45.246626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00266241,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4197,"num_read_kb":11008,"num_write":397,"num_write_kb":9044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"134'378","reported_seq":4809,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451504+0000","last_change":"2024-09-18T00:20:37.726301+0000","last_active":"2024-09-18T00:20:48.451504+0000","last_peered":"2024-09-18T00:20:48.451504+0000","last_clean":"2024-09-18T00:20:48.451504+0000","last_became_active":"2024-09-18T00:20:37.723504+0000","last_became_peered":"2024-09-18T00:20:37.723504+0000","last_unstale":"2024-09-18T00:20:48.451504+0000","last_undegraded":"2024-09-18T00:20:48.451504+0000","last_fullsized":"2024-09-18T00:20:48.451504+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:16:42.705342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001519245,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5107,"num_read_kb":32748,"num_write":405,"num_write_kb":13893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":3037482,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"133'368","reported_seq":4657,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463399+0000","last_change":"2024-09-18T00:20:17.577141+0000","last_active":"2024-09-18T00:20:49.463399+0000","last_peered":"2024-09-18T00:20:49.463399+0000","last_clean":"2024-09-18T00:20:49.463399+0000","last_became_active":"2024-09-18T00:18:16.438286+0000","last_became_peered":"2024-09-18T00:18:16.438286+0000","last_unstale":"2024-09-18T00:20:49.463399+0000","last_undegraded":"2024-09-18T00:20:49.463399+0000","last_fullsized":"2024-09-18T00:20:49.463399+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:09:46.156797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092097700000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4916,"num_read_kb":14028,"num_write":359,"num_write_kb":4057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":70145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"134'355","reported_seq":1976,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451372+0000","last_change":"2024-09-18T00:20:17.572303+0000","last_active":"2024-09-18T00:20:48.451372+0000","last_peered":"2024-09-18T00:20:48.451372+0000","last_clean":"2024-09-18T00:20:48.451372+0000","last_became_active":"2024-09-18T00:18:16.438556+0000","last_became_peered":"2024-09-18T00:18:16.438556+0000","last_unstale":"2024-09-18T00:20:48.451372+0000","last_undegraded":"2024-09-18T00:20:48.451372+0000","last_fullsized":"2024-09-18T00:20:48.451372+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:36.881982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019750979999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1630,"num_read_kb":9063,"num_write":386,"num_write_kb":8736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":47120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"133'748","reported_seq":3136,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463471+0000","last_change":"2024-09-18T00:20:17.579483+0000","last_active":"2024-09-18T00:20:49.463471+0000","last_peered":"2024-09-18T00:20:49.463471+0000","last_clean":"2024-09-18T00:20:49.463471+0000","last_became_active":"2024-09-18T00:18:16.440910+0000","last_became_peered":"2024-09-18T00:18:16.440910+0000","last_unstale":"2024-09-18T00:20:49.463471+0000","last_undegraded":"2024-09-18T00:20:49.463471+0000","last_fullsized":"2024-09-18T00:20:49.463471+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:00:51.845256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028046910000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4645,"num_read_kb":12707,"num_write":1147,"num_write_kb":3134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":65,"num_read":96502,"num_read_kb":273433,"num_write":19290,"num_write_kb":181728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1834,"num_bytes_recovered":25828399,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19623,"internal_metadata":0},"log_size":16196,"ondisk_log_size":16196,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20247,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55260,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316580,"statfs":{"total":96636764160,"available":96580177920,"internally_reserved":0,"allocated":1908736,"data_stored":1478380,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37172,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334668,"statfs":{"total":96636764160,"available":96598700032,"internally_reserved":0,"allocated":1867776,"data_stored":888012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":76,"seq":326417514518,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38212,"kb_used_data":1840,"kb_used_omap":9,"kb_used_meta":36342,"kb_avail":94333628,"statfs":{"total":96636764160,"available":96597635072,"internally_reserved":0,"allocated":1884160,"data_stored":893873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9346,"internal_metadata":37215102},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55228,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1871872,"data_stored":888031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41344,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330496,"statfs":{"total":96636764160,"available":96594427904,"internally_reserved":0,"allocated":1880064,"data_stored":893854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":139,"seq":597000454151,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":1844,"kb_used_omap":17,"kb_used_meta":30190,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":1888256,"data_stored":893892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17900,"internal_metadata":30915092},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542185,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73280,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298560,"statfs":{"total":96636764160,"available":96561725440,"internally_reserved":0,"allocated":1880064,"data_stored":893854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1925120,"data_stored":1478380,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13445,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6178,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:20:55.492 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T00:20:55.492 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:20:55.771 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:20:55.772 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-18T00:20:55.788 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-18T00:20:54.348193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":65,"num_read":96618,"num_read_kb":273533,"num_write":19451,"num_write_kb":184146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1839,"num_bytes_recovered":27009154,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16316,"ondisk_log_size":16316,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":365648,"kb_used_data":14752,"kb_used_omap":35,"kb_used_meta":350684,"kb_avail":754609072,"statfs":{"total":773094113280,"available":772719689728,"internally_reserved":0,"allocated":15106048,"data_stored":8308276,"data_compressed":13209,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":36807,"internal_metadata":359100473},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.911748"},"pg_stats":[{"pgid":"3.1f","version":"135'778","reported_seq":3008,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451528+0000","last_change":"2024-09-18T00:20:17.573423+0000","last_active":"2024-09-18T00:20:48.451528+0000","last_peered":"2024-09-18T00:20:48.451528+0000","last_clean":"2024-09-18T00:20:48.451528+0000","last_became_active":"2024-09-18T00:19:18.327350+0000","last_became_peered":"2024-09-18T00:19:18.327350+0000","last_unstale":"2024-09-18T00:20:48.451528+0000","last_undegraded":"2024-09-18T00:20:48.451528+0000","last_fullsized":"2024-09-18T00:20:48.451528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":778,"log_dups_size":0,"ondisk_log_size":778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:46:14.185856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024250489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4587,"num_read_kb":13915,"num_write":1017,"num_write_kb":7086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":2252743,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"133'465","reported_seq":1559,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095625+0000","last_change":"2024-09-18T00:20:37.723738+0000","last_active":"2024-09-18T00:20:51.095625+0000","last_peered":"2024-09-18T00:20:51.095625+0000","last_clean":"2024-09-18T00:20:51.095625+0000","last_became_active":"2024-09-18T00:20:37.722312+0000","last_became_peered":"2024-09-18T00:20:37.722312+0000","last_unstale":"2024-09-18T00:20:51.095625+0000","last_undegraded":"2024-09-18T00:20:51.095625+0000","last_fullsized":"2024-09-18T00:20:51.095625+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:14:23.427889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057225740000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1405,"num_read_kb":1924,"num_write":538,"num_write_kb":2847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":46741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"135'781","reported_seq":1532,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097665+0000","last_change":"2024-09-18T00:20:48.470232+0000","last_active":"2024-09-18T00:20:51.097665+0000","last_peered":"2024-09-18T00:20:51.097665+0000","last_clean":"2024-09-18T00:20:51.097665+0000","last_became_active":"2024-09-18T00:20:48.465769+0000","last_became_peered":"2024-09-18T00:20:48.465769+0000","last_unstale":"2024-09-18T00:20:51.097665+0000","last_undegraded":"2024-09-18T00:20:51.097665+0000","last_fullsized":"2024-09-18T00:20:51.097665+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:11:07.738987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032750750000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3226,"num_read_kb":4864,"num_write":1184,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":510608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"135'453","reported_seq":1953,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097692+0000","last_change":"2024-09-18T00:20:48.467870+0000","last_active":"2024-09-18T00:20:51.097692+0000","last_peered":"2024-09-18T00:20:51.097692+0000","last_clean":"2024-09-18T00:20:51.097692+0000","last_became_active":"2024-09-18T00:20:48.463565+0000","last_became_peered":"2024-09-18T00:20:48.463565+0000","last_unstale":"2024-09-18T00:20:51.097692+0000","last_undegraded":"2024-09-18T00:20:51.097692+0000","last_fullsized":"2024-09-18T00:20:51.097692+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:07:09.913555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002116522,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3432,"num_read_kb":9919,"num_write":470,"num_write_kb":3833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":72221,"num_keys_recovered":54,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"133'289","reported_seq":1665,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451394+0000","last_change":"2024-09-18T00:20:17.572140+0000","last_active":"2024-09-18T00:20:48.451394+0000","last_peered":"2024-09-18T00:20:48.451394+0000","last_clean":"2024-09-18T00:20:48.451394+0000","last_became_active":"2024-09-18T00:18:05.415810+0000","last_became_peered":"2024-09-18T00:18:05.415810+0000","last_unstale":"2024-09-18T00:20:48.451394+0000","last_undegraded":"2024-09-18T00:20:48.451394+0000","last_fullsized":"2024-09-18T00:20:48.451394+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:35:11.075422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015232959999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1498,"num_read_kb":2662,"num_write":281,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"140'226","reported_seq":3328,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459066+0000","last_change":"2024-09-18T00:20:30.686023+0000","last_active":"2024-09-18T00:20:48.459066+0000","last_peered":"2024-09-18T00:20:48.459066+0000","last_clean":"2024-09-18T00:20:48.459066+0000","last_became_active":"2024-09-18T00:20:30.637311+0000","last_became_peered":"2024-09-18T00:20:30.637311+0000","last_unstale":"2024-09-18T00:20:48.459066+0000","last_undegraded":"2024-09-18T00:20:48.459066+0000","last_fullsized":"2024-09-18T00:20:48.459066+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:56:01.105381+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012894545,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":2320,"num_read_kb":4551,"num_write":211,"num_write_kb":2083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":645956,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"140'384","reported_seq":1512,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462374+0000","last_change":"2024-09-18T00:20:30.693807+0000","last_active":"2024-09-18T00:20:49.462374+0000","last_peered":"2024-09-18T00:20:49.462374+0000","last_clean":"2024-09-18T00:20:49.462374+0000","last_became_active":"2024-09-18T00:20:30.640499+0000","last_became_peered":"2024-09-18T00:20:30.640499+0000","last_unstale":"2024-09-18T00:20:49.462374+0000","last_undegraded":"2024-09-18T00:20:49.462374+0000","last_fullsized":"2024-09-18T00:20:49.462374+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:26.838200+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018217595999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1131,"num_read_kb":4419,"num_write":389,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":1283165,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"133'327","reported_seq":3718,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095462+0000","last_change":"2024-09-18T00:20:17.694311+0000","last_active":"2024-09-18T00:20:51.095462+0000","last_peered":"2024-09-18T00:20:51.095462+0000","last_clean":"2024-09-18T00:20:51.095462+0000","last_became_active":"2024-09-18T00:18:05.406821+0000","last_became_peered":"2024-09-18T00:18:05.406821+0000","last_unstale":"2024-09-18T00:20:51.095462+0000","last_undegraded":"2024-09-18T00:20:51.095462+0000","last_fullsized":"2024-09-18T00:20:51.095462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:13:08.591812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022731449999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3612,"num_read_kb":21086,"num_write":298,"num_write_kb":8166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'372","reported_seq":1395,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451657+0000","last_change":"2024-09-18T00:20:17.573988+0000","last_active":"2024-09-18T00:20:48.451657+0000","last_peered":"2024-09-18T00:20:48.451657+0000","last_clean":"2024-09-18T00:20:48.451657+0000","last_became_active":"2024-09-18T00:19:18.328043+0000","last_became_peered":"2024-09-18T00:19:18.328043+0000","last_unstale":"2024-09-18T00:20:48.451657+0000","last_undegraded":"2024-09-18T00:20:48.451657+0000","last_fullsized":"2024-09-18T00:20:48.451657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:11:44.153392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025255270000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1318,"num_read_kb":6224,"num_write":322,"num_write_kb":4815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":491875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"135'690","reported_seq":2488,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097894+0000","last_change":"2024-09-18T00:20:48.471288+0000","last_active":"2024-09-18T00:20:51.097894+0000","last_peered":"2024-09-18T00:20:51.097894+0000","last_clean":"2024-09-18T00:20:51.097894+0000","last_became_active":"2024-09-18T00:20:48.467364+0000","last_became_peered":"2024-09-18T00:20:48.467364+0000","last_unstale":"2024-09-18T00:20:51.097894+0000","last_undegraded":"2024-09-18T00:20:51.097894+0000","last_fullsized":"2024-09-18T00:20:51.097894+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:25.555616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021376099999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4502,"num_read_kb":5989,"num_write":925,"num_write_kb":5689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1385456,"num_keys_recovered":29,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"133'489","reported_seq":1056,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463552+0000","last_change":"2024-09-18T00:20:17.577703+0000","last_active":"2024-09-18T00:20:49.463552+0000","last_peered":"2024-09-18T00:20:49.463552+0000","last_clean":"2024-09-18T00:20:49.463552+0000","last_became_active":"2024-09-18T00:18:16.438164+0000","last_became_peered":"2024-09-18T00:18:16.438164+0000","last_unstale":"2024-09-18T00:20:49.463552+0000","last_undegraded":"2024-09-18T00:20:49.463552+0000","last_fullsized":"2024-09-18T00:20:49.463552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:18:47.201515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014879730000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":1078,"num_read_kb":1346,"num_write":552,"num_write_kb":3993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"134'530","reported_seq":1981,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459503+0000","last_change":"2024-09-18T00:20:17.576876+0000","last_active":"2024-09-18T00:20:48.459503+0000","last_peered":"2024-09-18T00:20:48.459503+0000","last_clean":"2024-09-18T00:20:48.459503+0000","last_became_active":"2024-09-18T00:18:16.441758+0000","last_became_peered":"2024-09-18T00:18:16.441758+0000","last_unstale":"2024-09-18T00:20:48.459503+0000","last_undegraded":"2024-09-18T00:20:48.459503+0000","last_fullsized":"2024-09-18T00:20:48.459503+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:18:47.366485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050089940000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2844,"num_read_kb":5653,"num_write":555,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":103027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":38,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451888+0000","last_change":"2024-09-18T00:20:34.754460+0000","last_active":"2024-09-18T00:20:48.451888+0000","last_peered":"2024-09-18T00:20:48.451888+0000","last_clean":"2024-09-18T00:20:48.451888+0000","last_became_active":"2024-09-18T00:20:34.753675+0000","last_became_peered":"2024-09-18T00:20:34.753675+0000","last_unstale":"2024-09-18T00:20:48.451888+0000","last_undegraded":"2024-09-18T00:20:48.451888+0000","last_fullsized":"2024-09-18T00:20:48.451888+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:37.203669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089554000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"135'348","reported_seq":2423,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.452208+0000","last_change":"2024-09-18T00:20:30.727046+0000","last_active":"2024-09-18T00:20:48.452208+0000","last_peered":"2024-09-18T00:20:48.452208+0000","last_clean":"2024-09-18T00:20:48.452208+0000","last_became_active":"2024-09-18T00:20:30.648069+0000","last_became_peered":"2024-09-18T00:20:30.648069+0000","last_unstale":"2024-09-18T00:20:48.452208+0000","last_undegraded":"2024-09-18T00:20:48.452208+0000","last_fullsized":"2024-09-18T00:20:48.452208+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:36.621517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052221869999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2016,"num_read_kb":5374,"num_write":329,"num_write_kb":5799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":1635567,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":59,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462482+0000","last_change":"2024-09-18T00:20:34.694081+0000","last_active":"2024-09-18T00:20:49.462482+0000","last_peered":"2024-09-18T00:20:49.462482+0000","last_clean":"2024-09-18T00:20:49.462482+0000","last_became_active":"2024-09-18T00:20:34.693550+0000","last_became_peered":"2024-09-18T00:20:34.693550+0000","last_unstale":"2024-09-18T00:20:49.462482+0000","last_undegraded":"2024-09-18T00:20:49.462482+0000","last_fullsized":"2024-09-18T00:20:49.462482+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:34:59.142586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'665","reported_seq":3138,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095867+0000","last_change":"2024-09-18T00:20:17.695571+0000","last_active":"2024-09-18T00:20:51.095867+0000","last_peered":"2024-09-18T00:20:51.095867+0000","last_clean":"2024-09-18T00:20:51.095867+0000","last_became_active":"2024-09-18T00:18:16.440536+0000","last_became_peered":"2024-09-18T00:18:16.440536+0000","last_unstale":"2024-09-18T00:20:51.095867+0000","last_undegraded":"2024-09-18T00:20:51.095867+0000","last_fullsized":"2024-09-18T00:20:51.095867+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:35.729973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034870019999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":3858,"num_read_kb":13564,"num_write":915,"num_write_kb":9611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":73512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":257,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463801+0000","last_change":"2024-09-18T00:18:16.439964+0000","last_active":"2024-09-18T00:20:49.463801+0000","last_peered":"2024-09-18T00:20:49.463801+0000","last_clean":"2024-09-18T00:20:49.463801+0000","last_became_active":"2024-09-18T00:18:16.437150+0000","last_became_peered":"2024-09-18T00:18:16.437150+0000","last_unstale":"2024-09-18T00:20:49.463801+0000","last_undegraded":"2024-09-18T00:20:49.463801+0000","last_fullsized":"2024-09-18T00:20:49.463801+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:25.772294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"140'491","reported_seq":2308,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462407+0000","last_change":"2024-09-18T00:20:30.710100+0000","last_active":"2024-09-18T00:20:49.462407+0000","last_peered":"2024-09-18T00:20:49.462407+0000","last_clean":"2024-09-18T00:20:49.462407+0000","last_became_active":"2024-09-18T00:20:30.641845+0000","last_became_peered":"2024-09-18T00:20:30.641845+0000","last_unstale":"2024-09-18T00:20:49.462407+0000","last_undegraded":"2024-09-18T00:20:49.462407+0000","last_fullsized":"2024-09-18T00:20:49.462407+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:00:46.444023+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.013533892,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2426,"num_read_kb":4923,"num_write":423,"num_write_kb":4982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":1553378,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462313+0000","last_change":"2024-09-18T00:20:30.651121+0000","last_active":"2024-09-18T00:20:49.462313+0000","last_peered":"2024-09-18T00:20:49.462313+0000","last_clean":"2024-09-18T00:20:49.462313+0000","last_became_active":"2024-09-18T00:20:30.650373+0000","last_became_peered":"2024-09-18T00:20:30.650373+0000","last_unstale":"2024-09-18T00:20:49.462313+0000","last_undegraded":"2024-09-18T00:20:49.462313+0000","last_fullsized":"2024-09-18T00:20:49.462313+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:36:36.287145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"133'363","reported_seq":2714,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097799+0000","last_change":"2024-09-18T00:20:48.470308+0000","last_active":"2024-09-18T00:20:51.097799+0000","last_peered":"2024-09-18T00:20:51.097799+0000","last_clean":"2024-09-18T00:20:51.097799+0000","last_became_active":"2024-09-18T00:20:48.462691+0000","last_became_peered":"2024-09-18T00:20:48.462691+0000","last_unstale":"2024-09-18T00:20:51.097799+0000","last_undegraded":"2024-09-18T00:20:51.097799+0000","last_fullsized":"2024-09-18T00:20:51.097799+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:08.083732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001889571,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2213,"num_read_kb":12284,"num_write":366,"num_write_kb":5310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":675014,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"135'484","reported_seq":2135,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459435+0000","last_change":"2024-09-18T00:20:17.573853+0000","last_active":"2024-09-18T00:20:48.459435+0000","last_peered":"2024-09-18T00:20:48.459435+0000","last_clean":"2024-09-18T00:20:48.459435+0000","last_became_active":"2024-09-18T00:18:05.407889+0000","last_became_peered":"2024-09-18T00:18:05.407889+0000","last_unstale":"2024-09-18T00:20:48.459435+0000","last_undegraded":"2024-09-18T00:20:48.459435+0000","last_fullsized":"2024-09-18T00:20:48.459435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:40.520294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002502663,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":1766,"num_read_kb":5407,"num_write":446,"num_write_kb":5379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":203,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.098054+0000","last_change":"2024-09-18T00:20:48.461100+0000","last_active":"2024-09-18T00:20:51.098054+0000","last_peered":"2024-09-18T00:20:51.098054+0000","last_clean":"2024-09-18T00:20:51.098054+0000","last_became_active":"2024-09-18T00:20:48.460277+0000","last_became_peered":"2024-09-18T00:20:48.460277+0000","last_unstale":"2024-09-18T00:20:51.098054+0000","last_undegraded":"2024-09-18T00:20:51.098054+0000","last_fullsized":"2024-09-18T00:20:51.098054+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:01:02.391844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088189300000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"134'369","reported_seq":1075,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462205+0000","last_change":"2024-09-18T00:20:30.726339+0000","last_active":"2024-09-18T00:20:49.462205+0000","last_peered":"2024-09-18T00:20:49.462205+0000","last_clean":"2024-09-18T00:20:49.462205+0000","last_became_active":"2024-09-18T00:20:30.647777+0000","last_became_peered":"2024-09-18T00:20:30.647777+0000","last_unstale":"2024-09-18T00:20:49.462205+0000","last_undegraded":"2024-09-18T00:20:49.462205+0000","last_fullsized":"2024-09-18T00:20:49.462205+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:21:04.035067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011927055000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":740,"num_read_kb":1879,"num_write":446,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":1002237,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":289,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.461675+0000","last_change":"2024-09-18T00:17:57.477633+0000","last_active":"2024-09-18T00:20:49.461675+0000","last_peered":"2024-09-18T00:20:49.461675+0000","last_clean":"2024-09-18T00:20:49.461675+0000","last_became_active":"2024-09-18T00:17:55.631277+0000","last_became_peered":"2024-09-18T00:17:55.631277+0000","last_unstale":"2024-09-18T00:20:49.461675+0000","last_undegraded":"2024-09-18T00:20:49.461675+0000","last_fullsized":"2024-09-18T00:20:49.461675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:55.421684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001141204,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"135'502","reported_seq":2618,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451329+0000","last_change":"2024-09-18T00:20:17.571764+0000","last_active":"2024-09-18T00:20:48.451329+0000","last_peered":"2024-09-18T00:20:48.451329+0000","last_clean":"2024-09-18T00:20:48.451329+0000","last_became_active":"2024-09-18T00:18:05.410952+0000","last_became_peered":"2024-09-18T00:18:05.410952+0000","last_unstale":"2024-09-18T00:20:48.451329+0000","last_undegraded":"2024-09-18T00:20:48.451329+0000","last_fullsized":"2024-09-18T00:20:48.451329+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:15.435600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014272110000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3308,"num_read_kb":10125,"num_write":594,"num_write_kb":8641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":276,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463375+0000","last_change":"2024-09-18T00:20:30.657193+0000","last_active":"2024-09-18T00:20:49.463375+0000","last_peered":"2024-09-18T00:20:49.463375+0000","last_clean":"2024-09-18T00:20:49.463375+0000","last_became_active":"2024-09-18T00:20:30.656400+0000","last_became_peered":"2024-09-18T00:20:30.656400+0000","last_unstale":"2024-09-18T00:20:49.463375+0000","last_undegraded":"2024-09-18T00:20:49.463375+0000","last_fullsized":"2024-09-18T00:20:49.463375+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:50:27.874106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"134'493","reported_seq":2493,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095513+0000","last_change":"2024-09-18T00:20:17.697271+0000","last_active":"2024-09-18T00:20:51.095513+0000","last_peered":"2024-09-18T00:20:51.095513+0000","last_clean":"2024-09-18T00:20:51.095513+0000","last_became_active":"2024-09-18T00:19:18.330155+0000","last_became_peered":"2024-09-18T00:19:18.330155+0000","last_unstale":"2024-09-18T00:20:51.095513+0000","last_undegraded":"2024-09-18T00:20:51.095513+0000","last_fullsized":"2024-09-18T00:20:51.095513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:01:28.392225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005129074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2497,"num_read_kb":7710,"num_write":474,"num_write_kb":3192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":493176,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":122,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097622+0000","last_change":"2024-09-18T00:20:48.459774+0000","last_active":"2024-09-18T00:20:51.097622+0000","last_peered":"2024-09-18T00:20:51.097622+0000","last_clean":"2024-09-18T00:20:51.097622+0000","last_became_active":"2024-09-18T00:20:48.457808+0000","last_became_peered":"2024-09-18T00:20:48.457808+0000","last_unstale":"2024-09-18T00:20:51.097622+0000","last_undegraded":"2024-09-18T00:20:51.097622+0000","last_fullsized":"2024-09-18T00:20:51.097622+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:43:17.325507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":438,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463383+0000","last_change":"2024-09-18T00:18:19.466612+0000","last_active":"2024-09-18T00:20:49.463383+0000","last_peered":"2024-09-18T00:20:49.463383+0000","last_clean":"2024-09-18T00:20:49.463383+0000","last_became_active":"2024-09-18T00:18:19.465557+0000","last_became_peered":"2024-09-18T00:18:19.465557+0000","last_unstale":"2024-09-18T00:20:49.463383+0000","last_undegraded":"2024-09-18T00:20:49.463383+0000","last_fullsized":"2024-09-18T00:20:49.463383+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_clean_scrub_stamp":"2024-09-18T00:17:50.414591+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:08:40.389972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"140'661","reported_seq":2377,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095512+0000","last_change":"2024-09-18T00:20:30.742660+0000","last_active":"2024-09-18T00:20:51.095512+0000","last_peered":"2024-09-18T00:20:51.095512+0000","last_clean":"2024-09-18T00:20:51.095512+0000","last_became_active":"2024-09-18T00:20:30.656699+0000","last_became_peered":"2024-09-18T00:20:30.656699+0000","last_unstale":"2024-09-18T00:20:51.095512+0000","last_undegraded":"2024-09-18T00:20:51.095512+0000","last_fullsized":"2024-09-18T00:20:51.095512+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:37:48.645340+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0070126900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3348,"num_read_kb":6619,"num_write":872,"num_write_kb":5519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":1197799,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"134'383","reported_seq":2112,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.459075+0000","last_change":"2024-09-18T00:20:17.574821+0000","last_active":"2024-09-18T00:20:48.459075+0000","last_peered":"2024-09-18T00:20:48.459075+0000","last_clean":"2024-09-18T00:20:48.459075+0000","last_became_active":"2024-09-18T00:18:05.401097+0000","last_became_peered":"2024-09-18T00:18:05.401097+0000","last_unstale":"2024-09-18T00:20:48.459075+0000","last_undegraded":"2024-09-18T00:20:48.459075+0000","last_fullsized":"2024-09-18T00:20:48.459075+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:50:58.016930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003875547,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3069,"num_read_kb":9114,"num_write":378,"num_write_kb":6107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"140'412","reported_seq":2388,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451796+0000","last_change":"2024-09-18T00:20:37.726930+0000","last_active":"2024-09-18T00:20:48.451796+0000","last_peered":"2024-09-18T00:20:48.451796+0000","last_clean":"2024-09-18T00:20:48.451796+0000","last_became_active":"2024-09-18T00:20:37.721802+0000","last_became_peered":"2024-09-18T00:20:37.721802+0000","last_unstale":"2024-09-18T00:20:48.451796+0000","last_undegraded":"2024-09-18T00:20:48.451796+0000","last_fullsized":"2024-09-18T00:20:48.451796+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:37.117595+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.007118643,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":1971,"num_read_kb":7346,"num_write":375,"num_write_kb":8437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":8153977,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"134'425","reported_seq":1164,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095561+0000","last_change":"2024-09-18T00:20:17.696586+0000","last_active":"2024-09-18T00:20:51.095561+0000","last_peered":"2024-09-18T00:20:51.095561+0000","last_clean":"2024-09-18T00:20:51.095561+0000","last_became_active":"2024-09-18T00:18:05.406977+0000","last_became_peered":"2024-09-18T00:18:05.406977+0000","last_unstale":"2024-09-18T00:20:51.095561+0000","last_undegraded":"2024-09-18T00:20:51.095561+0000","last_fullsized":"2024-09-18T00:20:51.095561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:06:59.846008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044405699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1237,"num_read_kb":3121,"num_write":630,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"135'1438","reported_seq":3733,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097706+0000","last_change":"2024-09-18T00:20:48.467838+0000","last_active":"2024-09-18T00:20:51.097706+0000","last_peered":"2024-09-18T00:20:51.097706+0000","last_clean":"2024-09-18T00:20:51.097706+0000","last_became_active":"2024-09-18T00:20:48.462359+0000","last_became_peered":"2024-09-18T00:20:48.462359+0000","last_unstale":"2024-09-18T00:20:51.097706+0000","last_undegraded":"2024-09-18T00:20:51.097706+0000","last_fullsized":"2024-09-18T00:20:51.097706+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":1438,"log_dups_size":0,"ondisk_log_size":1438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:28:28.471113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032308089999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9631,"num_read_kb":9532,"num_write":2186,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":392734,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"134'780","reported_seq":3314,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463834+0000","last_change":"2024-09-18T00:20:17.579849+0000","last_active":"2024-09-18T00:20:49.463834+0000","last_peered":"2024-09-18T00:20:49.463834+0000","last_clean":"2024-09-18T00:20:49.463834+0000","last_became_active":"2024-09-18T00:19:18.333730+0000","last_became_peered":"2024-09-18T00:19:18.333730+0000","last_unstale":"2024-09-18T00:20:49.463834+0000","last_undegraded":"2024-09-18T00:20:49.463834+0000","last_fullsized":"2024-09-18T00:20:49.463834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:12:14.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029127850000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5845,"num_read_kb":7620,"num_write":1029,"num_write_kb":6233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":697208,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"134'352","reported_seq":1424,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463902+0000","last_change":"2024-09-18T00:20:17.578670+0000","last_active":"2024-09-18T00:20:49.463902+0000","last_peered":"2024-09-18T00:20:49.463902+0000","last_clean":"2024-09-18T00:20:49.463902+0000","last_became_active":"2024-09-18T00:18:16.440223+0000","last_became_peered":"2024-09-18T00:18:16.440223+0000","last_unstale":"2024-09-18T00:20:49.463902+0000","last_undegraded":"2024-09-18T00:20:49.463902+0000","last_fullsized":"2024-09-18T00:20:49.463902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:17.972019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002121001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1129,"num_read_kb":6709,"num_write":391,"num_write_kb":4357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":7258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"133'397","reported_seq":3507,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.464104+0000","last_change":"2024-09-18T00:20:17.579351+0000","last_active":"2024-09-18T00:20:49.464104+0000","last_peered":"2024-09-18T00:20:49.464104+0000","last_clean":"2024-09-18T00:20:49.464104+0000","last_became_active":"2024-09-18T00:18:16.441020+0000","last_became_peered":"2024-09-18T00:18:16.441020+0000","last_unstale":"2024-09-18T00:20:49.464104+0000","last_undegraded":"2024-09-18T00:20:49.464104+0000","last_fullsized":"2024-09-18T00:20:49.464104+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:01:45.246626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00266241,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4197,"num_read_kb":11008,"num_write":397,"num_write_kb":9044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"134'378","reported_seq":4809,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451504+0000","last_change":"2024-09-18T00:20:37.726301+0000","last_active":"2024-09-18T00:20:48.451504+0000","last_peered":"2024-09-18T00:20:48.451504+0000","last_clean":"2024-09-18T00:20:48.451504+0000","last_became_active":"2024-09-18T00:20:37.723504+0000","last_became_peered":"2024-09-18T00:20:37.723504+0000","last_unstale":"2024-09-18T00:20:48.451504+0000","last_undegraded":"2024-09-18T00:20:48.451504+0000","last_fullsized":"2024-09-18T00:20:48.451504+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:16:42.705342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001519245,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5107,"num_read_kb":32748,"num_write":405,"num_write_kb":13893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":3037482,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"133'368","reported_seq":4657,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463399+0000","last_change":"2024-09-18T00:20:17.577141+0000","last_active":"2024-09-18T00:20:49.463399+0000","last_peered":"2024-09-18T00:20:49.463399+0000","last_clean":"2024-09-18T00:20:49.463399+0000","last_became_active":"2024-09-18T00:18:16.438286+0000","last_became_peered":"2024-09-18T00:18:16.438286+0000","last_unstale":"2024-09-18T00:20:49.463399+0000","last_undegraded":"2024-09-18T00:20:49.463399+0000","last_fullsized":"2024-09-18T00:20:49.463399+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:09:46.156797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092097700000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4916,"num_read_kb":14028,"num_write":359,"num_write_kb":4057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":70145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"134'355","reported_seq":1976,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-18T00:20:48.451372+0000","last_change":"2024-09-18T00:20:17.572303+0000","last_active":"2024-09-18T00:20:48.451372+0000","last_peered":"2024-09-18T00:20:48.451372+0000","last_clean":"2024-09-18T00:20:48.451372+0000","last_became_active":"2024-09-18T00:18:16.438556+0000","last_became_peered":"2024-09-18T00:18:16.438556+0000","last_unstale":"2024-09-18T00:20:48.451372+0000","last_undegraded":"2024-09-18T00:20:48.451372+0000","last_fullsized":"2024-09-18T00:20:48.451372+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:36.881982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019750979999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1630,"num_read_kb":9063,"num_write":386,"num_write_kb":8736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":47120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"133'748","reported_seq":3136,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463471+0000","last_change":"2024-09-18T00:20:17.579483+0000","last_active":"2024-09-18T00:20:49.463471+0000","last_peered":"2024-09-18T00:20:49.463471+0000","last_clean":"2024-09-18T00:20:49.463471+0000","last_became_active":"2024-09-18T00:18:16.440910+0000","last_became_peered":"2024-09-18T00:18:16.440910+0000","last_unstale":"2024-09-18T00:20:49.463471+0000","last_undegraded":"2024-09-18T00:20:49.463471+0000","last_fullsized":"2024-09-18T00:20:49.463471+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:00:51.845256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028046910000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4645,"num_read_kb":12707,"num_write":1147,"num_write_kb":3134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":65,"num_read":96502,"num_read_kb":273433,"num_write":19290,"num_write_kb":181728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1834,"num_bytes_recovered":25828399,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19623,"internal_metadata":0},"log_size":16196,"ondisk_log_size":16196,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20247,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55260,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316580,"statfs":{"total":96636764160,"available":96580177920,"internally_reserved":0,"allocated":1908736,"data_stored":1478380,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37172,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334668,"statfs":{"total":96636764160,"available":96598700032,"internally_reserved":0,"allocated":1867776,"data_stored":888012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":76,"seq":326417514518,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38212,"kb_used_data":1840,"kb_used_omap":9,"kb_used_meta":36342,"kb_avail":94333628,"statfs":{"total":96636764160,"available":96597635072,"internally_reserved":0,"allocated":1884160,"data_stored":893873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9346,"internal_metadata":37215102},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55228,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316612,"statfs":{"total":96636764160,"available":96580210688,"internally_reserved":0,"allocated":1871872,"data_stored":888031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41344,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330496,"statfs":{"total":96636764160,"available":96594427904,"internally_reserved":0,"allocated":1880064,"data_stored":893854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":139,"seq":597000454151,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":1844,"kb_used_omap":17,"kb_used_meta":30190,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":1888256,"data_stored":893892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17900,"internal_metadata":30915092},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542185,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73280,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298560,"statfs":{"total":96636764160,"available":96561725440,"internally_reserved":0,"allocated":1880064,"data_stored":893854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1925120,"data_stored":1478380,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13445,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6178,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:20:55.791 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T00:20:56.079 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:20:56.079 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":157,"fsid":"b993f56a-c4ef-4602-bb41-f32ae406b45a","created":"2024-09-18T00:17:35.658788+0000","modified":"2024-09-18T00:20:49.455539+0000","last_up_change":"2024-09-18T00:20:29.617045+0000","last_in_change":"2024-09-18T00:20:33.661993+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T00:17:50.375367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T00:17:53.896718+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-18T00:18:03.715378+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":30,"snap_epoch":135,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bfda7950-3d83-435d-8313-2f14345096ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6801","nonce":40468291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6803","nonce":40468291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6807","nonce":40468291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":40468291},{"type":"v1","addr":"172.21.15.97:6805","nonce":40468291}]},"public_addr":"172.21.15.97:6801/40468291","cluster_addr":"172.21.15.97:6803/40468291","heartbeat_back_addr":"172.21.15.97:6807/40468291","heartbeat_front_addr":"172.21.15.97:6805/40468291","state":["exists","up"]},{"osd":1,"uuid":"35949061-aa30-4aff-b39d-90aebeffb115","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":106,"last_clean_end":126,"up_from":139,"up_thru":143,"down_at":127,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1355723817},{"type":"v1","addr":"172.21.15.97:6825","nonce":1355723817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1355723817},{"type":"v1","addr":"172.21.15.97:6827","nonce":1355723817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1355723817},{"type":"v1","addr":"172.21.15.97:6831","nonce":1355723817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1355723817},{"type":"v1","addr":"172.21.15.97:6829","nonce":1355723817}]},"public_addr":"172.21.15.97:6825/1355723817","cluster_addr":"172.21.15.97:6827/1355723817","heartbeat_back_addr":"172.21.15.97:6831/1355723817","heartbeat_front_addr":"172.21.15.97:6829/1355723817","state":["exists","up"]},{"osd":2,"uuid":"31b3e1c1-7648-44fe-9931-758ffe9a53ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6817","nonce":2425060925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6819","nonce":2425060925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6823","nonce":2425060925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2425060925},{"type":"v1","addr":"172.21.15.97:6821","nonce":2425060925}]},"public_addr":"172.21.15.97:6817/2425060925","cluster_addr":"172.21.15.97:6819/2425060925","heartbeat_back_addr":"172.21.15.97:6823/2425060925","heartbeat_front_addr":"172.21.15.97:6821/2425060925","state":["exists","up"]},{"osd":3,"uuid":"d7daebbc-0e33-42ab-97ee-a6c1b449ae8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6809","nonce":3512666164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6811","nonce":3512666164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6815","nonce":3512666164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3512666164},{"type":"v1","addr":"172.21.15.97:6813","nonce":3512666164}]},"public_addr":"172.21.15.97:6809/3512666164","cluster_addr":"172.21.15.97:6811/3512666164","heartbeat_back_addr":"172.21.15.97:6815/3512666164","heartbeat_front_addr":"172.21.15.97:6813/3512666164","state":["exists","up"]},{"osd":4,"uuid":"d55ca69b-5000-45ff-acc4-00f7c4763cad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6804","nonce":3191744356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6808","nonce":3191744356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6815","nonce":3191744356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":3191744356},{"type":"v1","addr":"172.21.15.114:6812","nonce":3191744356}]},"public_addr":"172.21.15.114:6804/3191744356","cluster_addr":"172.21.15.114:6808/3191744356","heartbeat_back_addr":"172.21.15.114:6815/3191744356","heartbeat_front_addr":"172.21.15.114:6812/3191744356","state":["exists","up"]},{"osd":5,"uuid":"196a329b-bd93-4cc2-85bb-64ed81d3918b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":55,"up_from":76,"up_thru":155,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":407304039},{"type":"v1","addr":"172.21.15.114:6801","nonce":407304039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6803","nonce":407304039},{"type":"v1","addr":"172.21.15.114:6805","nonce":407304039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6811","nonce":407304039},{"type":"v1","addr":"172.21.15.114:6813","nonce":407304039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6807","nonce":407304039},{"type":"v1","addr":"172.21.15.114:6809","nonce":407304039}]},"public_addr":"172.21.15.114:6801/407304039","cluster_addr":"172.21.15.114:6805/407304039","heartbeat_back_addr":"172.21.15.114:6813/407304039","heartbeat_front_addr":"172.21.15.114:6809/407304039","state":["exists","up"]},{"osd":6,"uuid":"dc9712f0-8858-467a-8397-d5eb97187165","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6825","nonce":1148348470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6827","nonce":1148348470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6831","nonce":1148348470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1148348470},{"type":"v1","addr":"172.21.15.114:6829","nonce":1148348470}]},"public_addr":"172.21.15.114:6825/1148348470","cluster_addr":"172.21.15.114:6827/1148348470","heartbeat_back_addr":"172.21.15.114:6831/1148348470","heartbeat_front_addr":"172.21.15.114:6829/1148348470","state":["exists","up"]},{"osd":7,"uuid":"b6d774ee-fe6d-462c-afa4-7ffd5d7343a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6817","nonce":2133497352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6819","nonce":2133497352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6823","nonce":2133497352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2133497352},{"type":"v1","addr":"172.21.15.114:6821","nonce":2133497352}]},"public_addr":"172.21.15.114:6817/2133497352","cluster_addr":"172.21.15.114:6819/2133497352","heartbeat_back_addr":"172.21.15.114:6823/2133497352","heartbeat_front_addr":"172.21.15.114:6821/2133497352","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.237241+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.504107+0000","dead_epoch":127},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.690795+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.197056+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.459497+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.805108+0000","dead_epoch":56},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.484317+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T00:17:44.741853+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.a","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T00:20:57.098 INFO:tasks.ceph:Scrubbing osd.0 2024-09-18T00:20:57.099 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-18T00:20:57.256 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:57.256 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T00:20:57.256 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:57.271 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-18T00:20:57.547 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 0 to deep-scrub 2024-09-18T00:20:57.566 INFO:tasks.ceph:Scrubbing osd.1 2024-09-18T00:20:57.566 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-18T00:20:57.702 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:57.702 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-18T00:20:57.702 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:57.717 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-18T00:20:57.993 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 1 to deep-scrub 2024-09-18T00:20:58.010 INFO:tasks.ceph:Scrubbing osd.2 2024-09-18T00:20:58.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-18T00:20:58.162 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:58.162 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T00:20:58.162 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:58.178 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-18T00:20:58.463 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 2 to deep-scrub 2024-09-18T00:20:58.482 INFO:tasks.ceph:Scrubbing osd.3 2024-09-18T00:20:58.482 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-18T00:20:58.634 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:58.635 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T00:20:58.635 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:58.650 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-18T00:20:58.935 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 3 to deep-scrub 2024-09-18T00:20:58.957 INFO:tasks.ceph:Scrubbing osd.4 2024-09-18T00:20:58.957 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-18T00:20:59.095 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:59.095 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T00:20:59.095 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:59.110 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-18T00:20:59.388 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 4 to deep-scrub 2024-09-18T00:20:59.407 INFO:tasks.ceph:Scrubbing osd.5 2024-09-18T00:20:59.407 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-18T00:20:59.560 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:20:59.560 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T00:20:59.560 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:20:59.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-18T00:20:59.852 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 5 to deep-scrub 2024-09-18T00:20:59.871 INFO:tasks.ceph:Scrubbing osd.6 2024-09-18T00:20:59.871 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-18T00:21:00.021 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:21:00.021 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T00:21:00.021 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:21:00.038 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-18T00:21:00.324 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 6 to deep-scrub 2024-09-18T00:21:00.343 INFO:tasks.ceph:Scrubbing osd.7 2024-09-18T00:21:00.344 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-18T00:21:00.492 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-18T00:21:00.492 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T00:21:00.492 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-18T00:21:00.508 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-18T00:21:00.795 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 7 to deep-scrub 2024-09-18T00:21:00.814 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:21:01.097 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:21:01.097 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-18T00:21:01.114 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-18T00:21:00.349652+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":65,"num_read":96618,"num_read_kb":273533,"num_write":19451,"num_write_kb":184146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1839,"num_bytes_recovered":27009154,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16319,"ondisk_log_size":16319,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":365680,"kb_used_data":14800,"kb_used_omap":35,"kb_used_meta":350684,"kb_avail":754609040,"statfs":{"total":773094113280,"available":772719656960,"internally_reserved":0,"allocated":15155200,"data_stored":8331644,"data_compressed":13209,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":36807,"internal_metadata":359100473},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.890518"},"pg_stats":[{"pgid":"3.1f","version":"135'778","reported_seq":3011,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.096451+0000","last_change":"2024-09-18T00:20:17.573423+0000","last_active":"2024-09-18T00:20:51.096451+0000","last_peered":"2024-09-18T00:20:51.096451+0000","last_clean":"2024-09-18T00:20:51.096451+0000","last_became_active":"2024-09-18T00:19:18.327350+0000","last_became_peered":"2024-09-18T00:19:18.327350+0000","last_unstale":"2024-09-18T00:20:51.096451+0000","last_undegraded":"2024-09-18T00:20:51.096451+0000","last_fullsized":"2024-09-18T00:20:51.096451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":778,"log_dups_size":0,"ondisk_log_size":778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024250489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4587,"num_read_kb":13915,"num_write":1017,"num_write_kb":7086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":2252743,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"133'465","reported_seq":1560,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095747+0000","last_change":"2024-09-18T00:20:37.723738+0000","last_active":"2024-09-18T00:20:51.095747+0000","last_peered":"2024-09-18T00:20:51.095747+0000","last_clean":"2024-09-18T00:20:51.095747+0000","last_became_active":"2024-09-18T00:20:37.722312+0000","last_became_peered":"2024-09-18T00:20:37.722312+0000","last_unstale":"2024-09-18T00:20:51.095747+0000","last_undegraded":"2024-09-18T00:20:51.095747+0000","last_fullsized":"2024-09-18T00:20:51.095747+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057225740000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1405,"num_read_kb":1924,"num_write":538,"num_write_kb":2847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":46741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"135'781","reported_seq":1532,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097665+0000","last_change":"2024-09-18T00:20:48.470232+0000","last_active":"2024-09-18T00:20:51.097665+0000","last_peered":"2024-09-18T00:20:51.097665+0000","last_clean":"2024-09-18T00:20:51.097665+0000","last_became_active":"2024-09-18T00:20:48.465769+0000","last_became_peered":"2024-09-18T00:20:48.465769+0000","last_unstale":"2024-09-18T00:20:51.097665+0000","last_undegraded":"2024-09-18T00:20:51.097665+0000","last_fullsized":"2024-09-18T00:20:51.097665+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:11:07.738987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032750750000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3226,"num_read_kb":4864,"num_write":1184,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":510608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"135'453","reported_seq":1953,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097692+0000","last_change":"2024-09-18T00:20:48.467870+0000","last_active":"2024-09-18T00:20:51.097692+0000","last_peered":"2024-09-18T00:20:51.097692+0000","last_clean":"2024-09-18T00:20:51.097692+0000","last_became_active":"2024-09-18T00:20:48.463565+0000","last_became_peered":"2024-09-18T00:20:48.463565+0000","last_unstale":"2024-09-18T00:20:51.097692+0000","last_undegraded":"2024-09-18T00:20:51.097692+0000","last_fullsized":"2024-09-18T00:20:51.097692+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:07:09.913555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002116522,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3432,"num_read_kb":9919,"num_write":470,"num_write_kb":3833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":72221,"num_keys_recovered":54,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"157'290","reported_seq":1677,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:58.724271+0000","last_change":"2024-09-18T00:20:58.724359+0000","last_active":"2024-09-18T00:20:58.724271+0000","last_peered":"2024-09-18T00:20:58.724271+0000","last_clean":"2024-09-18T00:20:58.724271+0000","last_became_active":"2024-09-18T00:18:05.415810+0000","last_became_peered":"2024-09-18T00:18:05.415810+0000","last_unstale":"2024-09-18T00:20:58.724271+0000","last_undegraded":"2024-09-18T00:20:58.724271+0000","last_fullsized":"2024-09-18T00:20:58.724271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'290","last_scrub_stamp":"2024-09-18T00:20:58.724127+0000","last_deep_scrub":"157'290","last_deep_scrub_stamp":"2024-09-18T00:20:58.724127+0000","last_clean_scrub_stamp":"2024-09-18T00:20:58.724127+0000","objects_scrubbed":1,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:36.463570+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0015232959999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1498,"num_read_kb":2662,"num_write":281,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"140'226","reported_seq":3330,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.098607+0000","last_change":"2024-09-18T00:20:30.686023+0000","last_active":"2024-09-18T00:20:51.098607+0000","last_peered":"2024-09-18T00:20:51.098607+0000","last_clean":"2024-09-18T00:20:51.098607+0000","last_became_active":"2024-09-18T00:20:30.637311+0000","last_became_peered":"2024-09-18T00:20:30.637311+0000","last_unstale":"2024-09-18T00:20:51.098607+0000","last_undegraded":"2024-09-18T00:20:51.098607+0000","last_fullsized":"2024-09-18T00:20:51.098607+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:56:01.105381+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012894545,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":2320,"num_read_kb":4551,"num_write":211,"num_write_kb":2083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":645956,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"140'384","reported_seq":1513,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462500+0000","last_change":"2024-09-18T00:20:30.693807+0000","last_active":"2024-09-18T00:20:49.462500+0000","last_peered":"2024-09-18T00:20:49.462500+0000","last_clean":"2024-09-18T00:20:49.462500+0000","last_became_active":"2024-09-18T00:20:30.640499+0000","last_became_peered":"2024-09-18T00:20:30.640499+0000","last_unstale":"2024-09-18T00:20:49.462500+0000","last_undegraded":"2024-09-18T00:20:49.462500+0000","last_fullsized":"2024-09-18T00:20:49.462500+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018217595999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1131,"num_read_kb":4419,"num_write":389,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":1283165,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"133'327","reported_seq":3719,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095514+0000","last_change":"2024-09-18T00:20:17.694311+0000","last_active":"2024-09-18T00:20:51.095514+0000","last_peered":"2024-09-18T00:20:51.095514+0000","last_clean":"2024-09-18T00:20:51.095514+0000","last_became_active":"2024-09-18T00:18:05.406821+0000","last_became_peered":"2024-09-18T00:18:05.406821+0000","last_unstale":"2024-09-18T00:20:51.095514+0000","last_undegraded":"2024-09-18T00:20:51.095514+0000","last_fullsized":"2024-09-18T00:20:51.095514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022731449999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3612,"num_read_kb":21086,"num_write":298,"num_write_kb":8166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'372","reported_seq":1398,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.096495+0000","last_change":"2024-09-18T00:20:17.573988+0000","last_active":"2024-09-18T00:20:51.096495+0000","last_peered":"2024-09-18T00:20:51.096495+0000","last_clean":"2024-09-18T00:20:51.096495+0000","last_became_active":"2024-09-18T00:19:18.328043+0000","last_became_peered":"2024-09-18T00:19:18.328043+0000","last_unstale":"2024-09-18T00:20:51.096495+0000","last_undegraded":"2024-09-18T00:20:51.096495+0000","last_fullsized":"2024-09-18T00:20:51.096495+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025255270000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1318,"num_read_kb":6224,"num_write":322,"num_write_kb":4815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":491875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"135'690","reported_seq":2488,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097894+0000","last_change":"2024-09-18T00:20:48.471288+0000","last_active":"2024-09-18T00:20:51.097894+0000","last_peered":"2024-09-18T00:20:51.097894+0000","last_clean":"2024-09-18T00:20:51.097894+0000","last_became_active":"2024-09-18T00:20:48.467364+0000","last_became_peered":"2024-09-18T00:20:48.467364+0000","last_unstale":"2024-09-18T00:20:51.097894+0000","last_undegraded":"2024-09-18T00:20:51.097894+0000","last_fullsized":"2024-09-18T00:20:51.097894+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:25.555616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021376099999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4502,"num_read_kb":5989,"num_write":925,"num_write_kb":5689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1385456,"num_keys_recovered":29,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"133'489","reported_seq":1056,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463552+0000","last_change":"2024-09-18T00:20:17.577703+0000","last_active":"2024-09-18T00:20:49.463552+0000","last_peered":"2024-09-18T00:20:49.463552+0000","last_clean":"2024-09-18T00:20:49.463552+0000","last_became_active":"2024-09-18T00:18:16.438164+0000","last_became_peered":"2024-09-18T00:18:16.438164+0000","last_unstale":"2024-09-18T00:20:49.463552+0000","last_undegraded":"2024-09-18T00:20:49.463552+0000","last_fullsized":"2024-09-18T00:20:49.463552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:18:47.201515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014879730000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":1078,"num_read_kb":1346,"num_write":552,"num_write_kb":3993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"134'530","reported_seq":1983,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.098853+0000","last_change":"2024-09-18T00:20:17.576876+0000","last_active":"2024-09-18T00:20:51.098853+0000","last_peered":"2024-09-18T00:20:51.098853+0000","last_clean":"2024-09-18T00:20:51.098853+0000","last_became_active":"2024-09-18T00:18:16.441758+0000","last_became_peered":"2024-09-18T00:18:16.441758+0000","last_unstale":"2024-09-18T00:20:51.098853+0000","last_undegraded":"2024-09-18T00:20:51.098853+0000","last_fullsized":"2024-09-18T00:20:51.098853+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:18:47.366485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050089940000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2844,"num_read_kb":5653,"num_write":555,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":103027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":40,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:50.453884+0000","last_change":"2024-09-18T00:20:34.754460+0000","last_active":"2024-09-18T00:20:50.453884+0000","last_peered":"2024-09-18T00:20:50.453884+0000","last_clean":"2024-09-18T00:20:50.453884+0000","last_became_active":"2024-09-18T00:20:34.753675+0000","last_became_peered":"2024-09-18T00:20:34.753675+0000","last_unstale":"2024-09-18T00:20:50.453884+0000","last_undegraded":"2024-09-18T00:20:50.453884+0000","last_fullsized":"2024-09-18T00:20:50.453884+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:37.203669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089554000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"135'348","reported_seq":2426,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.096393+0000","last_change":"2024-09-18T00:20:30.727046+0000","last_active":"2024-09-18T00:20:51.096393+0000","last_peered":"2024-09-18T00:20:51.096393+0000","last_clean":"2024-09-18T00:20:51.096393+0000","last_became_active":"2024-09-18T00:20:30.648069+0000","last_became_peered":"2024-09-18T00:20:30.648069+0000","last_unstale":"2024-09-18T00:20:51.096393+0000","last_undegraded":"2024-09-18T00:20:51.096393+0000","last_fullsized":"2024-09-18T00:20:51.096393+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052221869999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2016,"num_read_kb":5374,"num_write":329,"num_write_kb":5799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":1635567,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":60,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462570+0000","last_change":"2024-09-18T00:20:34.694081+0000","last_active":"2024-09-18T00:20:49.462570+0000","last_peered":"2024-09-18T00:20:49.462570+0000","last_clean":"2024-09-18T00:20:49.462570+0000","last_became_active":"2024-09-18T00:20:34.693550+0000","last_became_peered":"2024-09-18T00:20:34.693550+0000","last_unstale":"2024-09-18T00:20:49.462570+0000","last_undegraded":"2024-09-18T00:20:49.462570+0000","last_fullsized":"2024-09-18T00:20:49.462570+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"134'665","reported_seq":3139,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095911+0000","last_change":"2024-09-18T00:20:17.695571+0000","last_active":"2024-09-18T00:20:51.095911+0000","last_peered":"2024-09-18T00:20:51.095911+0000","last_clean":"2024-09-18T00:20:51.095911+0000","last_became_active":"2024-09-18T00:18:16.440536+0000","last_became_peered":"2024-09-18T00:18:16.440536+0000","last_unstale":"2024-09-18T00:20:51.095911+0000","last_undegraded":"2024-09-18T00:20:51.095911+0000","last_fullsized":"2024-09-18T00:20:51.095911+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034870019999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":3858,"num_read_kb":13564,"num_write":915,"num_write_kb":9611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":73512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":257,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463801+0000","last_change":"2024-09-18T00:18:16.439964+0000","last_active":"2024-09-18T00:20:49.463801+0000","last_peered":"2024-09-18T00:20:49.463801+0000","last_clean":"2024-09-18T00:20:49.463801+0000","last_became_active":"2024-09-18T00:18:16.437150+0000","last_became_peered":"2024-09-18T00:18:16.437150+0000","last_unstale":"2024-09-18T00:20:49.463801+0000","last_undegraded":"2024-09-18T00:20:49.463801+0000","last_fullsized":"2024-09-18T00:20:49.463801+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:25.772294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"140'491","reported_seq":2309,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462454+0000","last_change":"2024-09-18T00:20:30.710100+0000","last_active":"2024-09-18T00:20:49.462454+0000","last_peered":"2024-09-18T00:20:49.462454+0000","last_clean":"2024-09-18T00:20:49.462454+0000","last_became_active":"2024-09-18T00:20:30.641845+0000","last_became_peered":"2024-09-18T00:20:30.641845+0000","last_unstale":"2024-09-18T00:20:49.462454+0000","last_undegraded":"2024-09-18T00:20:49.462454+0000","last_fullsized":"2024-09-18T00:20:49.462454+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.013533892,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2426,"num_read_kb":4923,"num_write":423,"num_write_kb":4982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":1553378,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462352+0000","last_change":"2024-09-18T00:20:30.651121+0000","last_active":"2024-09-18T00:20:49.462352+0000","last_peered":"2024-09-18T00:20:49.462352+0000","last_clean":"2024-09-18T00:20:49.462352+0000","last_became_active":"2024-09-18T00:20:30.650373+0000","last_became_peered":"2024-09-18T00:20:30.650373+0000","last_unstale":"2024-09-18T00:20:49.462352+0000","last_undegraded":"2024-09-18T00:20:49.462352+0000","last_fullsized":"2024-09-18T00:20:49.462352+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"133'363","reported_seq":2714,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097799+0000","last_change":"2024-09-18T00:20:48.470308+0000","last_active":"2024-09-18T00:20:51.097799+0000","last_peered":"2024-09-18T00:20:51.097799+0000","last_clean":"2024-09-18T00:20:51.097799+0000","last_became_active":"2024-09-18T00:20:48.462691+0000","last_became_peered":"2024-09-18T00:20:48.462691+0000","last_unstale":"2024-09-18T00:20:51.097799+0000","last_undegraded":"2024-09-18T00:20:51.097799+0000","last_fullsized":"2024-09-18T00:20:51.097799+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:08.083732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001889571,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2213,"num_read_kb":12284,"num_write":366,"num_write_kb":5310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":675014,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"135'484","reported_seq":2137,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.098807+0000","last_change":"2024-09-18T00:20:17.573853+0000","last_active":"2024-09-18T00:20:51.098807+0000","last_peered":"2024-09-18T00:20:51.098807+0000","last_clean":"2024-09-18T00:20:51.098807+0000","last_became_active":"2024-09-18T00:18:05.407889+0000","last_became_peered":"2024-09-18T00:18:05.407889+0000","last_unstale":"2024-09-18T00:20:51.098807+0000","last_undegraded":"2024-09-18T00:20:51.098807+0000","last_fullsized":"2024-09-18T00:20:51.098807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:40.520294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002502663,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":1766,"num_read_kb":5407,"num_write":446,"num_write_kb":5379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":203,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.098054+0000","last_change":"2024-09-18T00:20:48.461100+0000","last_active":"2024-09-18T00:20:51.098054+0000","last_peered":"2024-09-18T00:20:51.098054+0000","last_clean":"2024-09-18T00:20:51.098054+0000","last_became_active":"2024-09-18T00:20:48.460277+0000","last_became_peered":"2024-09-18T00:20:48.460277+0000","last_unstale":"2024-09-18T00:20:51.098054+0000","last_undegraded":"2024-09-18T00:20:51.098054+0000","last_fullsized":"2024-09-18T00:20:51.098054+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:01:02.391844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088189300000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"134'369","reported_seq":1076,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.462249+0000","last_change":"2024-09-18T00:20:30.726339+0000","last_active":"2024-09-18T00:20:49.462249+0000","last_peered":"2024-09-18T00:20:49.462249+0000","last_clean":"2024-09-18T00:20:49.462249+0000","last_became_active":"2024-09-18T00:20:30.647777+0000","last_became_peered":"2024-09-18T00:20:30.647777+0000","last_unstale":"2024-09-18T00:20:49.462249+0000","last_undegraded":"2024-09-18T00:20:49.462249+0000","last_fullsized":"2024-09-18T00:20:49.462249+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011927055000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":740,"num_read_kb":1879,"num_write":446,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":1002237,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":298,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:58.646733+0000","last_change":"2024-09-18T00:20:58.646756+0000","last_active":"2024-09-18T00:20:58.646733+0000","last_peered":"2024-09-18T00:20:58.646733+0000","last_clean":"2024-09-18T00:20:58.646733+0000","last_became_active":"2024-09-18T00:17:55.631277+0000","last_became_peered":"2024-09-18T00:17:55.631277+0000","last_unstale":"2024-09-18T00:20:58.646733+0000","last_undegraded":"2024-09-18T00:20:58.646733+0000","last_fullsized":"2024-09-18T00:20:58.646733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:20:58.646650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:20:58.646650+0000","last_clean_scrub_stamp":"2024-09-18T00:20:58.646650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:14:54.697078+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001141204,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"157'504","reported_seq":2631,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:57.739858+0000","last_change":"2024-09-18T00:20:57.739887+0000","last_active":"2024-09-18T00:20:57.739858+0000","last_peered":"2024-09-18T00:20:57.739858+0000","last_clean":"2024-09-18T00:20:57.739858+0000","last_became_active":"2024-09-18T00:18:05.410952+0000","last_became_peered":"2024-09-18T00:18:05.410952+0000","last_unstale":"2024-09-18T00:20:57.739858+0000","last_undegraded":"2024-09-18T00:20:57.739858+0000","last_fullsized":"2024-09-18T00:20:57.739858+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'504","last_scrub_stamp":"2024-09-18T00:20:57.739785+0000","last_deep_scrub":"157'504","last_deep_scrub_stamp":"2024-09-18T00:20:57.739785+0000","last_clean_scrub_stamp":"2024-09-18T00:20:57.739785+0000","objects_scrubbed":2,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:22:07.975671+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0014272110000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3308,"num_read_kb":10125,"num_write":594,"num_write_kb":8641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":276,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463375+0000","last_change":"2024-09-18T00:20:30.657193+0000","last_active":"2024-09-18T00:20:49.463375+0000","last_peered":"2024-09-18T00:20:49.463375+0000","last_clean":"2024-09-18T00:20:49.463375+0000","last_became_active":"2024-09-18T00:20:30.656400+0000","last_became_peered":"2024-09-18T00:20:30.656400+0000","last_unstale":"2024-09-18T00:20:49.463375+0000","last_undegraded":"2024-09-18T00:20:49.463375+0000","last_fullsized":"2024-09-18T00:20:49.463375+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:50:27.874106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"134'493","reported_seq":2494,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095665+0000","last_change":"2024-09-18T00:20:17.697271+0000","last_active":"2024-09-18T00:20:51.095665+0000","last_peered":"2024-09-18T00:20:51.095665+0000","last_clean":"2024-09-18T00:20:51.095665+0000","last_became_active":"2024-09-18T00:19:18.330155+0000","last_became_peered":"2024-09-18T00:19:18.330155+0000","last_unstale":"2024-09-18T00:20:51.095665+0000","last_undegraded":"2024-09-18T00:20:51.095665+0000","last_fullsized":"2024-09-18T00:20:51.095665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005129074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2497,"num_read_kb":7710,"num_write":474,"num_write_kb":3192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":493176,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":122,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097622+0000","last_change":"2024-09-18T00:20:48.459774+0000","last_active":"2024-09-18T00:20:51.097622+0000","last_peered":"2024-09-18T00:20:51.097622+0000","last_clean":"2024-09-18T00:20:51.097622+0000","last_became_active":"2024-09-18T00:20:48.457808+0000","last_became_peered":"2024-09-18T00:20:48.457808+0000","last_unstale":"2024-09-18T00:20:51.097622+0000","last_undegraded":"2024-09-18T00:20:51.097622+0000","last_fullsized":"2024-09-18T00:20:51.097622+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:54.445610+0000","last_clean_scrub_stamp":"2024-09-18T00:17:54.445610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:43:17.325507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":438,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463383+0000","last_change":"2024-09-18T00:18:19.466612+0000","last_active":"2024-09-18T00:20:49.463383+0000","last_peered":"2024-09-18T00:20:49.463383+0000","last_clean":"2024-09-18T00:20:49.463383+0000","last_became_active":"2024-09-18T00:18:19.465557+0000","last_became_peered":"2024-09-18T00:18:19.465557+0000","last_unstale":"2024-09-18T00:20:49.463383+0000","last_undegraded":"2024-09-18T00:20:49.463383+0000","last_fullsized":"2024-09-18T00:20:49.463383+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:17:50.414591+0000","last_clean_scrub_stamp":"2024-09-18T00:17:50.414591+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:08:40.389972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"140'661","reported_seq":2378,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095582+0000","last_change":"2024-09-18T00:20:30.742660+0000","last_active":"2024-09-18T00:20:51.095582+0000","last_peered":"2024-09-18T00:20:51.095582+0000","last_clean":"2024-09-18T00:20:51.095582+0000","last_became_active":"2024-09-18T00:20:30.656699+0000","last_became_peered":"2024-09-18T00:20:30.656699+0000","last_unstale":"2024-09-18T00:20:51.095582+0000","last_undegraded":"2024-09-18T00:20:51.095582+0000","last_fullsized":"2024-09-18T00:20:51.095582+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0070126900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3348,"num_read_kb":6619,"num_write":872,"num_write_kb":5519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":1197799,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"134'383","reported_seq":2114,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.098800+0000","last_change":"2024-09-18T00:20:17.574821+0000","last_active":"2024-09-18T00:20:51.098800+0000","last_peered":"2024-09-18T00:20:51.098800+0000","last_clean":"2024-09-18T00:20:51.098800+0000","last_became_active":"2024-09-18T00:18:05.401097+0000","last_became_peered":"2024-09-18T00:18:05.401097+0000","last_unstale":"2024-09-18T00:20:51.098800+0000","last_undegraded":"2024-09-18T00:20:51.098800+0000","last_fullsized":"2024-09-18T00:20:51.098800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:50:58.016930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003875547,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3069,"num_read_kb":9114,"num_write":378,"num_write_kb":6107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"140'412","reported_seq":2390,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:50.453830+0000","last_change":"2024-09-18T00:20:37.726930+0000","last_active":"2024-09-18T00:20:50.453830+0000","last_peered":"2024-09-18T00:20:50.453830+0000","last_clean":"2024-09-18T00:20:50.453830+0000","last_became_active":"2024-09-18T00:20:37.721802+0000","last_became_peered":"2024-09-18T00:20:37.721802+0000","last_unstale":"2024-09-18T00:20:50.453830+0000","last_undegraded":"2024-09-18T00:20:50.453830+0000","last_fullsized":"2024-09-18T00:20:50.453830+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:37.117595+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.007118643,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":1971,"num_read_kb":7346,"num_write":375,"num_write_kb":8437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":8153977,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"134'425","reported_seq":1165,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.095692+0000","last_change":"2024-09-18T00:20:17.696586+0000","last_active":"2024-09-18T00:20:51.095692+0000","last_peered":"2024-09-18T00:20:51.095692+0000","last_clean":"2024-09-18T00:20:51.095692+0000","last_became_active":"2024-09-18T00:18:05.406977+0000","last_became_peered":"2024-09-18T00:18:05.406977+0000","last_unstale":"2024-09-18T00:20:51.095692+0000","last_undegraded":"2024-09-18T00:20:51.095692+0000","last_fullsized":"2024-09-18T00:20:51.095692+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044405699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1237,"num_read_kb":3121,"num_write":630,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"135'1438","reported_seq":3733,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.097706+0000","last_change":"2024-09-18T00:20:48.467838+0000","last_active":"2024-09-18T00:20:51.097706+0000","last_peered":"2024-09-18T00:20:51.097706+0000","last_clean":"2024-09-18T00:20:51.097706+0000","last_became_active":"2024-09-18T00:20:48.462359+0000","last_became_peered":"2024-09-18T00:20:48.462359+0000","last_unstale":"2024-09-18T00:20:51.097706+0000","last_undegraded":"2024-09-18T00:20:51.097706+0000","last_fullsized":"2024-09-18T00:20:51.097706+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":1438,"log_dups_size":0,"ondisk_log_size":1438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:28:28.471113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032308089999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9631,"num_read_kb":9532,"num_write":2186,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":392734,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"134'780","reported_seq":3314,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463834+0000","last_change":"2024-09-18T00:20:17.579849+0000","last_active":"2024-09-18T00:20:49.463834+0000","last_peered":"2024-09-18T00:20:49.463834+0000","last_clean":"2024-09-18T00:20:49.463834+0000","last_became_active":"2024-09-18T00:19:18.333730+0000","last_became_peered":"2024-09-18T00:19:18.333730+0000","last_unstale":"2024-09-18T00:20:49.463834+0000","last_undegraded":"2024-09-18T00:20:49.463834+0000","last_fullsized":"2024-09-18T00:20:49.463834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:12:14.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029127850000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5845,"num_read_kb":7620,"num_write":1029,"num_write_kb":6233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":697208,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"134'352","reported_seq":1424,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463902+0000","last_change":"2024-09-18T00:20:17.578670+0000","last_active":"2024-09-18T00:20:49.463902+0000","last_peered":"2024-09-18T00:20:49.463902+0000","last_clean":"2024-09-18T00:20:49.463902+0000","last_became_active":"2024-09-18T00:18:16.440223+0000","last_became_peered":"2024-09-18T00:18:16.440223+0000","last_unstale":"2024-09-18T00:20:49.463902+0000","last_undegraded":"2024-09-18T00:20:49.463902+0000","last_fullsized":"2024-09-18T00:20:49.463902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:48:17.972019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002121001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1129,"num_read_kb":6709,"num_write":391,"num_write_kb":4357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":7258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"133'397","reported_seq":3507,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.464104+0000","last_change":"2024-09-18T00:20:17.579351+0000","last_active":"2024-09-18T00:20:49.464104+0000","last_peered":"2024-09-18T00:20:49.464104+0000","last_clean":"2024-09-18T00:20:49.464104+0000","last_became_active":"2024-09-18T00:18:16.441020+0000","last_became_peered":"2024-09-18T00:18:16.441020+0000","last_unstale":"2024-09-18T00:20:49.464104+0000","last_undegraded":"2024-09-18T00:20:49.464104+0000","last_fullsized":"2024-09-18T00:20:49.464104+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:01:45.246626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00266241,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4197,"num_read_kb":11008,"num_write":397,"num_write_kb":9044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"134'378","reported_seq":4811,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:50.453820+0000","last_change":"2024-09-18T00:20:37.726301+0000","last_active":"2024-09-18T00:20:50.453820+0000","last_peered":"2024-09-18T00:20:50.453820+0000","last_clean":"2024-09-18T00:20:50.453820+0000","last_became_active":"2024-09-18T00:20:37.723504+0000","last_became_peered":"2024-09-18T00:20:37.723504+0000","last_unstale":"2024-09-18T00:20:50.453820+0000","last_undegraded":"2024-09-18T00:20:50.453820+0000","last_fullsized":"2024-09-18T00:20:50.453820+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:16:42.705342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001519245,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5107,"num_read_kb":32748,"num_write":405,"num_write_kb":13893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":3037482,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"133'368","reported_seq":4657,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463399+0000","last_change":"2024-09-18T00:20:17.577141+0000","last_active":"2024-09-18T00:20:49.463399+0000","last_peered":"2024-09-18T00:20:49.463399+0000","last_clean":"2024-09-18T00:20:49.463399+0000","last_became_active":"2024-09-18T00:18:16.438286+0000","last_became_peered":"2024-09-18T00:18:16.438286+0000","last_unstale":"2024-09-18T00:20:49.463399+0000","last_undegraded":"2024-09-18T00:20:49.463399+0000","last_fullsized":"2024-09-18T00:20:49.463399+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:09:46.156797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092097700000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4916,"num_read_kb":14028,"num_write":359,"num_write_kb":4057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":70145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"134'355","reported_seq":1979,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:51.096138+0000","last_change":"2024-09-18T00:20:17.572303+0000","last_active":"2024-09-18T00:20:51.096138+0000","last_peered":"2024-09-18T00:20:51.096138+0000","last_clean":"2024-09-18T00:20:51.096138+0000","last_became_active":"2024-09-18T00:18:16.438556+0000","last_became_peered":"2024-09-18T00:18:16.438556+0000","last_unstale":"2024-09-18T00:20:51.096138+0000","last_undegraded":"2024-09-18T00:20:51.096138+0000","last_fullsized":"2024-09-18T00:20:51.096138+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019750979999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1630,"num_read_kb":9063,"num_write":386,"num_write_kb":8736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":47120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"133'748","reported_seq":3136,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:49.463471+0000","last_change":"2024-09-18T00:20:17.579483+0000","last_active":"2024-09-18T00:20:49.463471+0000","last_peered":"2024-09-18T00:20:49.463471+0000","last_clean":"2024-09-18T00:20:49.463471+0000","last_became_active":"2024-09-18T00:18:16.440910+0000","last_became_peered":"2024-09-18T00:18:16.440910+0000","last_unstale":"2024-09-18T00:20:49.463471+0000","last_undegraded":"2024-09-18T00:20:49.463471+0000","last_fullsized":"2024-09-18T00:20:49.463471+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:18:04.505988+0000","last_clean_scrub_stamp":"2024-09-18T00:18:04.505988+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:00:51.845256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028046910000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4645,"num_read_kb":12707,"num_write":1147,"num_write_kb":3134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":65,"num_read":96502,"num_read_kb":273433,"num_write":19290,"num_write_kb":181728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1834,"num_bytes_recovered":25828399,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19623,"internal_metadata":0},"log_size":16199,"ondisk_log_size":16199,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20247,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55276,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316564,"statfs":{"total":96636764160,"available":96580161536,"internally_reserved":0,"allocated":1921024,"data_stored":1484222,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37188,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334652,"statfs":{"total":96636764160,"available":96598683648,"internally_reserved":0,"allocated":1880064,"data_stored":893854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":76,"seq":326417514519,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38212,"kb_used_data":1840,"kb_used_omap":9,"kb_used_meta":36342,"kb_avail":94333628,"statfs":{"total":96636764160,"available":96597635072,"internally_reserved":0,"allocated":1884160,"data_stored":893873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9346,"internal_metadata":37215102},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509482,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55236,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316604,"statfs":{"total":96636764160,"available":96580202496,"internally_reserved":0,"allocated":1884160,"data_stored":893873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509484,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41340,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330500,"statfs":{"total":96636764160,"available":96594432000,"internally_reserved":0,"allocated":1880064,"data_stored":893854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":139,"seq":597000454152,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":1844,"kb_used_omap":17,"kb_used_meta":30190,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":1888256,"data_stored":893892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17900,"internal_metadata":30915092},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73280,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298560,"statfs":{"total":96636764160,"available":96561725440,"internally_reserved":0,"allocated":1880064,"data_stored":893854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1484222,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13445,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6178,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:21:01.119 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.119 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.119 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.119 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.119 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.119 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.119 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.120 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.120 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.120 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.120 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.120 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-18T00:17:54.445610+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=17, tm_sec=54, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.120 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.120 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-18T00:17:54.445610+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=17, tm_sec=54, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.120 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.121 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-18T00:17:54.445610+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=17, tm_sec=54, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.121 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.121 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-18T00:17:54.445610+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=17, tm_sec=54, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.121 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.121 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.121 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-18T00:17:54.445610+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=17, tm_sec=54, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.121 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.121 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-18T00:17:54.445610+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=17, tm_sec=54, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.122 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.122 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-18T00:17:54.445610+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=17, tm_sec=54, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.122 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-18T00:17:50.414591+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=17, tm_sec=50, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.122 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.122 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.122 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.122 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.122 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.122 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.123 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.123 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.123 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.123 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.123 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.123 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-18T00:18:04.505988+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=18, tm_sec=4, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=0, tm_min=20, tm_sec=56, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T00:21:01.123 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-18T00:21:21.124 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T00:21:21.436 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-18T00:21:21.437 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-09-18T00:21:21.455 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-18T00:21:20.355516+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":65,"num_read":96618,"num_read_kb":273533,"num_write":19451,"num_write_kb":184146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1839,"num_bytes_recovered":27009154,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16389,"ondisk_log_size":16389,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":369808,"kb_used_data":14800,"kb_used_omap":35,"kb_used_meta":354844,"kb_avail":754604912,"statfs":{"total":773094113280,"available":772715429888,"internally_reserved":0,"allocated":15155200,"data_stored":8331644,"data_compressed":13209,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":36807,"internal_metadata":363360313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003595"},"pg_stats":[{"pgid":"3.1f","version":"135'778","reported_seq":3019,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:00.691455+0000","last_change":"2024-09-18T00:21:00.691498+0000","last_active":"2024-09-18T00:21:00.691455+0000","last_peered":"2024-09-18T00:21:00.691455+0000","last_clean":"2024-09-18T00:21:00.691455+0000","last_became_active":"2024-09-18T00:19:18.327350+0000","last_became_peered":"2024-09-18T00:19:18.327350+0000","last_unstale":"2024-09-18T00:21:00.691455+0000","last_undegraded":"2024-09-18T00:21:00.691455+0000","last_fullsized":"2024-09-18T00:21:00.691455+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"135'778","last_scrub_stamp":"2024-09-18T00:21:00.691323+0000","last_deep_scrub":"135'778","last_deep_scrub_stamp":"2024-09-18T00:21:00.691323+0000","last_clean_scrub_stamp":"2024-09-18T00:21:00.691323+0000","objects_scrubbed":0,"log_size":778,"log_dups_size":0,"ondisk_log_size":778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:48.749117+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0024250489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4587,"num_read_kb":13915,"num_write":1017,"num_write_kb":7086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":2252743,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"157'469","reported_seq":1572,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:04.522948+0000","last_change":"2024-09-18T00:21:04.523020+0000","last_active":"2024-09-18T00:21:04.522948+0000","last_peered":"2024-09-18T00:21:04.522948+0000","last_clean":"2024-09-18T00:21:04.522948+0000","last_became_active":"2024-09-18T00:20:37.722312+0000","last_became_peered":"2024-09-18T00:20:37.722312+0000","last_unstale":"2024-09-18T00:21:04.522948+0000","last_undegraded":"2024-09-18T00:21:04.522948+0000","last_fullsized":"2024-09-18T00:21:04.522948+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'469","last_scrub_stamp":"2024-09-18T00:21:04.522750+0000","last_deep_scrub":"157'469","last_deep_scrub_stamp":"2024-09-18T00:21:04.522750+0000","last_clean_scrub_stamp":"2024-09-18T00:21:04.522750+0000","objects_scrubbed":4,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:15:30.808938+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0057225740000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1405,"num_read_kb":1924,"num_write":538,"num_write_kb":2847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":46741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"157'783","reported_seq":1543,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:04.132368+0000","last_change":"2024-09-18T00:21:04.132439+0000","last_active":"2024-09-18T00:21:04.132368+0000","last_peered":"2024-09-18T00:21:04.132368+0000","last_clean":"2024-09-18T00:21:04.132368+0000","last_became_active":"2024-09-18T00:20:48.465769+0000","last_became_peered":"2024-09-18T00:20:48.465769+0000","last_unstale":"2024-09-18T00:21:04.132368+0000","last_undegraded":"2024-09-18T00:21:04.132368+0000","last_fullsized":"2024-09-18T00:21:04.132368+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'783","last_scrub_stamp":"2024-09-18T00:21:04.132172+0000","last_deep_scrub":"157'783","last_deep_scrub_stamp":"2024-09-18T00:21:04.132172+0000","last_clean_scrub_stamp":"2024-09-18T00:21:04.132172+0000","objects_scrubbed":2,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:18:04.001822+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0032750750000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3226,"num_read_kb":4864,"num_write":1184,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":510608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"157'454","reported_seq":1963,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:03.141489+0000","last_change":"2024-09-18T00:21:03.141559+0000","last_active":"2024-09-18T00:21:03.141489+0000","last_peered":"2024-09-18T00:21:03.141489+0000","last_clean":"2024-09-18T00:21:03.141489+0000","last_became_active":"2024-09-18T00:20:48.463565+0000","last_became_peered":"2024-09-18T00:20:48.463565+0000","last_unstale":"2024-09-18T00:21:03.141489+0000","last_undegraded":"2024-09-18T00:21:03.141489+0000","last_fullsized":"2024-09-18T00:21:03.141489+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'454","last_scrub_stamp":"2024-09-18T00:21:03.141287+0000","last_deep_scrub":"157'454","last_deep_scrub_stamp":"2024-09-18T00:21:03.141287+0000","last_clean_scrub_stamp":"2024-09-18T00:21:03.141287+0000","objects_scrubbed":1,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:57.277610+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.002116522,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3432,"num_read_kb":9919,"num_write":470,"num_write_kb":3833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":72221,"num_keys_recovered":54,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"157'290","reported_seq":1677,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:58.724271+0000","last_change":"2024-09-18T00:20:58.724359+0000","last_active":"2024-09-18T00:20:58.724271+0000","last_peered":"2024-09-18T00:20:58.724271+0000","last_clean":"2024-09-18T00:20:58.724271+0000","last_became_active":"2024-09-18T00:18:05.415810+0000","last_became_peered":"2024-09-18T00:18:05.415810+0000","last_unstale":"2024-09-18T00:20:58.724271+0000","last_undegraded":"2024-09-18T00:20:58.724271+0000","last_fullsized":"2024-09-18T00:20:58.724271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'290","last_scrub_stamp":"2024-09-18T00:20:58.724127+0000","last_deep_scrub":"157'290","last_deep_scrub_stamp":"2024-09-18T00:20:58.724127+0000","last_clean_scrub_stamp":"2024-09-18T00:20:58.724127+0000","objects_scrubbed":1,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:36.463570+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0015232959999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1498,"num_read_kb":2662,"num_write":281,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"157'230","reported_seq":3343,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:03.230024+0000","last_change":"2024-09-18T00:21:03.230216+0000","last_active":"2024-09-18T00:21:03.230024+0000","last_peered":"2024-09-18T00:21:03.230024+0000","last_clean":"2024-09-18T00:21:03.230024+0000","last_became_active":"2024-09-18T00:20:30.637311+0000","last_became_peered":"2024-09-18T00:20:30.637311+0000","last_unstale":"2024-09-18T00:21:03.230024+0000","last_undegraded":"2024-09-18T00:21:03.230024+0000","last_fullsized":"2024-09-18T00:21:03.230024+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'230","last_scrub_stamp":"2024-09-18T00:21:03.229802+0000","last_deep_scrub":"157'230","last_deep_scrub_stamp":"2024-09-18T00:21:03.229802+0000","last_clean_scrub_stamp":"2024-09-18T00:21:03.229802+0000","objects_scrubbed":4,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:48:21.363624+0000","scrub_duration":7,"objects_trimmed":2,"snaptrim_duration":0.012894545,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":2320,"num_read_kb":4551,"num_write":211,"num_write_kb":2083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":645956,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"157'385","reported_seq":1522,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:59.785144+0000","last_change":"2024-09-18T00:20:59.785268+0000","last_active":"2024-09-18T00:20:59.785144+0000","last_peered":"2024-09-18T00:20:59.785144+0000","last_clean":"2024-09-18T00:20:59.785144+0000","last_became_active":"2024-09-18T00:20:30.640499+0000","last_became_peered":"2024-09-18T00:20:30.640499+0000","last_unstale":"2024-09-18T00:20:59.785144+0000","last_undegraded":"2024-09-18T00:20:59.785144+0000","last_fullsized":"2024-09-18T00:20:59.785144+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'385","last_scrub_stamp":"2024-09-18T00:20:59.784855+0000","last_deep_scrub":"157'385","last_deep_scrub_stamp":"2024-09-18T00:20:59.784855+0000","last_clean_scrub_stamp":"2024-09-18T00:20:59.784855+0000","objects_scrubbed":1,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:24:38.605208+0000","scrub_duration":3,"objects_trimmed":2,"snaptrim_duration":0.018217595999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1131,"num_read_kb":4419,"num_write":389,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":1283165,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"157'330","reported_seq":3730,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:59.490507+0000","last_change":"2024-09-18T00:20:59.490532+0000","last_active":"2024-09-18T00:20:59.490507+0000","last_peered":"2024-09-18T00:20:59.490507+0000","last_clean":"2024-09-18T00:20:59.490507+0000","last_became_active":"2024-09-18T00:18:05.406821+0000","last_became_peered":"2024-09-18T00:18:05.406821+0000","last_unstale":"2024-09-18T00:20:59.490507+0000","last_undegraded":"2024-09-18T00:20:59.490507+0000","last_fullsized":"2024-09-18T00:20:59.490507+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'330","last_scrub_stamp":"2024-09-18T00:20:59.490448+0000","last_deep_scrub":"157'330","last_deep_scrub_stamp":"2024-09-18T00:20:59.490448+0000","last_clean_scrub_stamp":"2024-09-18T00:20:59.490448+0000","objects_scrubbed":3,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:55:30.275385+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0022731449999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3612,"num_read_kb":21086,"num_write":298,"num_write_kb":8166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"157'375","reported_seq":1409,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:01.744874+0000","last_change":"2024-09-18T00:21:01.744941+0000","last_active":"2024-09-18T00:21:01.744874+0000","last_peered":"2024-09-18T00:21:01.744874+0000","last_clean":"2024-09-18T00:21:01.744874+0000","last_became_active":"2024-09-18T00:19:18.328043+0000","last_became_peered":"2024-09-18T00:19:18.328043+0000","last_unstale":"2024-09-18T00:21:01.744874+0000","last_undegraded":"2024-09-18T00:21:01.744874+0000","last_fullsized":"2024-09-18T00:21:01.744874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'375","last_scrub_stamp":"2024-09-18T00:21:01.744678+0000","last_deep_scrub":"157'375","last_deep_scrub_stamp":"2024-09-18T00:21:01.744678+0000","last_clean_scrub_stamp":"2024-09-18T00:21:01.744678+0000","objects_scrubbed":3,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:32:55.411818+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0025255270000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":1318,"num_read_kb":6224,"num_write":322,"num_write_kb":4815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":491875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"157'691","reported_seq":2498,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:06.158547+0000","last_change":"2024-09-18T00:21:06.158615+0000","last_active":"2024-09-18T00:21:06.158547+0000","last_peered":"2024-09-18T00:21:06.158547+0000","last_clean":"2024-09-18T00:21:06.158547+0000","last_became_active":"2024-09-18T00:20:48.467364+0000","last_became_peered":"2024-09-18T00:20:48.467364+0000","last_unstale":"2024-09-18T00:21:06.158547+0000","last_undegraded":"2024-09-18T00:21:06.158547+0000","last_fullsized":"2024-09-18T00:21:06.158547+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'691","last_scrub_stamp":"2024-09-18T00:21:06.158313+0000","last_deep_scrub":"157'691","last_deep_scrub_stamp":"2024-09-18T00:21:06.158313+0000","last_clean_scrub_stamp":"2024-09-18T00:21:06.158313+0000","objects_scrubbed":1,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:22:24.782362+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0021376099999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4502,"num_read_kb":5989,"num_write":925,"num_write_kb":5689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1385456,"num_keys_recovered":29,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"157'491","reported_seq":1067,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:03.527639+0000","last_change":"2024-09-18T00:21:03.527797+0000","last_active":"2024-09-18T00:21:03.527639+0000","last_peered":"2024-09-18T00:21:03.527639+0000","last_clean":"2024-09-18T00:21:03.527639+0000","last_became_active":"2024-09-18T00:18:16.438164+0000","last_became_peered":"2024-09-18T00:18:16.438164+0000","last_unstale":"2024-09-18T00:21:03.527639+0000","last_undegraded":"2024-09-18T00:21:03.527639+0000","last_fullsized":"2024-09-18T00:21:03.527639+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'491","last_scrub_stamp":"2024-09-18T00:21:03.527409+0000","last_deep_scrub":"157'491","last_deep_scrub_stamp":"2024-09-18T00:21:03.527409+0000","last_clean_scrub_stamp":"2024-09-18T00:21:03.527409+0000","objects_scrubbed":2,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:14.014775+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0014879730000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":1078,"num_read_kb":1346,"num_write":552,"num_write_kb":3993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"157'533","reported_seq":1995,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:02.202517+0000","last_change":"2024-09-18T00:21:02.202584+0000","last_active":"2024-09-18T00:21:02.202517+0000","last_peered":"2024-09-18T00:21:02.202517+0000","last_clean":"2024-09-18T00:21:02.202517+0000","last_became_active":"2024-09-18T00:18:16.441758+0000","last_became_peered":"2024-09-18T00:18:16.441758+0000","last_unstale":"2024-09-18T00:21:02.202517+0000","last_undegraded":"2024-09-18T00:21:02.202517+0000","last_fullsized":"2024-09-18T00:21:02.202517+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'533","last_scrub_stamp":"2024-09-18T00:21:02.202300+0000","last_deep_scrub":"157'533","last_deep_scrub_stamp":"2024-09-18T00:21:02.202300+0000","last_clean_scrub_stamp":"2024-09-18T00:21:02.202300+0000","objects_scrubbed":3,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:04:23.309132+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0050089940000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2844,"num_read_kb":5653,"num_write":555,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":103027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:01.029720+0000","last_change":"2024-09-18T00:21:01.029778+0000","last_active":"2024-09-18T00:21:01.029720+0000","last_peered":"2024-09-18T00:21:01.029720+0000","last_clean":"2024-09-18T00:21:01.029720+0000","last_became_active":"2024-09-18T00:20:34.753675+0000","last_became_peered":"2024-09-18T00:20:34.753675+0000","last_unstale":"2024-09-18T00:21:01.029720+0000","last_undegraded":"2024-09-18T00:21:01.029720+0000","last_fullsized":"2024-09-18T00:21:01.029720+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:21:01.029501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:21:01.029501+0000","last_clean_scrub_stamp":"2024-09-18T00:21:01.029501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:35:37.903915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089554000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"157'351","reported_seq":2437,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:02.742919+0000","last_change":"2024-09-18T00:21:02.743108+0000","last_active":"2024-09-18T00:21:02.742919+0000","last_peered":"2024-09-18T00:21:02.742919+0000","last_clean":"2024-09-18T00:21:02.742919+0000","last_became_active":"2024-09-18T00:20:30.648069+0000","last_became_peered":"2024-09-18T00:20:30.648069+0000","last_unstale":"2024-09-18T00:21:02.742919+0000","last_undegraded":"2024-09-18T00:21:02.742919+0000","last_fullsized":"2024-09-18T00:21:02.742919+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'351","last_scrub_stamp":"2024-09-18T00:21:02.742699+0000","last_deep_scrub":"157'351","last_deep_scrub_stamp":"2024-09-18T00:21:02.742699+0000","last_clean_scrub_stamp":"2024-09-18T00:21:02.742699+0000","objects_scrubbed":3,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:57.439163+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0052221869999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2016,"num_read_kb":5374,"num_write":329,"num_write_kb":5799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":1635567,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":68,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:02.749677+0000","last_change":"2024-09-18T00:21:02.749791+0000","last_active":"2024-09-18T00:21:02.749677+0000","last_peered":"2024-09-18T00:21:02.749677+0000","last_clean":"2024-09-18T00:21:02.749677+0000","last_became_active":"2024-09-18T00:20:34.693550+0000","last_became_peered":"2024-09-18T00:20:34.693550+0000","last_unstale":"2024-09-18T00:21:02.749677+0000","last_undegraded":"2024-09-18T00:21:02.749677+0000","last_fullsized":"2024-09-18T00:21:02.749677+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:21:02.749342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:21:02.749342+0000","last_clean_scrub_stamp":"2024-09-18T00:21:02.749342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:03:14.282142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"157'670","reported_seq":3152,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:01.467101+0000","last_change":"2024-09-18T00:21:01.467168+0000","last_active":"2024-09-18T00:21:01.467101+0000","last_peered":"2024-09-18T00:21:01.467101+0000","last_clean":"2024-09-18T00:21:01.467101+0000","last_became_active":"2024-09-18T00:18:16.440536+0000","last_became_peered":"2024-09-18T00:18:16.440536+0000","last_unstale":"2024-09-18T00:21:01.467101+0000","last_undegraded":"2024-09-18T00:21:01.467101+0000","last_fullsized":"2024-09-18T00:21:01.467101+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'670","last_scrub_stamp":"2024-09-18T00:21:01.466919+0000","last_deep_scrub":"157'670","last_deep_scrub_stamp":"2024-09-18T00:21:01.466919+0000","last_clean_scrub_stamp":"2024-09-18T00:21:01.466919+0000","objects_scrubbed":5,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:17.965163+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0034870019999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":3858,"num_read_kb":13564,"num_write":915,"num_write_kb":9611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":73512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":266,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:01.479117+0000","last_change":"2024-09-18T00:21:01.479176+0000","last_active":"2024-09-18T00:21:01.479117+0000","last_peered":"2024-09-18T00:21:01.479117+0000","last_clean":"2024-09-18T00:21:01.479117+0000","last_became_active":"2024-09-18T00:18:16.437150+0000","last_became_peered":"2024-09-18T00:18:16.437150+0000","last_unstale":"2024-09-18T00:21:01.479117+0000","last_undegraded":"2024-09-18T00:21:01.479117+0000","last_fullsized":"2024-09-18T00:21:01.479117+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:21:01.478928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:21:01.478928+0000","last_clean_scrub_stamp":"2024-09-18T00:21:01.478928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:01.286595+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"157'494","reported_seq":2320,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:00.777550+0000","last_change":"2024-09-18T00:21:00.777654+0000","last_active":"2024-09-18T00:21:00.777550+0000","last_peered":"2024-09-18T00:21:00.777550+0000","last_clean":"2024-09-18T00:21:00.777550+0000","last_became_active":"2024-09-18T00:20:30.641845+0000","last_became_peered":"2024-09-18T00:20:30.641845+0000","last_unstale":"2024-09-18T00:21:00.777550+0000","last_undegraded":"2024-09-18T00:21:00.777550+0000","last_fullsized":"2024-09-18T00:21:00.777550+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'494","last_scrub_stamp":"2024-09-18T00:21:00.777366+0000","last_deep_scrub":"157'494","last_deep_scrub_stamp":"2024-09-18T00:21:00.777366+0000","last_clean_scrub_stamp":"2024-09-18T00:21:00.777366+0000","objects_scrubbed":3,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:25:15.605167+0000","scrub_duration":3,"objects_trimmed":4,"snaptrim_duration":0.013533892,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2426,"num_read_kb":4923,"num_write":423,"num_write_kb":4982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":1553378,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":63,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:58.785175+0000","last_change":"2024-09-18T00:20:58.785222+0000","last_active":"2024-09-18T00:20:58.785175+0000","last_peered":"2024-09-18T00:20:58.785175+0000","last_clean":"2024-09-18T00:20:58.785175+0000","last_became_active":"2024-09-18T00:20:30.650373+0000","last_became_peered":"2024-09-18T00:20:30.650373+0000","last_unstale":"2024-09-18T00:20:58.785175+0000","last_undegraded":"2024-09-18T00:20:58.785175+0000","last_fullsized":"2024-09-18T00:20:58.785175+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:20:58.785084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:20:58.785084+0000","last_clean_scrub_stamp":"2024-09-18T00:20:58.785084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:23:27.865156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"133'363","reported_seq":2724,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:05.112850+0000","last_change":"2024-09-18T00:21:05.112850+0000","last_active":"2024-09-18T00:21:05.112850+0000","last_peered":"2024-09-18T00:21:05.112850+0000","last_clean":"2024-09-18T00:21:05.112850+0000","last_became_active":"2024-09-18T00:20:48.462691+0000","last_became_peered":"2024-09-18T00:20:48.462691+0000","last_unstale":"2024-09-18T00:21:05.112850+0000","last_undegraded":"2024-09-18T00:21:05.112850+0000","last_fullsized":"2024-09-18T00:21:05.112850+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"133'363","last_scrub_stamp":"2024-09-18T00:21:05.112592+0000","last_deep_scrub":"133'363","last_deep_scrub_stamp":"2024-09-18T00:21:05.112592+0000","last_clean_scrub_stamp":"2024-09-18T00:21:05.112592+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:37:55.947305+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001889571,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2213,"num_read_kb":12284,"num_write":366,"num_write_kb":5310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":675014,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"157'487","reported_seq":2149,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:01.224899+0000","last_change":"2024-09-18T00:21:01.224965+0000","last_active":"2024-09-18T00:21:01.224899+0000","last_peered":"2024-09-18T00:21:01.224899+0000","last_clean":"2024-09-18T00:21:01.224899+0000","last_became_active":"2024-09-18T00:18:05.407889+0000","last_became_peered":"2024-09-18T00:18:05.407889+0000","last_unstale":"2024-09-18T00:21:01.224899+0000","last_undegraded":"2024-09-18T00:21:01.224899+0000","last_fullsized":"2024-09-18T00:21:01.224899+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'487","last_scrub_stamp":"2024-09-18T00:21:01.224712+0000","last_deep_scrub":"157'487","last_deep_scrub_stamp":"2024-09-18T00:21:01.224712+0000","last_clean_scrub_stamp":"2024-09-18T00:21:01.224712+0000","objects_scrubbed":3,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:54:25.192921+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.002502663,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":1766,"num_read_kb":5407,"num_write":446,"num_write_kb":5379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"157'3","reported_seq":214,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:01.115868+0000","last_change":"2024-09-18T00:21:01.115868+0000","last_active":"2024-09-18T00:21:01.115868+0000","last_peered":"2024-09-18T00:21:01.115868+0000","last_clean":"2024-09-18T00:21:01.115868+0000","last_became_active":"2024-09-18T00:20:48.460277+0000","last_became_peered":"2024-09-18T00:20:48.460277+0000","last_unstale":"2024-09-18T00:21:01.115868+0000","last_undegraded":"2024-09-18T00:21:01.115868+0000","last_fullsized":"2024-09-18T00:21:01.115868+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'3","last_scrub_stamp":"2024-09-18T00:21:01.115596+0000","last_deep_scrub":"157'3","last_deep_scrub_stamp":"2024-09-18T00:21:01.115596+0000","last_clean_scrub_stamp":"2024-09-18T00:21:01.115596+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:06:44.656537+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00088189300000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"157'372","reported_seq":1087,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:01.735757+0000","last_change":"2024-09-18T00:21:01.735836+0000","last_active":"2024-09-18T00:21:01.735757+0000","last_peered":"2024-09-18T00:21:01.735757+0000","last_clean":"2024-09-18T00:21:01.735757+0000","last_became_active":"2024-09-18T00:20:30.647777+0000","last_became_peered":"2024-09-18T00:20:30.647777+0000","last_unstale":"2024-09-18T00:21:01.735757+0000","last_undegraded":"2024-09-18T00:21:01.735757+0000","last_fullsized":"2024-09-18T00:21:01.735757+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'372","last_scrub_stamp":"2024-09-18T00:21:01.735528+0000","last_deep_scrub":"157'372","last_deep_scrub_stamp":"2024-09-18T00:21:01.735528+0000","last_clean_scrub_stamp":"2024-09-18T00:21:01.735528+0000","objects_scrubbed":3,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:40:43.865741+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.011927055000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":740,"num_read_kb":1879,"num_write":446,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":1002237,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":298,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:58.646733+0000","last_change":"2024-09-18T00:20:58.646756+0000","last_active":"2024-09-18T00:20:58.646733+0000","last_peered":"2024-09-18T00:20:58.646733+0000","last_clean":"2024-09-18T00:20:58.646733+0000","last_became_active":"2024-09-18T00:17:55.631277+0000","last_became_peered":"2024-09-18T00:17:55.631277+0000","last_unstale":"2024-09-18T00:20:58.646733+0000","last_undegraded":"2024-09-18T00:20:58.646733+0000","last_fullsized":"2024-09-18T00:20:58.646733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:20:58.646650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:20:58.646650+0000","last_clean_scrub_stamp":"2024-09-18T00:20:58.646650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:14:54.697078+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001141204,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"157'504","reported_seq":2631,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:57.739858+0000","last_change":"2024-09-18T00:20:57.739887+0000","last_active":"2024-09-18T00:20:57.739858+0000","last_peered":"2024-09-18T00:20:57.739858+0000","last_clean":"2024-09-18T00:20:57.739858+0000","last_became_active":"2024-09-18T00:18:05.410952+0000","last_became_peered":"2024-09-18T00:18:05.410952+0000","last_unstale":"2024-09-18T00:20:57.739858+0000","last_undegraded":"2024-09-18T00:20:57.739858+0000","last_fullsized":"2024-09-18T00:20:57.739858+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'504","last_scrub_stamp":"2024-09-18T00:20:57.739785+0000","last_deep_scrub":"157'504","last_deep_scrub_stamp":"2024-09-18T00:20:57.739785+0000","last_clean_scrub_stamp":"2024-09-18T00:20:57.739785+0000","objects_scrubbed":2,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:22:07.975671+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0014272110000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3308,"num_read_kb":10125,"num_write":594,"num_write_kb":8641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":285,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:09.497128+0000","last_change":"2024-09-18T00:21:09.497187+0000","last_active":"2024-09-18T00:21:09.497128+0000","last_peered":"2024-09-18T00:21:09.497128+0000","last_clean":"2024-09-18T00:21:09.497128+0000","last_became_active":"2024-09-18T00:20:30.656400+0000","last_became_peered":"2024-09-18T00:20:30.656400+0000","last_unstale":"2024-09-18T00:21:09.497128+0000","last_undegraded":"2024-09-18T00:21:09.497128+0000","last_fullsized":"2024-09-18T00:21:09.497128+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T00:21:09.496924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T00:21:09.496924+0000","last_clean_scrub_stamp":"2024-09-18T00:21:09.496924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:19:55.238257+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"134'493","reported_seq":2502,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:02.453336+0000","last_change":"2024-09-18T00:21:02.453392+0000","last_active":"2024-09-18T00:21:02.453336+0000","last_peered":"2024-09-18T00:21:02.453336+0000","last_clean":"2024-09-18T00:21:02.453336+0000","last_became_active":"2024-09-18T00:19:18.330155+0000","last_became_peered":"2024-09-18T00:19:18.330155+0000","last_unstale":"2024-09-18T00:21:02.453336+0000","last_undegraded":"2024-09-18T00:21:02.453336+0000","last_fullsized":"2024-09-18T00:21:02.453336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'493","last_scrub_stamp":"2024-09-18T00:21:02.453146+0000","last_deep_scrub":"134'493","last_deep_scrub_stamp":"2024-09-18T00:21:02.453146+0000","last_clean_scrub_stamp":"2024-09-18T00:21:02.453146+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:38:49.769234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005129074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2497,"num_read_kb":7710,"num_write":474,"num_write_kb":3192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":493176,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"157'2","reported_seq":133,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:00.151703+0000","last_change":"2024-09-18T00:21:00.151703+0000","last_active":"2024-09-18T00:21:00.151703+0000","last_peered":"2024-09-18T00:21:00.151703+0000","last_clean":"2024-09-18T00:21:00.151703+0000","last_became_active":"2024-09-18T00:20:48.457808+0000","last_became_peered":"2024-09-18T00:20:48.457808+0000","last_unstale":"2024-09-18T00:21:00.151703+0000","last_undegraded":"2024-09-18T00:21:00.151703+0000","last_fullsized":"2024-09-18T00:21:00.151703+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'2","last_scrub_stamp":"2024-09-18T00:21:00.151447+0000","last_deep_scrub":"157'2","last_deep_scrub_stamp":"2024-09-18T00:21:00.151447+0000","last_clean_scrub_stamp":"2024-09-18T00:21:00.151447+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:48:53.514889+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"157'119","reported_seq":449,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:07.571430+0000","last_change":"2024-09-18T00:21:07.571513+0000","last_active":"2024-09-18T00:21:07.571430+0000","last_peered":"2024-09-18T00:21:07.571430+0000","last_clean":"2024-09-18T00:21:07.571430+0000","last_became_active":"2024-09-18T00:18:19.465557+0000","last_became_peered":"2024-09-18T00:18:19.465557+0000","last_unstale":"2024-09-18T00:21:07.571430+0000","last_undegraded":"2024-09-18T00:21:07.571430+0000","last_fullsized":"2024-09-18T00:21:07.571430+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'119","last_scrub_stamp":"2024-09-18T00:21:07.571232+0000","last_deep_scrub":"157'119","last_deep_scrub_stamp":"2024-09-18T00:21:07.571232+0000","last_clean_scrub_stamp":"2024-09-18T00:21:07.571232+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:03:58.279933+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"157'662","reported_seq":2387,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:03.485163+0000","last_change":"2024-09-18T00:21:03.485316+0000","last_active":"2024-09-18T00:21:03.485163+0000","last_peered":"2024-09-18T00:21:03.485163+0000","last_clean":"2024-09-18T00:21:03.485163+0000","last_became_active":"2024-09-18T00:20:30.656699+0000","last_became_peered":"2024-09-18T00:20:30.656699+0000","last_unstale":"2024-09-18T00:21:03.485163+0000","last_undegraded":"2024-09-18T00:21:03.485163+0000","last_fullsized":"2024-09-18T00:21:03.485163+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'662","last_scrub_stamp":"2024-09-18T00:21:03.484939+0000","last_deep_scrub":"157'662","last_deep_scrub_stamp":"2024-09-18T00:21:03.484939+0000","last_clean_scrub_stamp":"2024-09-18T00:21:03.484939+0000","objects_scrubbed":1,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:20.294438+0000","scrub_duration":3,"objects_trimmed":2,"snaptrim_duration":0.0070126900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3348,"num_read_kb":6619,"num_write":872,"num_write_kb":5519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":1197799,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"157'384","reported_seq":2124,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:00.184409+0000","last_change":"2024-09-18T00:21:00.184468+0000","last_active":"2024-09-18T00:21:00.184409+0000","last_peered":"2024-09-18T00:21:00.184409+0000","last_clean":"2024-09-18T00:21:00.184409+0000","last_became_active":"2024-09-18T00:18:05.401097+0000","last_became_peered":"2024-09-18T00:18:05.401097+0000","last_unstale":"2024-09-18T00:21:00.184409+0000","last_undegraded":"2024-09-18T00:21:00.184409+0000","last_fullsized":"2024-09-18T00:21:00.184409+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'384","last_scrub_stamp":"2024-09-18T00:21:00.184218+0000","last_deep_scrub":"157'384","last_deep_scrub_stamp":"2024-09-18T00:21:00.184218+0000","last_clean_scrub_stamp":"2024-09-18T00:21:00.184218+0000","objects_scrubbed":1,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:27:49.477571+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.003875547,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3069,"num_read_kb":9114,"num_write":378,"num_write_kb":6107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"157'417","reported_seq":2404,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:03.127497+0000","last_change":"2024-09-18T00:21:03.127564+0000","last_active":"2024-09-18T00:21:03.127497+0000","last_peered":"2024-09-18T00:21:03.127497+0000","last_clean":"2024-09-18T00:21:03.127497+0000","last_became_active":"2024-09-18T00:20:37.721802+0000","last_became_peered":"2024-09-18T00:20:37.721802+0000","last_unstale":"2024-09-18T00:21:03.127497+0000","last_undegraded":"2024-09-18T00:21:03.127497+0000","last_fullsized":"2024-09-18T00:21:03.127497+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'417","last_scrub_stamp":"2024-09-18T00:21:03.127307+0000","last_deep_scrub":"157'417","last_deep_scrub_stamp":"2024-09-18T00:21:03.127307+0000","last_clean_scrub_stamp":"2024-09-18T00:21:03.127307+0000","objects_scrubbed":5,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:28.726110+0000","scrub_duration":11,"objects_trimmed":2,"snaptrim_duration":0.007118643,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":1971,"num_read_kb":7346,"num_write":375,"num_write_kb":8437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":8153977,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"134'425","reported_seq":1173,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:00.445390+0000","last_change":"2024-09-18T00:21:00.445450+0000","last_active":"2024-09-18T00:21:00.445390+0000","last_peered":"2024-09-18T00:21:00.445390+0000","last_clean":"2024-09-18T00:21:00.445390+0000","last_became_active":"2024-09-18T00:18:05.406977+0000","last_became_peered":"2024-09-18T00:18:05.406977+0000","last_unstale":"2024-09-18T00:21:00.445390+0000","last_undegraded":"2024-09-18T00:21:00.445390+0000","last_fullsized":"2024-09-18T00:21:00.445390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'425","last_scrub_stamp":"2024-09-18T00:21:00.445173+0000","last_deep_scrub":"134'425","last_deep_scrub_stamp":"2024-09-18T00:21:00.445173+0000","last_clean_scrub_stamp":"2024-09-18T00:21:00.445173+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:36:59.506405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044405699999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1237,"num_read_kb":3121,"num_write":630,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"135'1438","reported_seq":3742,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:02.162422+0000","last_change":"2024-09-18T00:21:02.162553+0000","last_active":"2024-09-18T00:21:02.162422+0000","last_peered":"2024-09-18T00:21:02.162422+0000","last_clean":"2024-09-18T00:21:02.162422+0000","last_became_active":"2024-09-18T00:20:48.462359+0000","last_became_peered":"2024-09-18T00:20:48.462359+0000","last_unstale":"2024-09-18T00:21:02.162422+0000","last_undegraded":"2024-09-18T00:21:02.162422+0000","last_fullsized":"2024-09-18T00:21:02.162422+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"135'1438","last_scrub_stamp":"2024-09-18T00:21:02.162075+0000","last_deep_scrub":"135'1438","last_deep_scrub_stamp":"2024-09-18T00:21:02.162075+0000","last_clean_scrub_stamp":"2024-09-18T00:21:02.162075+0000","objects_scrubbed":0,"log_size":1438,"log_dups_size":0,"ondisk_log_size":1438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:58:32.768455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032308089999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9631,"num_read_kb":9532,"num_write":2186,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":392734,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"134'780","reported_seq":3323,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:08.513148+0000","last_change":"2024-09-18T00:21:08.513207+0000","last_active":"2024-09-18T00:21:08.513148+0000","last_peered":"2024-09-18T00:21:08.513148+0000","last_clean":"2024-09-18T00:21:08.513148+0000","last_became_active":"2024-09-18T00:19:18.333730+0000","last_became_peered":"2024-09-18T00:19:18.333730+0000","last_unstale":"2024-09-18T00:21:08.513148+0000","last_undegraded":"2024-09-18T00:21:08.513148+0000","last_fullsized":"2024-09-18T00:21:08.513148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'780","last_scrub_stamp":"2024-09-18T00:21:08.512947+0000","last_deep_scrub":"134'780","last_deep_scrub_stamp":"2024-09-18T00:21:08.512947+0000","last_clean_scrub_stamp":"2024-09-18T00:21:08.512947+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:06:46.730707+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0029127850000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5845,"num_read_kb":7620,"num_write":1029,"num_write_kb":6233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":697208,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"157'353","reported_seq":1434,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:04.498111+0000","last_change":"2024-09-18T00:21:04.498178+0000","last_active":"2024-09-18T00:21:04.498111+0000","last_peered":"2024-09-18T00:21:04.498111+0000","last_clean":"2024-09-18T00:21:04.498111+0000","last_became_active":"2024-09-18T00:18:16.440223+0000","last_became_peered":"2024-09-18T00:18:16.440223+0000","last_unstale":"2024-09-18T00:21:04.498111+0000","last_undegraded":"2024-09-18T00:21:04.498111+0000","last_fullsized":"2024-09-18T00:21:04.498111+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'353","last_scrub_stamp":"2024-09-18T00:21:04.497933+0000","last_deep_scrub":"157'353","last_deep_scrub_stamp":"2024-09-18T00:21:04.497933+0000","last_clean_scrub_stamp":"2024-09-18T00:21:04.497933+0000","objects_scrubbed":1,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:27:22.926437+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.002121001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":1129,"num_read_kb":6709,"num_write":391,"num_write_kb":4357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":7258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"157'402","reported_seq":3521,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:06.535101+0000","last_change":"2024-09-18T00:21:06.535184+0000","last_active":"2024-09-18T00:21:06.535101+0000","last_peered":"2024-09-18T00:21:06.535101+0000","last_clean":"2024-09-18T00:21:06.535101+0000","last_became_active":"2024-09-18T00:18:16.441020+0000","last_became_peered":"2024-09-18T00:18:16.441020+0000","last_unstale":"2024-09-18T00:21:06.535101+0000","last_undegraded":"2024-09-18T00:21:06.535101+0000","last_fullsized":"2024-09-18T00:21:06.535101+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'402","last_scrub_stamp":"2024-09-18T00:21:06.534892+0000","last_deep_scrub":"157'402","last_deep_scrub_stamp":"2024-09-18T00:21:06.534892+0000","last_clean_scrub_stamp":"2024-09-18T00:21:06.534892+0000","objects_scrubbed":5,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:28:31.468782+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00266241,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4197,"num_read_kb":11008,"num_write":397,"num_write_kb":9044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"157'380","reported_seq":4822,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:02.085352+0000","last_change":"2024-09-18T00:21:02.085572+0000","last_active":"2024-09-18T00:21:02.085352+0000","last_peered":"2024-09-18T00:21:02.085352+0000","last_clean":"2024-09-18T00:21:02.085352+0000","last_became_active":"2024-09-18T00:20:37.723504+0000","last_became_peered":"2024-09-18T00:20:37.723504+0000","last_unstale":"2024-09-18T00:21:02.085352+0000","last_undegraded":"2024-09-18T00:21:02.085352+0000","last_fullsized":"2024-09-18T00:21:02.085352+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'380","last_scrub_stamp":"2024-09-18T00:21:02.085121+0000","last_deep_scrub":"157'380","last_deep_scrub_stamp":"2024-09-18T00:21:02.085121+0000","last_clean_scrub_stamp":"2024-09-18T00:21:02.085121+0000","objects_scrubbed":2,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:03:40.210466+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001519245,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":5107,"num_read_kb":32748,"num_write":405,"num_write_kb":13893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":3037482,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"157'370","reported_seq":4668,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:02.499990+0000","last_change":"2024-09-18T00:21:02.500070+0000","last_active":"2024-09-18T00:21:02.499990+0000","last_peered":"2024-09-18T00:21:02.499990+0000","last_clean":"2024-09-18T00:21:02.499990+0000","last_became_active":"2024-09-18T00:18:16.438286+0000","last_became_peered":"2024-09-18T00:18:16.438286+0000","last_unstale":"2024-09-18T00:21:02.499990+0000","last_undegraded":"2024-09-18T00:21:02.499990+0000","last_fullsized":"2024-09-18T00:21:02.499990+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'370","last_scrub_stamp":"2024-09-18T00:21:02.499783+0000","last_deep_scrub":"157'370","last_deep_scrub_stamp":"2024-09-18T00:21:02.499783+0000","last_clean_scrub_stamp":"2024-09-18T00:21:02.499783+0000","objects_scrubbed":2,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:37:41.929252+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00092097700000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4916,"num_read_kb":14028,"num_write":359,"num_write_kb":4057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":70145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"157'359","reported_seq":1991,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:20:59.703285+0000","last_change":"2024-09-18T00:20:59.703320+0000","last_active":"2024-09-18T00:20:59.703285+0000","last_peered":"2024-09-18T00:20:59.703285+0000","last_clean":"2024-09-18T00:20:59.703285+0000","last_became_active":"2024-09-18T00:18:16.438556+0000","last_became_peered":"2024-09-18T00:18:16.438556+0000","last_unstale":"2024-09-18T00:20:59.703285+0000","last_undegraded":"2024-09-18T00:20:59.703285+0000","last_fullsized":"2024-09-18T00:20:59.703285+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'359","last_scrub_stamp":"2024-09-18T00:20:59.703208+0000","last_deep_scrub":"157'359","last_deep_scrub_stamp":"2024-09-18T00:20:59.703208+0000","last_clean_scrub_stamp":"2024-09-18T00:20:59.703208+0000","objects_scrubbed":4,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:13:19.215648+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0019750979999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1630,"num_read_kb":9063,"num_write":386,"num_write_kb":8736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":47120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"157'752","reported_seq":3149,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-18T00:21:05.542035+0000","last_change":"2024-09-18T00:21:05.542107+0000","last_active":"2024-09-18T00:21:05.542035+0000","last_peered":"2024-09-18T00:21:05.542035+0000","last_clean":"2024-09-18T00:21:05.542035+0000","last_became_active":"2024-09-18T00:18:16.440910+0000","last_became_peered":"2024-09-18T00:18:16.440910+0000","last_unstale":"2024-09-18T00:21:05.542035+0000","last_undegraded":"2024-09-18T00:21:05.542035+0000","last_fullsized":"2024-09-18T00:21:05.542035+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'752","last_scrub_stamp":"2024-09-18T00:21:05.541833+0000","last_deep_scrub":"157'752","last_deep_scrub_stamp":"2024-09-18T00:21:05.541833+0000","last_clean_scrub_stamp":"2024-09-18T00:21:05.541833+0000","objects_scrubbed":4,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:13:38.806413+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0028046910000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":4645,"num_read_kb":12707,"num_write":1147,"num_write_kb":3134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":65,"num_read":96502,"num_read_kb":273433,"num_write":19290,"num_write_kb":181728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1834,"num_bytes_recovered":25828399,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19623,"internal_metadata":0},"log_size":16265,"ondisk_log_size":16265,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20247,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509484,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55268,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316572,"statfs":{"total":96636764160,"available":96580169728,"internally_reserved":0,"allocated":1921024,"data_stored":1484222,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37180,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334660,"statfs":{"total":96636764160,"available":96598691840,"internally_reserved":0,"allocated":1880064,"data_stored":893854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":76,"seq":326417514523,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38208,"kb_used_data":1840,"kb_used_omap":9,"kb_used_meta":36342,"kb_avail":94333632,"statfs":{"total":96636764160,"available":96597639168,"internally_reserved":0,"allocated":1884160,"data_stored":893873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9346,"internal_metadata":37215102},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59392,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312448,"statfs":{"total":96636764160,"available":96575946752,"internally_reserved":0,"allocated":1884160,"data_stored":893873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41340,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330500,"statfs":{"total":96636764160,"available":96594432000,"internally_reserved":0,"allocated":1880064,"data_stored":893854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":139,"seq":597000454156,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":1844,"kb_used_omap":17,"kb_used_meta":30190,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":1888256,"data_stored":893892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17900,"internal_metadata":30915092},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542190,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73276,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298564,"statfs":{"total":96636764160,"available":96561729536,"internally_reserved":0,"allocated":1880064,"data_stored":893854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1484222,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6171,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7038,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13445,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6178,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T00:21:21.458 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-18T00:21:21.792 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-18T00:21:21.793 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-18T00:21:21.793 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-18T00:21:21.793 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T00:21:27.071 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-18T00:21:27.894 INFO:tasks.ceph.osd.0:Stopped 2024-09-18T00:21:27.895 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-18T00:21:27.895 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T00:21:27.951 INFO:tasks.ceph.osd.1:Stopped 2024-09-18T00:21:27.952 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-18T00:21:27.952 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T00:21:28.049 INFO:tasks.ceph.osd.2:Stopped 2024-09-18T00:21:28.050 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-18T00:21:28.050 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T00:21:28.112 INFO:tasks.ceph.osd.3:Stopped 2024-09-18T00:21:28.112 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-18T00:21:28.112 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T00:21:28.183 INFO:tasks.ceph.osd.4:Stopped 2024-09-18T00:21:28.183 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-18T00:21:28.183 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T00:21:28.258 INFO:tasks.ceph.osd.5:Stopped 2024-09-18T00:21:28.259 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-18T00:21:28.259 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T00:21:28.335 INFO:tasks.ceph.osd.6:Stopped 2024-09-18T00:21:28.335 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-18T00:21:28.335 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T00:21:28.432 INFO:tasks.ceph.osd.7:Stopped 2024-09-18T00:21:28.432 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-18T00:21:28.432 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-18T00:21:28.433 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T00:21:28.462 INFO:tasks.ceph.mgr.y:Stopped 2024-09-18T00:21:28.462 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-18T00:21:28.462 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T00:21:28.491 INFO:tasks.ceph.mgr.x:Stopped 2024-09-18T00:21:28.491 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-18T00:21:28.492 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-18T00:21:28.492 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T00:21:28.519 INFO:tasks.ceph.mon.a:Stopped 2024-09-18T00:21:28.519 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-18T00:21:28.519 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T00:21:28.578 INFO:tasks.ceph.mon.c:Stopped 2024-09-18T00:21:28.578 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-18T00:21:28.578 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T00:21:28.602 INFO:tasks.ceph.mon.b:Stopped 2024-09-18T00:21:28.603 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-18T00:21:28.603 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-18T00:21:28.640 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-18T00:21:28.640 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-18T00:21:28.932 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-18T00:21:28.932 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-18T00:21:29.101 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-18T00:21:29.101 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-18T00:21:29.323 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi097.front.sepia.ceph.com 2024-09-18T00:21:29.323 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-18T00:21:29.471 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-18T00:21:29.472 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-18T00:21:29.594 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-18T00:21:29.594 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-18T00:21:29.861 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-18T00:21:29.861 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-18T00:21:30.030 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-18T00:21:30.030 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-18T00:21:30.196 INFO:tasks.ceph:Archiving mon data... 2024-09-18T00:21:30.197 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909638/data/mon.a.tgz 2024-09-18T00:21:30.197 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-09-18T00:21:30.203 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.bmkun6linI 2024-09-18T00:21:30.203 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.bmkun6linI 2024-09-18T00:21:30.341 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.bmkun6linI 2024-09-18T00:21:30.401 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.bmkun6linI is 384KB 2024-09-18T00:21:30.464 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.bmkun6linI 2024-09-18T00:21:30.471 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909638/data/mon.c.tgz 2024-09-18T00:21:30.471 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-09-18T00:21:30.518 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.rQjJ6m2jxa 2024-09-18T00:21:30.518 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.rQjJ6m2jxa 2024-09-18T00:21:30.688 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.rQjJ6m2jxa 2024-09-18T00:21:30.749 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.rQjJ6m2jxa is 521KB 2024-09-18T00:21:30.812 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.rQjJ6m2jxa 2024-09-18T00:21:30.817 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909638/data/mon.b.tgz 2024-09-18T00:21:30.818 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-09-18T00:21:30.823 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.913ZKDozSc 2024-09-18T00:21:30.824 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.913ZKDozSc 2024-09-18T00:21:30.996 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.913ZKDozSc 2024-09-18T00:21:31.057 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.913ZKDozSc is 521KB 2024-09-18T00:21:31.120 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.913ZKDozSc 2024-09-18T00:21:31.126 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-18T00:21:31.126 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T00:21:31.129 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T00:21:31.187 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-18T00:21:31.229 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-18T00:21:31.230 INFO:tasks.ceph:Archiving crash dumps... 2024-09-18T00:21:31.231 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909638/remote/smithi097/crash 2024-09-18T00:21:31.232 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T00:21:31.244 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909638/remote/smithi114/crash 2024-09-18T00:21:31.245 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T00:21:31.285 INFO:tasks.ceph:Compressing logs... 2024-09-18T00:21:31.285 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T00:21:31.291 DEBUG:teuthology.orchestra.run.smithi114:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T00:21:31.313 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38577.log 2024-09-18T00:21:31.314 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33902.log 2024-09-18T00:21:31.314 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38577.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38577.log.gz 2024-09-18T00:21:31.314 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46959.log 2024-09-18T00:21:31.315 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.33902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48311.log 2024-09-18T00:21:31.315 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33902.log.gz 2024-09-18T00:21:31.315 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46959.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46959.log.gz 2024-09-18T00:21:31.315 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38631.log 2024-09-18T00:21:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40308.log 2024-09-18T00:21:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48311.log: /var/log/ceph/ceph-client.admin.38631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30763.log 2024-09-18T00:21:31.317 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48311.log.gz 2024-09-18T00:21:31.317 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38631.log.gz 2024-09-18T00:21:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40308.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40308.log.gz 2024-09-18T00:21:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47260.log 2024-09-18T00:21:31.318 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30763.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-18T00:21:31.318 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30763.log.gz 2024-09-18T00:21:31.318 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47260.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47260.log.gz 2024-09-18T00:21:31.318 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47064.log 2024-09-18T00:21:31.321 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31696.log 2024-09-18T00:21:31.321 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47064.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.47064.log.gz /var/log/ceph/ceph-client.admin.47011.log 2024-09-18T00:21:31.321 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-18T00:21:31.321 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32459.log 2024-09-18T00:21:31.321 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31696.log.gz 2024-09-18T00:21:31.321 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47271.log 2024-09-18T00:21:31.321 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47011.log.gz 2024-09-18T00:21:31.322 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32459.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32459.log.gz 2024-09-18T00:21:31.322 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37363.log 2024-09-18T00:21:31.322 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30558.log 2024-09-18T00:21:31.322 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47271.log.gz 2024-09-18T00:21:31.322 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47791.log 2024-09-18T00:21:31.322 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37363.log.gz 2024-09-18T00:21:31.322 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39418.log 2024-09-18T00:21:31.322 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30558.log.gz 2024-09-18T00:21:31.322 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32151.log 2024-09-18T00:21:31.322 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47791.log.gz 2024-09-18T00:21:31.322 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30766.log 2024-09-18T00:21:31.322 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39418.log.gz 2024-09-18T00:21:31.325 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32151.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-osd.0.log -- replaced with /var/log/ceph/ceph-client.admin.32151.log.gz 2024-09-18T00:21:31.326 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-18T00:21:31.326 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39622.log 2024-09-18T00:21:31.326 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30766.log.gz 2024-09-18T00:21:31.326 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31179.log 2024-09-18T00:21:31.326 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48051.log 2024-09-18T00:21:31.326 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39622.log.gz 2024-09-18T00:21:31.326 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38439.log 2024-09-18T00:21:31.326 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31179.log.gz 2024-09-18T00:21:31.326 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47245.log 2024-09-18T00:21:31.326 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48051.log.gz 2024-09-18T00:21:31.327 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38439.log.gz 2024-09-18T00:21:31.327 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.36774.log 2024-09-18T00:21:31.327 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46647.log 2024-09-18T00:21:31.327 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47245.log.gz 2024-09-18T00:21:31.327 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/objectstore_tool.36774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43956.log 2024-09-18T00:21:31.327 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46647.log.gz 2024-09-18T00:21:31.327 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46751.log 2024-09-18T00:21:31.327 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34885.log 2024-09-18T00:21:31.330 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43956.log: /var/log/ceph/ceph-client.admin.46751.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43956.log.gz 2024-09-18T00:21:31.330 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46751.log.gz 2024-09-18T00:21:31.330 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40878.log 2024-09-18T00:21:31.330 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26299.log 2024-09-18T00:21:31.330 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34885.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34885.log.gz 2024-09-18T00:21:31.330 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40878.log.gz 2024-09-18T00:21:31.330 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47464.log 2024-09-18T00:21:31.330 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38715.log 2024-09-18T00:21:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.26299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26299.log.gz 2024-09-18T00:21:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34913.log 2024-09-18T00:21:31.331 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47464.log.gz 2024-09-18T00:21:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46621.log 2024-09-18T00:21:31.331 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38715.log.gz 2024-09-18T00:21:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.34913.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34913.log.gz 2024-09-18T00:21:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39866.log 2024-09-18T00:21:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.34126.log 2024-09-18T00:21:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46621.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46621.log.gz 2024-09-18T00:21:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39866.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39866.log.gz 2024-09-18T00:21:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32425.log 2024-09-18T00:21:31.331 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/objectstore_tool.34126.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31081.log 2024-09-18T00:21:31.335 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32425.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32425.log.gz 2024-09-18T00:21:31.335 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39791.log 2024-09-18T00:21:31.335 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47625.log 2024-09-18T00:21:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31081.log: /var/log/ceph/ceph-client.admin.39791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31081.log.gz 2024-09-18T00:21:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39791.log.gz 2024-09-18T00:21:31.336 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47251.log 2024-09-18T00:21:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47625.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47625.log.gz 2024-09-18T00:21:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47739.log 2024-09-18T00:21:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38464.log 2024-09-18T00:21:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47251.log: 0.0%/var/log/ceph/ceph-client.admin.47739.log: -- replaced with /var/log/ceph/ceph-client.admin.47251.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40022.log 2024-09-18T00:21:31.336 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% 2024-09-18T00:21:31.336 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47739.log.gz 2024-09-18T00:21:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41822.log 2024-09-18T00:21:31.336 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38464.log.gz 2024-09-18T00:21:31.336 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40022.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40022.log.gz 2024-09-18T00:21:31.337 INFO:teuthology.orchestra.run.smithi097.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.36774.log.gz 2024-09-18T00:21:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46933.log 2024-09-18T00:21:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41822.log.gz 2024-09-18T00:21:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30179.log 2024-09-18T00:21:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42184.log 2024-09-18T00:21:31.337 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46933.log.gz 2024-09-18T00:21:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40610.log 2024-09-18T00:21:31.337 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30179.log.gz 2024-09-18T00:21:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42184.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42184.log.gz 2024-09-18T00:21:31.337 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39483.log 2024-09-18T00:21:31.344 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32044.log 2024-09-18T00:21:31.344 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40610.log.gz 2024-09-18T00:21:31.344 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39483.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39483.log.gz 2024-09-18T00:21:31.344 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47255.log 2024-09-18T00:21:31.344 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42150.log 2024-09-18T00:21:31.344 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32044.log.gz 2024-09-18T00:21:31.344 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47255.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47255.log.gz 2024-09-18T00:21:31.344 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48181.log 2024-09-18T00:21:31.344 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42150.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-18T00:21:31.344 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42150.log.gz 2024-09-18T00:21:31.345 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31173.log 2024-09-18T00:21:31.345 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48181.log.gz 2024-09-18T00:21:31.345 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.31173.log: 0.0%gzip -5 --verbose -- /var/log/ceph/objectstore_tool.42385.log 2024-09-18T00:21:31.345 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31173.log.gz 2024-09-18T00:21:31.345 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38685.log 2024-09-18T00:21:31.345 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/objectstore_tool.42385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46829.log 2024-09-18T00:21:31.345 INFO:teuthology.orchestra.run.smithi097.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.34126.log.gz 2024-09-18T00:21:31.345 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38986.log 2024-09-18T00:21:31.345 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38685.log.gz 2024-09-18T00:21:31.345 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46829.log.gz 2024-09-18T00:21:31.345 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26446.log 2024-09-18T00:21:31.345 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-client.admin.38986.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47472.log 2024-09-18T00:21:31.345 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38986.log.gz 2024-09-18T00:21:31.346 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.26446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26446.log.gz 2024-09-18T00:21:31.346 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30764.log 2024-09-18T00:21:31.346 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37601.log 2024-09-18T00:21:31.346 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47472.log.gz 2024-09-18T00:21:31.346 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30764.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.32491.log -- replaced with /var/log/ceph/ceph-client.admin.30764.log.gz 2024-09-18T00:21:31.346 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-18T00:21:31.346 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.37601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40241.log 2024-09-18T00:21:31.346 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37601.log.gz 2024-09-18T00:21:31.346 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41926.log 2024-09-18T00:21:31.346 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32491.log.gz 2024-09-18T00:21:31.346 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31576.log 2024-09-18T00:21:31.346 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40241.log.gz 2024-09-18T00:21:31.347 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41926.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-18T00:21:31.347 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41926.log.gz 2024-09-18T00:21:31.353 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41953.log 2024-09-18T00:21:31.353 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31576.log.gz 2024-09-18T00:21:31.353 INFO:teuthology.orchestra.run.smithi097.stderr:gzip/var/log/ceph/ceph-mon.a.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31671.log 2024-09-18T00:21:31.353 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39554.log 2024-09-18T00:21:31.353 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41953.log.gz 2024-09-18T00:21:31.353 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31671.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31671.log.gz -5 2024-09-18T00:21:31.353 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.30350.log 2024-09-18T00:21:31.353 INFO:teuthology.orchestra.run.smithi097.stderr: 95.2%/var/log/ceph/ceph-client.admin.39554.log: -- replaced with /var/log/ceph/objectstore_tool.42385.log.gz 2024-09-18T00:21:31.353 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39554.log.gz 2024-09-18T00:21:31.353 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31029.log 2024-09-18T00:21:31.353 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42079.log 2024-09-18T00:21:31.353 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30350.log.gz 2024-09-18T00:21:31.354 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30355.log 2024-09-18T00:21:31.354 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31029.log.gz 2024-09-18T00:21:31.354 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42079.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42079.log.gz -5 2024-09-18T00:21:31.354 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47059.log 2024-09-18T00:21:31.354 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30249.log 2024-09-18T00:21:31.354 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30355.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30355.log.gz 2024-09-18T00:21:31.354 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41744.log 2024-09-18T00:21:31.354 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47059.log.gz 2024-09-18T00:21:31.354 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.30249.log: /var/log/ceph/ceph-client.admin.46595.log 2024-09-18T00:21:31.354 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30249.log.gz 2024-09-18T00:21:31.354 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40882.log 2024-09-18T00:21:31.354 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41744.log.gz 2024-09-18T00:21:31.354 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46595.log.gz 2024-09-18T00:21:31.354 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30353.log 2024-09-18T00:21:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30951.log 2024-09-18T00:21:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40882.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40882.log.gz 2024-09-18T00:21:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30353.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30353.log.gz 2024-09-18T00:21:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48077.log 2024-09-18T00:21:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30951.log.gz 2024-09-18T00:21:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31390.log 2024-09-18T00:21:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40175.log 2024-09-18T00:21:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48077.log.gz 2024-09-18T00:21:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31390.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.44764.log 2024-09-18T00:21:31.355 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31390.log.gz 2024-09-18T00:21:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40048.log 2024-09-18T00:21:31.355 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40175.log.gz 2024-09-18T00:21:31.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/objectstore_tool.44764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31304.log 2024-09-18T00:21:31.356 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46855.log 2024-09-18T00:21:31.356 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40048.log.gz 2024-09-18T00:21:31.356 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46512.log 2024-09-18T00:21:31.356 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31304.log.gz 2024-09-18T00:21:31.364 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26329.log 2024-09-18T00:21:31.364 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26377.log 2024-09-18T00:21:31.364 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.26329.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-18T00:21:31.364 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26329.log.gz 2024-09-18T00:21:31.364 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.26377.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26377.log.gz 2024-09-18T00:21:31.364 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-18T00:21:31.364 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26401.log 2024-09-18T00:21:31.364 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-18T00:21:31.364 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.26401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26449.log 2024-09-18T00:21:31.365 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26401.log.gz 2024-09-18T00:21:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26353.log 2024-09-18T00:21:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.26449.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26449.log.gz 2024-09-18T00:21:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.30647.log 2024-09-18T00:21:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-18T00:21:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.26353.log: /var/log/ceph/objectstore_tool.30647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26353.log.gz 2024-09-18T00:21:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26425.log 2024-09-18T00:21:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T00:21:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.26425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26497.log 2024-09-18T00:21:31.365 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26425.log.gz 2024-09-18T00:21:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.31430.log 2024-09-18T00:21:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T00:21:31.365 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.26497.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26497.log.gz 2024-09-18T00:21:31.366 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/objectstore_tool.31430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26473.log 2024-09-18T00:21:31.366 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph.audit.log: 85.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T00:21:31.371 INFO:teuthology.orchestra.run.smithi114.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T00:21:31.371 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-18T00:21:31.371 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.26473.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26473.log.gz -5 2024-09-18T00:21:31.371 INFO:teuthology.orchestra.run.smithi114.stderr: --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-18T00:21:31.371 INFO:teuthology.orchestra.run.smithi114.stderr: 95.3% -- replaced with /var/log/ceph/objectstore_tool.30647.log.gz/var/log/ceph/ceph-mon.b.log: 2024-09-18T00:21:31.371 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.4.log: 93.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-18T00:21:31.372 INFO:teuthology.orchestra.run.smithi114.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.31430.log.gz 2024-09-18T00:21:31.377 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46855.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46855.log.gz -5 2024-09-18T00:21:31.377 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35892.log 2024-09-18T00:21:31.377 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46512.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46512.log.gz -5 2024-09-18T00:21:31.377 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.30555.log 2024-09-18T00:21:31.378 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31488.log 2024-09-18T00:21:31.378 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35892.log.gz 2024-09-18T00:21:31.378 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30555.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.30555.log.gz /var/log/ceph/ceph-client.admin.46538.log 2024-09-18T00:21:31.378 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-18T00:21:31.378 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31488.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.33309.log 0.0% 2024-09-18T00:21:31.378 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31488.log.gz 2024-09-18T00:21:31.378 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48285.log 2024-09-18T00:21:31.378 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46538.log.gz 2024-09-18T00:21:31.378 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/objectstore_tool.33309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38805.log 2024-09-18T00:21:31.378 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48233.log 2024-09-18T00:21:31.378 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48285.log.gz 2024-09-18T00:21:31.379 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38805.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38805.log.gz 2024-09-18T00:21:31.379 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35824.log 2024-09-18T00:21:31.379 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48259.log 2024-09-18T00:21:31.379 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48233.log: /var/log/ceph/ceph-client.admin.35824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48233.log.gz 2024-09-18T00:21:31.379 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35824.log.gz 2024-09-18T00:21:31.379 INFO:teuthology.orchestra.run.smithi097.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.44764.log.gz 2024-09-18T00:21:31.379 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47273.log 2024-09-18T00:21:31.379 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30557.log 2024-09-18T00:21:31.379 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48259.log: /var/log/ceph/ceph-client.admin.47273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30559.log 2024-09-18T00:21:31.379 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48259.log.gz 2024-09-18T00:21:31.379 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47273.log.gz 2024-09-18T00:21:31.379 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30557.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30557.log.gz 2024-09-18T00:21:31.380 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48103.log 2024-09-18T00:21:31.380 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39457.log 2024-09-18T00:21:31.380 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30559.log: /var/log/ceph/ceph-client.admin.48103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39037.log 2024-09-18T00:21:31.380 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30559.log.gz 2024-09-18T00:21:31.380 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48103.log.gz 2024-09-18T00:21:31.380 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39457.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39457.log.gz 2024-09-18T00:21:31.380 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30547.log 2024-09-18T00:21:31.380 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35725.log 2024-09-18T00:21:31.380 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39037.log: /var/log/ceph/ceph-client.admin.30547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46699.log 2024-09-18T00:21:31.380 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39037.log.gz 2024-09-18T00:21:31.380 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30547.log.gz 2024-09-18T00:21:31.380 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35725.log: 95.3% -- replaced with /var/log/ceph/objectstore_tool.33309.log.gz 2024-09-18T00:21:31.380 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35725.log.gz 2024-09-18T00:21:31.380 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39079.log 2024-09-18T00:21:31.381 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40267.log 2024-09-18T00:21:31.381 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46699.log: /var/log/ceph/ceph-client.admin.39079.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46699.log.gz 2024-09-18T00:21:31.381 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-18T00:21:31.381 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39079.log.gz 2024-09-18T00:21:31.381 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47624.log 2024-09-18T00:21:31.381 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40267.log.gz 2024-09-18T00:21:31.381 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38904.log 2024-09-18T00:21:31.381 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47624.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.47624.log.gz --verbose 2024-09-18T00:21:31.381 INFO:teuthology.orchestra.run.smithi097.stderr: -- /var/log/ceph/ceph-client.admin.31003.log 2024-09-18T00:21:31.381 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38904.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.35947.log 2024-09-18T00:21:31.381 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38904.log.gz 2024-09-18T00:21:31.381 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47921.log 2024-09-18T00:21:31.381 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31003.log.gz 2024-09-18T00:21:31.381 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47049.log 2024-09-18T00:21:31.382 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/objectstore_tool.35947.log: /var/log/ceph/ceph-client.admin.47921.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.43202.log 2024-09-18T00:21:31.382 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47921.log.gz 2024-09-18T00:21:31.382 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47049.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47049.log.gz 2024-09-18T00:21:31.382 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38549.log 2024-09-18T00:21:31.382 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/objectstore_tool.43202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30351.log 2024-09-18T00:21:31.382 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38549.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38549.log.gz 2024-09-18T00:21:31.382 INFO:teuthology.orchestra.run.smithi097.stderr: 95.3% -- replaced with /var/log/ceph/objectstore_tool.35947.log.gz 2024-09-18T00:21:31.382 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30765.log 2024-09-18T00:21:31.382 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30351.log.gz 2024-09-18T00:21:31.382 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30354.log 2024-09-18T00:21:31.382 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39250.log 2024-09-18T00:21:31.382 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30765.log.gz 2024-09-18T00:21:31.382 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30354.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30354.log.gz 2024-09-18T00:21:31.382 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39317.log 2024-09-18T00:21:31.383 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39250.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-18T00:21:31.383 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39250.log.gz 2024-09-18T00:21:31.383 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39317.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39317.log.gz 2024-09-18T00:21:31.383 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39752.log 2024-09-18T00:21:31.383 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31736.log 2024-09-18T00:21:31.383 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39752.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39752.log.gz 2024-09-18T00:21:31.383 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40445.log 2024-09-18T00:21:31.383 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31736.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31736.log.gz 2024-09-18T00:21:31.383 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31986.log 2024-09-18T00:21:31.383 INFO:teuthology.orchestra.run.smithi097.stderr: 95.3%/var/log/ceph/ceph-client.admin.40445.log: -- replaced with /var/log/ceph/objectstore_tool.43202.log.gz 2024-09-18T00:21:31.383 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40445.log.gz 2024-09-18T00:21:31.391 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30767.log 2024-09-18T00:21:31.391 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48337.log 2024-09-18T00:21:31.391 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31986.log.gz 2024-09-18T00:21:31.391 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40356.log 2024-09-18T00:21:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30767.log: /var/log/ceph/ceph-client.admin.48337.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30767.log.gz 2024-09-18T00:21:31.392 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48337.log.gz 2024-09-18T00:21:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31137.log 2024-09-18T00:21:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46803.log 2024-09-18T00:21:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40356.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40356.log.gz 2024-09-18T00:21:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31137.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31137.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47261.log 2024-09-18T00:21:31.392 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-18T00:21:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30743.log 2024-09-18T00:21:31.392 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0%/var/log/ceph/ceph-client.admin.47261.log: -- replaced with /var/log/ceph/ceph-client.admin.46803.log.gz 2024-09-18T00:21:31.392 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47261.log.gz 2024-09-18T00:21:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47057.log 2024-09-18T00:21:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38839.log 2024-09-18T00:21:31.392 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30743.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30743.log.gz 2024-09-18T00:21:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47057.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47057.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30345.log 2024-09-18T00:21:31.393 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-18T00:21:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38839.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38839.log.gz 2024-09-18T00:21:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30977.log 2024-09-18T00:21:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.31169.log 2024-09-18T00:21:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30345.log: /var/log/ceph/ceph-client.admin.30977.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30345.log.gz 2024-09-18T00:21:31.393 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30977.log.gz 2024-09-18T00:21:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40215.log 2024-09-18T00:21:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32155.log 2024-09-18T00:21:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.31169.log: /var/log/ceph/ceph-client.admin.40215.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40215.log.gz 2024-09-18T00:21:31.393 INFO:teuthology.orchestra.run.smithi097.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.0.31169.log.gz 2024-09-18T00:21:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48207.log 2024-09-18T00:21:31.393 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26548.log 2024-09-18T00:21:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32155.log: /var/log/ceph/ceph-client.admin.48207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32155.log.gz 2024-09-18T00:21:31.394 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48207.log.gz 2024-09-18T00:21:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39927.log 2024-09-18T00:21:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31853.log 2024-09-18T00:21:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.26548.log: /var/log/ceph/ceph-client.admin.39927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26548.log.gz 2024-09-18T00:21:31.394 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39927.log.gz 2024-09-18T00:21:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39179.log 2024-09-18T00:21:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42109.log 2024-09-18T00:21:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31853.log: /var/log/ceph/ceph-client.admin.39179.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31853.log.gz 2024-09-18T00:21:31.394 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39179.log.gz 2024-09-18T00:21:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32263.log 2024-09-18T00:21:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32916.log 2024-09-18T00:21:31.394 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42109.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42109.log.gz 2024-09-18T00:21:31.395 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32263.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32263.log.gz 2024-09-18T00:21:31.395 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T00:21:31.395 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32916.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32916.log.gz 2024-09-18T00:21:31.398 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47473.log 2024-09-18T00:21:31.399 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40495.log 2024-09-18T00:21:31.399 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.47473.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47473.log.gz 2024-09-18T00:21:31.399 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32377.log 2024-09-18T00:21:31.399 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40495.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40495.log.gz 2024-09-18T00:21:31.399 INFO:teuthology.orchestra.run.smithi097.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T00:21:31.399 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39669.log 2024-09-18T00:21:31.399 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32377.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32377.log.gz 2024-09-18T00:21:31.399 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29946.log 2024-09-18T00:21:31.399 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42397.log 2024-09-18T00:21:31.399 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39669.log.gz 2024-09-18T00:21:31.399 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.29946.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29946.log.gz 2024-09-18T00:21:31.399 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45646.log 2024-09-18T00:21:31.399 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32373.log 2024-09-18T00:21:31.399 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42397.log: /var/log/ceph/ceph-client.admin.45646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42397.log.gz 2024-09-18T00:21:31.400 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45646.log.gz 2024-09-18T00:21:31.400 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47061.log 2024-09-18T00:21:31.400 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32147.log 2024-09-18T00:21:31.403 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32373.log: /var/log/ceph/ceph-client.admin.47061.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32373.log.gz 2024-09-18T00:21:31.403 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47061.log.gz 2024-09-18T00:21:31.403 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46673.log 2024-09-18T00:21:31.403 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40150.log 2024-09-18T00:21:31.403 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32147.log: /var/log/ceph/ceph-client.admin.46673.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32147.log.gz 2024-09-18T00:21:31.404 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46673.log.gz 2024-09-18T00:21:31.404 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31804.log 2024-09-18T00:21:31.404 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47065.log 2024-09-18T00:21:31.404 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40150.log: /var/log/ceph/ceph-client.admin.31804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40150.log.gz 2024-09-18T00:21:31.404 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31804.log.gz 2024-09-18T00:21:31.404 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30791.log 2024-09-18T00:21:31.404 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30275.log 2024-09-18T00:21:31.404 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47065.log.gz 2024-09-18T00:21:31.404 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30791.log.gz 2024-09-18T00:21:31.404 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42229.log 2024-09-18T00:21:31.404 INFO:teuthology.orchestra.run.smithi097.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-18T00:21:31.404 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30275.log.gz 2024-09-18T00:21:31.405 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31055.log 2024-09-18T00:21:31.407 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42229.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-18T00:21:31.407 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42229.log.gz 2024-09-18T00:21:31.407 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47609.log 2024-09-18T00:21:31.408 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31055.log.gz 2024-09-18T00:21:31.408 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47623.log 2024-09-18T00:21:31.408 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48387.log 2024-09-18T00:21:31.408 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47609.log: /var/log/ceph/ceph-client.admin.47623.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47609.log.gz 2024-09-18T00:21:31.408 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47623.log.gz 2024-09-18T00:21:31.408 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47272.log 2024-09-18T00:21:31.408 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30229.log 2024-09-18T00:21:31.408 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48387.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48387.log.gz 2024-09-18T00:21:31.408 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47272.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36527.log 2024-09-18T00:21:31.408 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-18T00:21:31.408 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30229.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30229.log.gz 2024-09-18T00:21:31.408 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30301.log 2024-09-18T00:21:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47465.log 2024-09-18T00:21:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36527.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36527.log.gz 2024-09-18T00:21:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30301.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30301.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45620.log 2024-09-18T00:21:31.411 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-18T00:21:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47465.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47465.log.gz 2024-09-18T00:21:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30747.log 2024-09-18T00:21:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45184.log 2024-09-18T00:21:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45620.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45620.log.gz 2024-09-18T00:21:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30747.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30747.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32084.log 2024-09-18T00:21:31.411 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-18T00:21:31.413 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45184.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45184.log.gz 2024-09-18T00:21:31.413 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46569.log 2024-09-18T00:21:31.413 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T00:21:31.413 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32084.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32084.log.gz 2024-09-18T00:21:31.413 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46569.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46569.log.gz 2024-09-18T00:21:31.413 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47056.log 2024-09-18T00:21:31.415 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.audit.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T00:21:31.416 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41794.log 2024-09-18T00:21:31.417 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31879.log 2024-09-18T00:21:31.417 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47056.log: /var/log/ceph/ceph-client.admin.41794.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47056.log.gz 2024-09-18T00:21:31.417 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41794.log.gz 2024-09-18T00:21:31.417 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47453.log 2024-09-18T00:21:31.418 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47463.log 2024-09-18T00:21:31.418 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31879.log: /var/log/ceph/ceph-client.admin.47453.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31879.log.gz 2024-09-18T00:21:31.418 INFO:teuthology.orchestra.run.smithi097.stderr: 26.0% -- replaced with /var/log/ceph/ceph-client.admin.47453.log.gz 2024-09-18T00:21:31.418 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39133.log 2024-09-18T00:21:31.419 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48025.log 2024-09-18T00:21:31.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47463.log: /var/log/ceph/ceph-client.admin.39133.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47463.log.gz 2024-09-18T00:21:31.419 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39133.log.gz 2024-09-18T00:21:31.419 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32306.log 2024-09-18T00:21:31.421 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35768.log 2024-09-18T00:21:31.421 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48025.log: /var/log/ceph/ceph-client.admin.32306.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48025.log.gz 2024-09-18T00:21:31.421 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32306.log.gz 2024-09-18T00:21:31.421 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38773.log 2024-09-18T00:21:31.422 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31982.log 2024-09-18T00:21:31.422 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35768.log: /var/log/ceph/ceph-client.admin.38773.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35768.log.gz 2024-09-18T00:21:31.422 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38773.log.gz 2024-09-18T00:21:31.422 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30339.log 2024-09-18T00:21:31.423 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47817.log 2024-09-18T00:21:31.423 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31982.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31982.log.gz 2024-09-18T00:21:31.424 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30339.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39217.log 2024-09-18T00:21:31.424 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-18T00:21:31.424 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47817.log.gz 2024-09-18T00:21:31.424 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47895.log 2024-09-18T00:21:31.424 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47947.log 2024-09-18T00:21:31.425 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39217.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39217.log.gz 2024-09-18T00:21:31.425 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47895.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47895.log.gzgzip 2024-09-18T00:21:31.426 INFO:teuthology.orchestra.run.smithi097.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48129.log 2024-09-18T00:21:31.426 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47947.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47947.log.gz 2024-09-18T00:21:31.426 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47999.log 2024-09-18T00:21:31.426 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47765.log 2024-09-18T00:21:31.427 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48129.log.gz 2024-09-18T00:21:31.427 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47999.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47999.log.gz 2024-09-18T00:21:31.427 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42295.log 2024-09-18T00:21:31.427 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47765.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47765.log.gz 2024-09-18T00:21:31.427 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39931.log 2024-09-18T00:21:31.429 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46725.log 2024-09-18T00:21:31.429 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42295.log: /var/log/ceph/ceph-client.admin.39931.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42295.log.gz 2024-09-18T00:21:31.429 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39931.log.gz 2024-09-18T00:21:31.429 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39897.log 2024-09-18T00:21:31.429 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46777.log 2024-09-18T00:21:31.429 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46725.log: /var/log/ceph/ceph-client.admin.39897.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46725.log.gz 2024-09-18T00:21:31.430 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39897.log.gz 2024-09-18T00:21:31.430 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41901.log 2024-09-18T00:21:31.431 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42291.log 2024-09-18T00:21:31.431 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46777.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46777.log.gz 2024-09-18T00:21:31.431 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41901.log.gz 2024-09-18T00:21:31.431 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38183.log 2024-09-18T00:21:31.431 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42291.log.gz 2024-09-18T00:21:31.431 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32113.log 2024-09-18T00:21:31.433 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31609.log 2024-09-18T00:21:31.433 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38183.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38183.log.gz 2024-09-18T00:21:31.433 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32113.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21504.log 2024-09-18T00:21:31.433 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32113.log.gz 2024-09-18T00:21:31.433 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31609.log.gz 2024-09-18T00:21:31.433 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.32527.log 2024-09-18T00:21:31.434 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47843.log 2024-09-18T00:21:31.434 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.tmp-client.admin.21504.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21504.log.gz 2024-09-18T00:21:31.434 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/objectstore_tool.32527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40559.log 2024-09-18T00:21:31.435 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47843.log.gz 2024-09-18T00:21:31.435 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31107.log 2024-09-18T00:21:31.436 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40559.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40559.log.gz 2024-09-18T00:21:31.436 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40394.log 2024-09-18T00:21:31.437 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31107.log.gz 2024-09-18T00:21:31.437 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40074.log 2024-09-18T00:21:31.438 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40394.log.gz 2024-09-18T00:21:31.438 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41569.log 2024-09-18T00:21:31.439 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.40074.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40074.log.gz 2024-09-18T00:21:31.439 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41638.log 2024-09-18T00:21:31.440 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41569.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41569.log.gz 2024-09-18T00:21:31.440 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32331.log 2024-09-18T00:21:31.441 INFO:teuthology.orchestra.run.smithi097.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.32527.log.gz 2024-09-18T00:21:31.441 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41638.log.gz 2024-09-18T00:21:31.441 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48155.log 2024-09-18T00:21:31.443 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46985.log 2024-09-18T00:21:31.443 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.32331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32331.log.gz 2024-09-18T00:21:31.443 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48155.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30551.log 2024-09-18T00:21:31.443 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-18T00:21:31.443 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46985.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46985.log.gz 2024-09-18T00:21:31.445 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48363.log 2024-09-18T00:21:31.445 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31515.log 2024-09-18T00:21:31.445 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30551.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30551.log.gz 2024-09-18T00:21:31.445 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48363.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39665.log 2024-09-18T00:21:31.445 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-18T00:21:31.445 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31515.log.gz 2024-09-18T00:21:31.446 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39583.log 2024-09-18T00:21:31.447 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47613.log 2024-09-18T00:21:31.447 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39665.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39665.log.gz 2024-09-18T00:21:31.447 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39583.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36531.log 2024-09-18T00:21:31.447 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39583.log.gz 2024-09-18T00:21:31.447 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47613.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47613.log.gz 2024-09-18T00:21:31.448 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.34950.log 2024-09-18T00:21:31.448 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30349.log 2024-09-18T00:21:31.448 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36531.log.gz 2024-09-18T00:21:31.448 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/objectstore_tool.34950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39989.log 2024-09-18T00:21:31.449 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30349.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30349.log.gz 2024-09-18T00:21:31.449 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46881.log 2024-09-18T00:21:31.449 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39989.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39989.log.gz 2024-09-18T00:21:31.450 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30563.log 2024-09-18T00:21:31.450 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46881.log.gz 2024-09-18T00:21:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47973.log 2024-09-18T00:21:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30563.log.gz 2024-09-18T00:21:31.452 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.43992.log 2024-09-18T00:21:31.452 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47973.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47973.log.gz 2024-09-18T00:21:31.453 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42010.log 2024-09-18T00:21:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/objectstore_tool.43992.log: 95.2% -- replaced with /var/log/ceph/objectstore_tool.34950.log.gz 2024-09-18T00:21:31.460 INFO:teuthology.orchestra.run.smithi097.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.43992.log.gz 2024-09-18T00:21:31.461 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31345.log 2024-09-18T00:21:31.462 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41707.log 2024-09-18T00:21:31.462 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42010.log: /var/log/ceph/ceph-client.admin.31345.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42010.log.gz 2024-09-18T00:21:31.462 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31345.log.gz 2024-09-18T00:21:31.462 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39832.log 2024-09-18T00:21:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30533.log 2024-09-18T00:21:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41707.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41707.log.gz 2024-09-18T00:21:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39832.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39832.log.gz 2024-09-18T00:21:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44779.log 2024-09-18T00:21:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30533.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30533.log.gz 2024-09-18T00:21:31.465 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38899.log 2024-09-18T00:21:31.465 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31447.log 2024-09-18T00:21:31.465 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44779.log: /var/log/ceph/ceph-client.admin.38899.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44779.log.gz 2024-09-18T00:21:31.465 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38899.log.gz 2024-09-18T00:21:31.465 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47869.log 2024-09-18T00:21:31.467 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35867.log 2024-09-18T00:21:31.467 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31447.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31447.log.gz 2024-09-18T00:21:31.467 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47869.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31935.log 2024-09-18T00:21:31.467 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47869.log.gz 2024-09-18T00:21:31.467 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.35867.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35867.log.gz 2024-09-18T00:21:31.467 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41612.log 2024-09-18T00:21:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42624.log 2024-09-18T00:21:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.31935.log: /var/log/ceph/ceph-client.admin.41612.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31935.log.gz 2024-09-18T00:21:31.469 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41612.log.gz 2024-09-18T00:21:31.469 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47050.log 2024-09-18T00:21:31.469 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30204.log 2024-09-18T00:21:31.469 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42624.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42624.log.gz 2024-09-18T00:21:31.470 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47050.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47050.log.gz 2024-09-18T00:21:31.471 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39321.log 2024-09-18T00:21:31.471 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.30204.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30204.log.gz 2024-09-18T00:21:31.471 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46907.log 2024-09-18T00:21:31.471 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.37593.log 2024-09-18T00:21:31.471 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.39321.log: /var/log/ceph/ceph-client.admin.46907.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39321.log.gz 2024-09-18T00:21:31.471 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46907.log.gz 2024-09-18T00:21:31.478 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/objectstore_tool.37593.log: 95.3% -- replaced with /var/log/ceph/objectstore_tool.37593.log.gz 2024-09-18T00:21:31.516 INFO:teuthology.orchestra.run.smithi114.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-18T00:21:31.564 INFO:teuthology.orchestra.run.smithi097.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-18T00:21:31.643 INFO:teuthology.orchestra.run.smithi097.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-18T00:21:31.732 INFO:teuthology.orchestra.run.smithi114.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-18T00:21:31.789 INFO:teuthology.orchestra.run.smithi097.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-18T00:21:32.011 INFO:teuthology.orchestra.run.smithi097.stderr: 94.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-18T00:21:32.072 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-18T00:21:32.574 INFO:teuthology.orchestra.run.smithi114.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-18T00:21:33.036 INFO:teuthology.orchestra.run.smithi114.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-18T00:21:33.289 INFO:teuthology.orchestra.run.smithi097.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-18T00:21:33.482 INFO:teuthology.orchestra.run.smithi114.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-18T00:21:33.484 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-18T00:21:33.484 INFO:teuthology.orchestra.run.smithi114.stderr:real 0m2.148s 2024-09-18T00:21:33.484 INFO:teuthology.orchestra.run.smithi114.stderr:user 0m5.334s 2024-09-18T00:21:33.484 INFO:teuthology.orchestra.run.smithi114.stderr:sys 0m0.263s 2024-09-18T00:21:33.897 INFO:teuthology.orchestra.run.smithi097.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-18T00:21:33.899 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-18T00:21:33.899 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m2.605s 2024-09-18T00:21:33.899 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m6.162s 2024-09-18T00:21:33.899 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m0.323s 2024-09-18T00:21:33.899 INFO:tasks.ceph:Archiving logs... 2024-09-18T00:21:33.900 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909638/remote/smithi097/log 2024-09-18T00:21:33.901 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T00:21:34.781 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909638/remote/smithi114/log 2024-09-18T00:21:34.782 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T00:21:35.430 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-18T00:21:35.439 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-18T00:21:35.439 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T00:21:35.441 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T00:21:35.497 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-18T00:21:35.506 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-18T00:21:35.506 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T00:21:35.509 DEBUG:teuthology.orchestra.run.smithi114:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T00:21:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T00:21:35.519 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================== 2024-09-18T00:21:35.519 INFO:teuthology.orchestra.run.smithi097.stdout:*hv01.front.sepi 67.205.162.81 3 u 60 64 77 0.080 -0.410 0.738 2024-09-18T00:21:35.519 INFO:teuthology.orchestra.run.smithi097.stdout:+hv02.front.sepi 96.245.170.99 3 u 53 64 77 0.103 +0.919 0.778 2024-09-18T00:21:35.519 INFO:teuthology.orchestra.run.smithi097.stdout:+hv03.front.sepi 74.6.168.72 3 u 54 64 77 0.078 -2.258 0.725 2024-09-18T00:21:35.519 INFO:teuthology.orchestra.run.smithi097.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:21:35.521 INFO:teuthology.orchestra.run.smithi114.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T00:21:35.521 INFO:teuthology.orchestra.run.smithi114.stdout:============================================================================== 2024-09-18T00:21:35.521 INFO:teuthology.orchestra.run.smithi114.stdout:*hv01.front.sepi 67.205.162.81 3 u 57 64 77 0.124 -1.037 1.117 2024-09-18T00:21:35.521 INFO:teuthology.orchestra.run.smithi114.stdout:+hv02.front.sepi 96.245.170.99 3 u 57 64 77 0.095 +0.229 1.158 2024-09-18T00:21:35.521 INFO:teuthology.orchestra.run.smithi114.stdout:+hv03.front.sepi 74.6.168.72 3 u 54 64 77 0.123 -2.940 1.473 2024-09-18T00:21:35.521 INFO:teuthology.orchestra.run.smithi114.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-18T00:21:35.521 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-18T00:21:35.531 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-18T00:21:35.532 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-18T00:21:35.566 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-18T00:21:35.599 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-18T00:21:35.633 INFO:teuthology.task.internal:Duration was 841.340785 seconds 2024-09-18T00:21:35.633 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-18T00:21:35.666 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-18T00:21:35.667 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T00:21:35.669 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T00:21:35.709 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-18T00:21:35.709 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-09-18T00:21:35.709 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T00:21:35.759 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi114.front.sepia.ceph.com 2024-09-18T00:21:35.759 DEBUG:teuthology.orchestra.run.smithi114:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T00:21:35.769 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-18T00:21:35.769 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T00:21:35.803 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T00:21:35.837 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-18T00:21:35.837 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T00:21:35.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T00:21:35.948 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-18T00:21:35.958 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-18T00:21:35.958 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T00:21:35.970 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T00:21:35.981 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-18T00:21:36.000 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T00:21:36.016 DEBUG:teuthology.orchestra.run.smithi114:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T00:21:36.024 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-09-18T00:21:36.036 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = core 2024-09-18T00:21:36.081 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T00:21:36.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:21:36.089 DEBUG:teuthology.orchestra.run.smithi114:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T00:21:36.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:21:36.125 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-18T00:21:36.135 INFO:teuthology.task.internal:Transferring archived files... 2024-09-18T00:21:36.136 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909638/remote/smithi097 2024-09-18T00:21:36.136 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T00:21:36.162 DEBUG:teuthology.misc:Transferring archived files from smithi114:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909638/remote/smithi114 2024-09-18T00:21:36.163 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T00:21:36.190 INFO:teuthology.task.internal:Removing archive directory... 2024-09-18T00:21:36.190 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T00:21:36.203 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T00:21:36.234 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-18T00:21:36.244 INFO:teuthology.task.internal:Not uploading archives. 2024-09-18T00:21:36.245 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-18T00:21:36.266 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-18T00:21:36.267 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T00:21:36.269 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T00:21:36.273 INFO:teuthology.orchestra.run.smithi097.stdout: 658670 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 00:21 /home/ubuntu/cephtest 2024-09-18T00:21:36.282 INFO:teuthology.orchestra.run.smithi114.stdout: 658637 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 00:21 /home/ubuntu/cephtest 2024-09-18T00:21:36.283 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-18T00:21:36.301 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-18T00:21:36.370 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} duration: 841.3407845497131 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-18T00:21:36.370 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T00:21:36.454 INFO:teuthology.run:pass